Writing CHDK to the internal flash

obi

27

Writing CHDK to the internal flash

« on: 02 / December / 2023, 13:59:12 »

Advertisements

!Warning! Please don't use the instructions (if any) in this post without knowing what you are doing. CHDK is supposed to be loaded into your camera's memory upon bootup and is normally a non-permanent facility that enhances the capabilities of your camera. This thread is created to see the possibility of carefully writing CHDK permanently on the camera's internal flash. You have been warned! Only try if you are ready to brick your camera.

Hi guys,

I have a very old and scratched SX230HS that I would like to write CHDK permanently to. I normally use this Camera for the Intervalometer script and have an EZ Share WiFi SD card in it which lacks the write protect button. Without mugging with the SD card, I want to be able to boot the Camera automatically into CHDK and was thinking to try to write CHDK directly to the internal flash. I would like to find a software solution rather than a some hardware hack on the Camera or SD card.

I am looking after a procedure to write CHDK permanently on the camera's internal flash. I am not a programmer but have good experience with Linux and software. I tried searching on the forum without any success. I had short DM with a senior member where I was told that it is possible but might not be so easy.

Anyone here who has tried something like that on some camera?

Cheers

Logged

reyalp

14082

Re: Writing CHDK to the internal flash

« Reply #1 on: 03 / December / 2023, 03:30:03 »

As I mentioned in PM, loading CHDK from internal flash would take a non-trivial amount of programming and reverse engineering work. If this is purely about running from a card without a lock switch, I would say it's a very risky and labor intensive way to do it and would be far from my first choice.

Given the limited applicability and high risk of bricking, it's not something I'd personally put much time into, but OTOH some might see it as a fun challenge for its own sake and that's certainly a valid reason.

So what would it take?

First, you have to identify a safe, ~150-200 KB of currently unused flash. The Canon ROM isn't a filesystem, it's just a chunk of addressable memory. Firmware dumps generally have some pretty long runs of 0xFF so you could probably just YOLO it into the middle of one of those, but you might get a nasty surprise if it were actually used. Actually coming up with convincing evidence a particular chunk is never used would be more challenging.

But you can't just plop an existing CHDK binary in ROM and jump to from the boot code. The normal CHDK boot process works like this
1) The firmware starts, and boots the OS enough to do things like SD card access
2) If the card is locked and other conditions are met, it looks for DISKBOOT.BIN in the root of the SD card
3) If found, it loads DISKBOOT.BIN, decodes it, copies it to 0x1900 (on digic 5 and earlier) and jumps there. 0x1900 normally contains critical parts of the Canon OS, so at that point, the Canon OS is trashed and no longer running. Since the goal of CHDK is to run along side the Canon OS instead of replacing it, additional steps are required.
4) In a CHDK DISKBOOT, the code at the start is called loader, and is compiled to run at 0x1900. This is a small piece of code which copies the rest of the binary (which is data, as a far as the loader is concerned) to a predetermined, platform and build option specific safe place in RAM, and then jumps to it. This copied code is the main CHDK executable, which is compiled to run at the "safe" address, and includes both code (text section) and initialized data (data section). The code also expects to have space for uninitialized variables (bss) at a particular location in the safe space.
5) The start of the copied code is the platform boot code, which then duplicates the normal Canon OS startup, up to (and beyond) the point where DISKBOOT would normally be loaded, modifying the process to ensure the Canon code doesn't use the memory where CHDK was loaded to and doesn't try to start DISKBOOT again, modified tasks are started etc.

So if you want CHDK in ROM, you need to build an image that is either set up to execute directly from the address in ROM, while still having data and bss in RAM, or modify the loader so that it does the typical copy, but copies from ROM instead. The second option would likely be simpler (though inefficient), and you'd need to make sure the loader itself doesn't write to its own data or bss (offhand, I don't think it does).

You also need to modify the Canon boot process somewhere. If you successfully configured CHDK to run from ROM it might be tempting immediately in the the ports boot.c, but this would risk unexpected side effects, because CHDK normally only runs after the Canon OS has booted and initialized some hardware once. A simpler and safer place to do it would be around where DISKBOOT.BIN is loaded, but you'd likely want to retain DISKBOOT capability to provide more chance of recovery if the ROM mod goes wrong.

Note that even if you make DISKBOOT load or run from ROM, without significant additional modifications, modules, scripts and CFG files would all still be expected to be on the card, so you wouldn't really have a standalone CHDK, you'd still need a compatible build installed on the card.

This brings me to the other thing I suggested in PM: Rather making a custom build to load from ROM, why not just modify the Canon DISKBOOT logic to skip the lock check? The Canon code typically calls a function to check some MMIO for the lock and has a conditional branch based on the return value, so just changing the conditional to a NOP (or unconditional branch) should have the result of causing DISKBOOT to be loaded regardless of the lock if present, and normal boot to proceed (with a very slight delay) if not.

Logged

Don't forget what the H stands for.

obi

27

Re: Writing CHDK to the internal flash

« Reply #2 on: 04 / December / 2023, 02:04:19 »

Thank you very much for your detailed explanation. It would indeed be easier to somehow shot-circuit the DISKBOOT logic to ignore the lock on the SD card. From what I understood so far in this case is that I would need to somehow modify the original PRIMARY.BIN dump to achieve that or I got it wrong?

Logged

obi

27

Re: Writing CHDK to the internal flash

« Reply #3 on: 04 / December / 2023, 09:41:26 »

... I think I understood a bit. So I will have the load the PRIMARY.BIN in Ghidra and try my luck there. I did try and saw lots of stuff in there.

To get into things I would like to start with something simpler that is already kind of identified. What I would like to test is to disable formatting the SD card when CHDK is loaded (no matter if in ALT mode or not). Some months ago I formatted my card while attempting to delete photos/videos. That wiped my CHDK and the scripts. I thought it would be good to somehow disable format option with a small warning that formatting has been disabled. I curiously grepped and found some already identified addresses:

~/chdk/trunk/platform/a1100/sub/100c$ grep -iR format
stubs_entry.S:// EXMEM_FORMATTER 2
funcs_by_name.csv:0xffd10f30,FmtMenu_ExecuteQuickFormat_FW
funcs_by_name.csv:0xffca419c,handle_PTP_OC_FormatStore
funcs_by_name.csv:0xffddf328,task_FORMAT_TASK_NAME
funcs_by_name.csv:0xffddf224,task_FORMAT_TASK_PROGRESS
grep: PRIMARY.BIN: binary file matches
funcs_by_address.csv:0xffca419c,handle_PTP_OC_FormatStore
funcs_by_address.csv:0xffd10f30,FmtMenu_ExecuteQuickFormat_FW
funcs_by_address.csv:0xffddf224,task_FORMAT_TASK_PROGRESS
funcs_by_address.csv:0xffddf328,task_FORMAT_TASK_NAME

I am using your camera model as it is supported by the QEMU emulator. To start with, I would like to try to disable FmtMenu_ExecuteQuickFormat_FW (I don't see any convincing function name for the full format though). How can I disable it? Will I have to then save the PRIMARY.BIN, put it in the sub dir, re-build CHDK, and install it on my CAM to test it? Ideally I would like to test it on the emulator first.

Moreover, I thought if ever I am able to identify the code to disable the SD card's lock check, how will I be able to test it on the emulator? Maybe some QEMU setting like read-only=on on the QEMU need to be translated to a ro SD card or this can only be tested directly on the cam?

Logged

reyalp

14082

Re: Writing CHDK to the internal flash

« Reply #4 on: 04 / December / 2023, 15:35:23 »

Quote from: obi on 04 / December / 2023, 09:41:26

To get into things I would like to start with something simpler that is already kind of identified. What I would like to test is to disable formatting the SD card when CHDK is loaded (no matter if in ALT mode or not).

This is definitely not simpler.

Quote

I curiously grepped and found some already identified addresses:

The things in funcs_by_*.csv are addresses of functions identified by the CHDK sig finder. The names often come from the Canon firmware, but they don't really tell you what they do or if / how they are used in normal operation. To understand how it's used in the firmware, you need to look at the decompile / disassembly in Ghidra and actually understand how it's used.

The functions ending in _FW are eventprocs, which are generally part of Canon's development or service functions and not used in normal operation. So something like FmtMenu_ExecuteQuickFormat_FW will point you toward the parts of the firmware related to formatting, but disabling it likely won't disable formatting in the normal UI (there are a few exceptions where the firmware calls eventprocs in normal operation).

Quote

Will I have to then save the PRIMARY.BIN, put it in the sub dir, re-build CHDK, and install it on my CAM to test it? Ideally I would like to test it on the emulator first.

No. In the CHDK source, PRIMARY.BIN source is only used to find things in the firmware.

To modify the Canon firmware you would need to identify the address of the instruction(s) you want to change, and modify the the actual ROM on the camera, using Canon functions like WriteToROM (it's actually more complicated, because while individual bytes can be *written*, only larger blocks can be *erased*, meaning "write" can only change 1s to 0s).

To modify the ROM in the qemu, you could either either modify the .bin file containing the firmware, or use gdb to change the desired address. In this qemu implementation, the "ROM" can actually be written with normal memory access, so in a gdb script you can do something the bit at the bottom of https://github.com/reticulatedpines/qemu-eos/blob/qemu-eos-v4.2.1/magiclantern/cam_config/A1100/debugmsg.gdb

Code: [Select]

b *0xFFC090B4
commands
 set MEM(0x2234) = 0x200000
 c
end

to automatically change ROM at startup. Addresses above are for something different, obviously.

Quote

Moreover, I thought if ever I am able to identify the code to disable the SD card's lock check, how will I be able to test it on the emulator? Maybe some QEMU setting like read-only=on on the QEMU need to be translated to a ro SD card or this can only be tested directly on the cam?

As noted in an earlier post, the A1100 port in QEMU is currently hard coded to return that the SD is in the locked state. So you'd want to change that to unlocked. The status on that cam is in 0xC0223024, so you'd want to change qemu hw/eos/eos.c eos_handle_gpio to return 0 for that address for A1100. (my branch on https://github.com/reyalpchdk/qemu-eos/tree/powershot-bootflag allows this to be set from the QEMU command line, but IIRC names_are_hard wasn't thrilled with the implementation and I haven't got around to doing a pull request)

FWIW, it turns out that srsa_4c did something very similar to what you want https://chdk.setepontos.com/index.php?topic=13473.0 (modifying the BOOTDISK string check rather than the lock check, but the principle should be exactly the same)

See also https://chdk.setepontos.com/index.php?topic=13893.msg141739#msg141739

Do heed the warnings, even if you don't care much about the camera, I'd really suggest against trying to modify ROM without being confident you know exactly what you want to change and what each step does to accomplish it.

Logged

Don't forget what the H stands for.

obi

27

Re: Writing CHDK to the internal flash

« Reply #5 on: 05 / December / 2023, 16:25:19 »

Thanks for the reply.

How did you figure out that 0xC0223024 is the lock status for A1100? My Ghidra is showing ?? on that address (I have lots of bookmarks with errors anyhow).

I could not figure out how to use your QEMU option while using your branch to make the SD card lock. However, as you described, I was able to add the following code and that makes the card unlocked. Works on the emulator then.

Code: [Select]

...
        case 0x3024:
            msg = "A1100: SD card lock settting";
            // https://chdk.setepontos.com/index.php?topic=14826.msg150122#msg150122
            // echo $(printf "0x%X\n" $((0xC0223024 & 0xFFFF)))
            ret = 0;
            break;
...

One thing I observed is that when I press backspace (to correct a command for example) in the QEMU serial console, it gets stuck and I have to restart QEMU. Do you also have the same problem? I am not using Putty as I am on Linux directly.

Interesting, I see that waterwingz was exactly looking for what I am trying to do but I am not sure how come on the link

https://chdk.setepontos.com/index.php?topic=13473.msg137528#msg137528

the excerpt from srsa_4c has nice comments while I still can't find out what address I need. I followed your document on Ghidra a couple of times already on freshly created projects. Any tips here would be greatly appreciated.

So if we had to do it for srsa_4c's cam, how would the GDB script look like?

Also a question regarding the boot process. In your A1100 machine, I was copying the firmware dump to the DISKBOOT.BIN file in a hope that the emulator will get stuck in a boot loop but it did not. What is preventing that?

Logged

reyalp

14082

Re: Writing CHDK to the internal flash

« Reply #6 on: 05 / December / 2023, 19:50:30 »

Quote from: obi on 05 / December / 2023, 16:25:19

Thanks for the reply.

How did you figure out that 0xC0223024 is the lock status for A1100?

Searching for the string "diskboot" finds function FUN_ffc18798, which references strings like StartDiskBoot and DISKBOOT.BIN.

The caller of that function is FUN_ffc18754, which checks several return values and calls FUN_ffc18798 or not based on the results. Two of those checks are calls to FUN_ffc30bd0 with either 0 or 1, which accesses either c022301c or c0223024 based on the result. Testing confirms that *24 is the SD lock (IIRC the other is SD card present)

Another way you could find this, in platform/a1100/sub/100.c boot.c, you'll find

Code: [Select]

              //   "BL      sub_FFC18754\n"   // //start diskboot.bin, //StartDiskboot --> removed

commenting out the call to this function to avoid a boot loop in CHDK.

Note you should be able to use the same methods to find the equivalent in SX230.

Quote

My Ghidra is showing ?? on that address

It should, since that's an MMIO address whose value is unknown to Ghidra.

Quote

(I have lots of bookmarks with errors anyhow).

It's normal to have some error bookmarks, but if you followed https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra there shouldn't be too many. Note there are a lot of "analysis" bookmarks which aren't errors, you can use the gear icon on that window to show just errors and warnings.

Quote

I could not figure out how to use your QEMU option while using your branch to make the SD card lock.

The command line syntax is quite obscure, it's like

Code: [Select]

-M A1100,firmware=boot=0

Quote

Code: [Select]
... case 0x3024: msg = "A1100: SD card lock settting"; // https://chdk.setepontos.com/index.php?topic=14826.msg150122#msg150122 // echo $(printf "0x%X\n" $((0xC0223024 & 0xFFFF))) ret = 0; break; ...

Nice work, you're a coder now

Quote

One thing I observed is that when I press backspace (to correct a command for example) in the QEMU serial console, it gets stuck and I have to restart QEMU. Do you also have the same problem? I am not using Putty as I am on Linux directly.

Yes, I get that too. It's quite annoying, but not so annoying I spent much time looking for a solution, maybe some qemu serial setting can avoid it. The GUI serial port interface doesn't do that, but has a worse problem (https://github.com/reticulatedpines/qemu-eos/issues/6)

Quote

So if we had to do it for srsa_4c's cam, how would the GDB script look like?

The b address is a breakpoint, that is, where you want execution to stop to make your change. Since you want to permanently (for the duration of the QEMU session) change "ROM" this could at the very start (ffc00000 on a1100).

The MEM part is the part you want to change. In srca_4c's post, the SD lock check is at FFC2ACAC. The value you want to set is 0, which happens to be the arm instruction andeq r0,r0

So a script could look like

Code: [Select]

b *0xFFC00000
commands
 set MEM(0xFFC2ACAC) = 0x0
 c
end

A1100 would be the same, except the mem address would be ffc18784. From a quick test, this seems to work, booting CHDK when the card is unlocked.

Note you can also change "ROM" in qemu code, see the various patch_* functions called from eos_init_common

Quote

Also a question regarding the boot process. In your A1100 machine, I was copying the firmware dump to the DISKBOOT.BIN file in a hope that the emulator will get stuck in a boot loop but it did not. What is preventing that?

I'm not sure what you mean? The firmware dump and diskboot.bin are entirely different things. If the camera tried to boot a firmware dump, it should crash immediately because it's not encoded with dancing bits, but even if it were, the firmware is compiled to run at ffc00000, not in RAM.
If it's just completely ignoring your diskboot, your probably haven't successfully made the SD image bootable, (or maybe failing because the file is too large?). Of course, without hacking FUN_ffc18754 it will also ignore it if it thinks the SD is unlocked.

Logged

Don't forget what the H stands for.

obi

27

Re: Writing CHDK to the internal flash

« Reply #7 on: 07 / December / 2023, 15:35:38 »

Thanks for the help.

Regarding Ghidra, I did a small terrible mistake where I chose big-endian as it was coming first in the search. Had to repeat the Firmware_analysis_with_Ghidra procedure many times until I found out. Now I only have 5-6 errors. Things make a bit more sense in there. I will explore and try to understand the SD card formatting logic.

Quote

The firmware dump and diskboot.bin are entirely different things.

Yea I got that. I was just testing something stupid where I copied the firmware dump to the diskboot.bin file and was hoping that the A1100 emulator will get stuck in a boot loop but as you said, there are other conditions that prevent that.

Logged