G1X mark ii Firmware Dump

randuhmm

12

G1X mark ii Firmware Dump

« on: 05 / February / 2021, 02:50:12 »

Advertisements

Hello, I just dumped the firmware from my G1X mark ii. I am ready and willing to learn how to assist with the porting of this firmware if someone is able to assist. Here is the link to the firmware I dumped:

https://www.dropbox.com/s/w8ehxhz7mx5juwx/PRIMARY.BIN?dl=0

Logged

randuhmm

12

Re: G1X mark ii Firmware Dump

« Reply #1 on: 05 / February / 2021, 02:52:39 »

Gonna look over this wiki page tomorrow and the weekend to get started with analyzing the FW binary:
https://chdk.fandom.com/wiki/Adding_support_for_a_new_camera

Logged

reyalp

14117

Re: G1X mark ii Firmware Dump

« Reply #2 on: 05 / February / 2021, 13:58:39 »

Quote from: randuhmm on 05 / February / 2021, 02:50:12

Hello, I just dumped the firmware from my G1X mark ii. I am ready and willing to learn how to assist with the porting of this firmware if someone is able to assist. Here is the link to the firmware I dumped:

https://www.dropbox.com/s/w8ehxhz7mx5juwx/PRIMARY.BIN?dl=0

Welcome, and thanks for posting that. It's firmware 120a, which I haven't seen before. A dump of 101a was posted previously.

Quote from: randuhmm on 05 / February / 2021, 02:52:39

Gonna look over this wiki page tomorrow and the weekend to get started with analyzing the FW binary:
https://chdk.fandom.com/wiki/Adding_support_for_a_new_camera

That's a good starting point, but beware some of it is outdated, and some digic 6 specifics aren't covered.

I highly recommend Ghidra https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra for firmware analysis.

The digic 6 page also covers some other digic 6 specific tools https://chdk.fandom.com/wiki/Digic_6_Porting

This should be a fairly straightforward port. I've included the 101a firmware in my sig finder test set for a long time, so that should work pretty well. G1X II uses DryOS 54p3. SX700 is the only existing port with this version, so would be a good candidate for a reference port.

Logged

Don't forget what the H stands for.

randuhmm

12

Re: G1X mark ii Firmware Dump

« Reply #3 on: 05 / February / 2021, 14:23:49 »

Quote from: reyalp on 05 / February / 2021, 13:58:39

Welcome, and thanks for posting that. It's firmware 120a, which I haven't seen before. A dump of 101a was posted previously.

Thanks for the warm welcome! I stumbled across this project in researching how to use an external trigger for taking manual time-lapse photos on my G1X. I am super excited for some more advanced features and scripting on my camera.

Do you have a link to where the dump of the 101a version is? I haven't seen anything for the mark ii in my search and I probably would have tried to rollback my firmware to an older version to give this a shot before going down the rabbit hole of porting a whole new firmware. Also, it's possible that the 101a is only for the mark i version, I don't see it on the list of available firmware for the mark ii.

I am gonna try to flesh out this wiki page with additional info as I find it: https://chdk.fandom.com/wiki/G1_X_Mark_II

Quote from: reyalp on 05 / February / 2021, 13:58:39

That's a good starting point, but beware some of it is outdated, and some digic 6 specifics aren't covered.

I highly recommend Ghidra https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra for firmware analysis.

The digic 6 page also covers some other digic 6 specific tools https://chdk.fandom.com/wiki/Digic_6_Porting

This should be a fairly straightforward port. I've included the 101a firmware in my sig finder test set for a long time, so that should work pretty well. G1X II uses DryOS 54p3. SX700 is the only existing port with this version, so would be a good candidate for a reference port.

Thanks for the pointers, I struggled a bit trying to decompile the firmware last night. I tried using the latest freeware version of IDA by hex-rays (https://www.hex-rays.com/products/ida/support/download_freeware/) but I ran into a wall when trying to navigate the updated ui. I don't think this version supports ARM processors.

Also, if anyone else wants to start inspecting the firmware, I think the contents of the "CBDUMPER.LOG" might be useful to locate an entry point:

Code: [Select]

Started
FC000000 dry
done

Logged

reyalp

14117

Re: G1X mark ii Firmware Dump

« Reply #4 on: 05 / February / 2021, 16:47:53 »

Quote from: randuhmm on 05 / February / 2021, 14:23:49

Do you have a link to where the dump of the 101a version is? I haven't seen anything for the mark ii in my search and I probably would have tried to rollback my firmware to an older version to give this a shot before going down the rabbit hole of porting a whole new firmware. Also, it's possible that the 101a is only for the mark i version, I don't see it on the list of available firmware for the
mark ii.

No, the 101a I was talking about is a mk2 firmware. The mk1 is a digic 5 camera, completely unrelated to the mk2 as far as CHDK is concerned. The firmware versions are per model, so different models having the same version like "101a" isn't significant.

I guess the dump was posted here https://chdk.setepontos.com/index.php?topic=12074.0
I don't see it in the dumps archive, I'll re-upload it somewhere later.

However, I definitely would not recommend trying to downgrade. 101a was never ported either, so there would be no benefit, and official Canon updates are available for 120a https://www.usa.canon.com/internet/portal/us/home/support/details/cameras/point-and-shoot-digital-cameras/advanced-cameras/powershot-g1-x-mark-ii?subtab=downloads-firmware

That makes it better to port 120a, because any users will be able to upgrade to the latest firmware and use it, while downgrading is generally not an option.

Quote

Thanks for the pointers, I struggled a bit trying to decompile the firmware last night. I tried using the latest freeware version of IDA by hex-rays (https://www.hex-rays.com/products/ida/support/download_freeware/) but I ran into a wall when trying to navigate the updated ui. I don't think this version supports ARM processors.

Yes, you'd need the paid version of IDA. Unless you already have it and are familiar with it, I'd definitely recommend ghidra, since it's free and the CHDK specific scripts are quite helpful.

Quote

Also, if anyone else wants to start inspecting the firmware, I think the contents of the "CBDUMPER.LOG" might be useful to locate an entry point:

Yep, all the digic 6 models so far have the same start address.

Logged

Don't forget what the H stands for.

randuhmm

12

Re: G1X mark ii Firmware Dump

« Reply #5 on: 06 / February / 2021, 00:23:28 »

I was able to follow the steps here to get the firmware analyzed using Ghidra: https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra

I am now trying to figure out what the next steps are for the porting. Ultimately getting towards a buildable and testable package that I can install on a SD card and test out on my camera would be nice to get to this weekend. I am sure there are a ton of things I am missing and I am trying to keep some notes of my findings. For example, I had some issues running the command:

Code: [Select]

make PLATFORM=g1x2 PLATFORMSUB=120a rebuild-stubs
It only worked when I added some additional params like this:

Code: [Select]

make PLATFORM=g1x2 PLATFORMSUB=120a \
  TARGET_PRIMARY=/path/to/PRIMARY.BIN \
  OPT_CAPSTONE_TOOLS=1 \
  CAPSTONE_TOOLS_INC=-I/usr/include/capstone/ \
  CAPSTONE_TOOLS_LINK=-lcapstone rebuild-stubs

I later located the tutorial (https://chdk.fandom.com/wiki/Digic_6-7_Porting#Configuring_CHDK_capstone_tools) on how to enable capstone tools, but my way worked for me on using an ubuntu disto on windows with WSL2. I did need to install a few packages to get everything working:

Code: [Select]

sudo apt install gcc-arm-none-eabi
sudo apt install libcapstone-dev

Once installing these packages I was able to get the make command working. Importing the resulting files into the Ghidra was pretty straightforward, although I don't fully understand all the finer details regarding what I was doing.

I think I am done for today, but I will try to make some more time this weekend to dig into the files produced by these steps and see what is needed to perform an actual build perhaps using the SX700 as a guide.

Logged

reyalp

14117

Re: G1X mark ii Firmware Dump

« Reply #6 on: 06 / February / 2021, 01:05:51 »

Quote from: randuhmm on 06 / February / 2021, 00:23:28

I was able to follow the steps here to get the firmware analyzed using Ghidra: https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra

Ultimately getting towards a buildable and testable package that I can install on a SD card and test out on my camera would be nice to get to this weekend.

Depending on your experience with these things, a usable build in one weekend might be a stretch, but blinking an LED should be within reach

I posted a somewhat recent overview here: https://chdk.setepontos.com/index.php?topic=5592.msg144892#msg144892 (the mentions of code-gen don't apply to Digic 6 though, you should generate the disassembly with capis instead)

Quote

It only worked when I added some additional params like this:
Code: [Select]
make PLATFORM=g1x2 PLATFORMSUB=120a \ TARGET_PRIMARY=/path/to/PRIMARY.BIN \ OPT_CAPSTONE_TOOLS=1 \ CAPSTONE_TOOLS_INC=-I/usr/include/capstone/ \ CAPSTONE_TOOLS_LINK=-lcapstone rebuild-stubs

You can put these settings in localbuildconf.inc. Copy buildconf.inc to localbuildconf.inc and edit to taste.

Which CHDK source branch are you using? I'd recommend using the trunk for a digic 6 port.

Quote

Code: [Select]
sudo apt install gcc-arm-none-eabi sudo apt install libcapstone-dev

You should check that the arm gcc version is one of the supported versions (4, 5, 8, 9 or 10), otherwise you might end up with a build that doesn't work. The trunk makefiles should check this. ~~IIRC, the version supplied by ubuntu 20.04 is 6~~, which I think had problems (but I don't recall the details.)
edit: I remembered wrong, according to https://launchpad.net/ubuntu/focal/+package/gcc-arm-none-eabi it's 9, which is supported.

The capstone version must be 4.x (or 3.x built from source with the patch from tools applied)

FWIW, you can get a native windows toolchain from https://chdk.setepontos.com/index.php?topic=12752.0

There is also a docker based toolchain https://chdk.fandom.com/wiki/Compiling_CHDK_With_Docker

Quote

Once installing these packages I was able to get the make command working. Importing the resulting files into the Ghidra was pretty straightforward, although I don't fully understand all the finer details regarding what I was doing.

The CHDK scripts mainly set up the memory map (so various bits of code and data appear at their real addresses), and name the functions found by finsig_thumb2.

For porting, the Ghidra version tracking tool is quite useful https://chdk.fandom.com/wiki/Ghidra_Version_Tracking_workflow_for_porting

If you want realtime advice, I'm sometimes in the #chdk IRC channel on freenode.

« Last Edit: 10 / February / 2021, 01:46:11 by reyalp »

Logged

Don't forget what the H stands for.

randuhmm

12

Re: G1X mark ii Firmware Dump

« Reply #7 on: 09 / February / 2021, 20:47:38 »

Quote from: reyalp on 06 / February / 2021, 01:05:51

Which CHDK source branch are you using?

I am using the 1.6 version (which is the trunk, I think?).

Quote from: reyalp on 06 / February / 2021, 01:05:51

You should check that the arm gcc version is one of the supported versions (4, 5, 8, 9 or 10), otherwise you might end up with a build that doesn't work. The trunk makefiles should check this. IIRC, the version supplied by ubuntu 20.04 is 6, which I think had problems (but I don't recall the details.)

gcc --version indicates I am using version 9.0.3

So I wasn't able to work on this over the weekend, however I was able to dig into the build process a bit today. Currently I am able to build the CHDK firmware for my camera using WSL running Ubuntu 20.04 without errors. I end up with two zip files in the bin directory:

Code: [Select]

g1x2-120a-1.6.0-5729.zip
g1x2-120a-1.6.0-5729-full.zip

However, when unpacking the generated firmware packages onto a SD card prepared for booting, I am unable to get some sort of response from the camera. I suspect I am missing something in the firmware config or code to properly hijack the boot process, but I am not sure yet what that might be. There are some documents which detail the boot procedure more which I might dig into to better understand how CHDK intercepts and injects it's own boot procedure.

Most of the code I have in the platform/g1x2 and platform/g1x2/sub/120a directories is simply a copy+pasta of the code found in the "sx700hs" locations. This might also be the reason nothing happens when I try to load the firmware from the SD on boot.

So to summarize I have the following ready to go:

Dumped and analyzed firmware
Skeleton files in the "platform" and "loader" dirs for my camera/firmware (some of this is from the analyzed firmware, others are NULL_SUB / FAKEDEF or copied from the sx700hs)
Working toolchain for compiling and linking the firmware
Software for preparing a Bootable SD (EOScard) and a camera to test with

I think blinking an LED would be a great next step here, so I will look over some of the forum posts that aim to help with that. Any other feedback is greatly appreciated. Cheers!

Logged

waterwingz

12140

Re: G1X mark ii Firmware Dump

« Reply #8 on: 09 / February / 2021, 21:49:20 »

Quote from: randuhmm on 09 / February / 2021, 20:47:38

So I wasn't able to work on this over the weekend, however I was able to dig into the build process a bit today. Currently I am able to build the CHDK firmware for my camera using WSL running Ubuntu 20.04 without errors. I end up with two zip files in the bin directory:
Code: [Select]
g1x2-120a-1.6.0-5729.zip g1x2-120a-1.6.0-5729-full.zipHowever, when unpacking the generated firmware packages onto a SD card prepared for booting, I am unable to get some sort of response from the camera. I suspect I am missing something in the firmware config or code to properly hijack the boot process, but I am not sure yet what that might be. There are some documents which detail the boot procedure more which I might dig into to better understand how CHDK intercepts and injects it's own boot procedure.

Did you prepare your SD card properly?

https://chdk.fandom.com/wiki/Prepare_your_SD_card

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14117

Re: G1X mark ii Firmware Dump

« Reply #9 on: 09 / February / 2021, 22:47:02 »

Quote from: randuhmm on 09 / February / 2021, 20:47:38

Quote from: reyalp on 06 / February / 2021, 01:05:51
Which CHDK source branch are you using?
I am using the 1.6 version (which is the trunk, I think?).

Yes.

Quote

gcc --version indicates I am using version 9.0.3

That's the host compiler. The one with potential compatibility problems is the arm compiler. Try
arm-none-eabi-gcc --version

Quote

Most of the code I have in the platform/g1x2 and platform/g1x2/sub/120a directories is simply a copy+pasta of the code found in the "sx700hs" locations. This might also be the reason nothing happens when I try to load the firmware from the SD on boot.

From your description in IRC (cam boots normally, shows card locked) it's a problem making the card bootable, not anything to do with code. Unless you've actually implemented all the early boot code but not started any tasks.

Uncorrected copy / paste code would definitely be a problem, but the result would almost certainly be a crash. It would not in any case prevent the camera from loading DISKBOOT.BIN

Quote

I think blinking an LED would be a great next step here, so I will look over some of the forum posts that aim to help with that. Any other feedback is greatly appreciated. Cheers!

You need to solve the booting problem first, but the next thing I would do is put a blinker loop in loader main.c. You can find a commented example in g7x.

Logged

Don't forget what the H stands for.