G1X mark ii Firmware Dump - page 2 - DryOS Development - CHDK Forum supplierdeeply
« previous next »
Pages: 1 [2] 3   Go Down

G1X mark ii Firmware Dump

  • 28 Replies
  • 11279 Views
*

Offline randuhmm

  • *
  • 12
Re: G1X mark ii Firmware Dump
« Reply #10 on: 10 / February / 2021, 14:10:12 »
Advertisements
After our discussion on IRC I tried following the SD card preparation procedure a few more times from scratch and I was still unable to get any response from the camera on boot. I picked up a newer 32GB card and it should arrive tomorrow so I can test that as well with the same steps. Still, this seems like it should work with my 8GB card.

One strange thing I just noticed is that, at random times, the "BOOTDISK" string seems to be replaced with spaces at the 0x1E0 location and moved to the 0x05C location. I have no idea how this is happening because I lock the SD card as soon as I remove it from my PC before I insert it into the camera. I can't figure out how to reproduce this behavior. Anyways, I have confirmed that the "BOOTDISK" string is in the 0x1E0 location before and after booting my camera so it seems like there is something else I am missing here. If I were to try another incompatible firmware (Say a G1X or SX700HS firmware), would I be able to verify that the SD card is bootable by crashing the camera?



Regarding the arm compiler version, it looks like I am running 9.2.1 in my WSL distro:
Code: [Select]
arm-none-eabi-gcc --version
arm-none-eabi-gcc (15:9-2019-q4-0ubuntu1) 9.2.1 20191025 (release) [ARM/arm-9-branch revision 277599]
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Logged
*

Offline reyalp

  • ******
  • 14082
Re: G1X mark ii Firmware Dump
« Reply #11 on: 10 / February / 2021, 14:46:42 »
Quote from: randuhmm on 10 / February / 2021, 14:10:12
I picked up a newer 32GB card and it should arrive tomorrow so I can test that as well with the same steps. Still, this seems like it should work with my 8GB card.
Yes, it should work on any card you can format FAT32.  (64GB+ default to exFAT, which means you have to jump through more hoops to get them formatted, but they still work if formatted FAT32)

Quote
One strange thing I just noticed is that, at random times, the "BOOTDISK" string seems to be replaced with spaces at the 0x1E0 location and moved to the 0x05C location. I have no idea how this is happening because I lock the SD card as soon as I remove it from my PC before I insert it into the camera.
It definitely should not be there, and might potentially cause problems. Formatting again and ensuring the string ONLY appears the correct location might be worth trying, if you haven't already.

Quote
I can't figure out how to reproduce this behavior. Anyways, I have confirmed that the "BOOTDISK" string is in the 0x1E0 location before and after booting my camera so it seems like there is something else I am missing here. If I were to try another incompatible firmware (Say a G1X or SX700HS firmware), would I be able to verify that the SD card is bootable by crashing the camera?
Well, you could try, but it's extremely unlikely to help. Everything points to your card not being recognized as bootable.

The camera doesn't examine or sanity check the contents of DISKBOOT.BIN it all. It just loads the file, blindly decodes the simple "dancingbits" XOR encoding, and executes it. So there is really no way a bad file can prevent it from being loaded. (edit: I'd forgotten, but this isn't quite true, as randuhmm discovered, it does have a couple of checks, which prevents it from loading a completely unencoded diskboot) The things that could prevent it are the card lock (or camera lock detect), the file system, the DISKBOOT string, or potentially other files in the root of the SD confusing the Canon firmware.

The last item is really only known to show up when files are copied from a Mac (see https://chdk.fandom.com/wiki/FAQ/Mac#You_may_have_issues_with_permissions.) but presumably would be possible to trigger in other ways.

Quote
Regarding the arm compiler version, it looks like I am running 9.2.1 in my WSL distro:
Code: [Select]
arm-none-eabi-gcc --version
arm-none-eabi-gcc (15:9-2019-q4-0ubuntu1) 9.2.1 20191025 (release) [ARM/arm-9-branch revision 277599]
Yup, that should be fine. I was mistaken about the arm gcc package version.
« Last Edit: 11 / February / 2021, 19:06:04 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: G1X mark ii Firmware Dump
« Reply #12 on: 10 / February / 2021, 17:24:20 »
Quote from: randuhmm on 10 / February / 2021, 14:10:12
One strange thing I just noticed is that, at random times, the "BOOTDISK" string seems to be replaced with spaces at the 0x1E0 location and moved to the 0x05C location.


I haven't done this on Windows for a long time; but that looks more like the first sector of the first partition, not the disk boot block.


If I remember correctly you have to open the RAW disk in HxD, not the drive letter.


Phil.

Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline waterwingz

  • ******
  • 12139
    • Waterwingz
Re: G1X mark ii Firmware Dump
« Reply #13 on: 10 / February / 2021, 17:34:23 »
Quote from: randuhmm on 10 / February / 2021, 14:10:12
After our discussion on IRC I tried following the SD card preparation procedure a few more times from scratch and I was still unable to get any response from the camera on boot. I picked up a newer 32GB card and it should arrive tomorrow so I can test that as well with the same steps. Still, this seems like it should work with my 8GB card.
Use one of the apps for preparing an SD card for CHDK use.  STICK or CHIMP. 

Pick any recent camera and let the utility do its work.  Then overwrite the resultant cards files with the files and folders for your G1X Mark ii port. 
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline randuhmm

  • *
  • 12
Re: G1X mark ii Firmware Dump
« Reply #14 on: 11 / February / 2021, 00:13:15 »
So after some in-depth troubleshooting on IRC, we identified the root of the issue being the platform / makefile.inc missing the required property:

NEED_ENCODED_DISKBOOT=12

This was needed for the correct dancing bits to be selected for encoding the DISKIMAGE.BIN file. After this was setup properly, I was able to get the boot SD card to work and also got the LED blinking using the address "0xd20b0884".

Now, my next step is preparing the boot.c file using the tools for decompiling the existing FW. Gonna start looking into that very soon. Thanks for all the help and pointers, everyone and especially reyalp.
Logged
*

Offline randuhmm

  • *
  • 12
Re: G1X mark ii Firmware Dump
« Reply #15 on: 14 / February / 2021, 21:42:17 »
Quick update:

I have spent a few evenings on IRC getting coached through setting up the "boot.c" file properly to correctly initialize and hook into the FW. I am currently working on getting all the proper values set for native functions and flags in various files.

Also, on a side note, I am currently tracking my code revisions using a local git repo. I just wanted a simple way to view all my changes in a historical tree. I used to use SVN but it has been awhile so I just used git for now since it's familiar. I don't have a remote setup so this is all just local (I do have local backups). However, I am curious what the next step is to get this code in the correct repository and branch.
Logged
*

Offline reyalp

  • ******
  • 14082
Re: G1X mark ii Firmware Dump
« Reply #16 on: 14 / February / 2021, 22:23:36 »
Quote from: randuhmm on 14 / February / 2021, 21:42:17
Also, on a side note, I am currently tracking my code revisions using a local git repo. I just wanted a simple way to view all my changes in a historical tree. I used to use SVN but it has been awhile so I just used git for now since it's familiar. I don't have a remote setup so this is all just local (I do have local backups). However, I am curious what the next step is to get this code in the correct repository and branch.
Usually, people post a patch against the current trunk in https://chdk.setepontos.com/index.php?topic=650.1640 or the porting thread when the port is relatively complete. You're welcome to post WIP code here, or in a personal repo elsewhere.

Unfinished ports can also be added to SVN with the autobuild disabled.
Logged
Don't forget what the H stands for.
*

Offline randuhmm

  • *
  • 12
Re: G1X mark ii Firmware Dump
« Reply #17 on: 15 / February / 2021, 02:43:11 »
OK, I'll probably post a snapshot of the added files and code tomorrow and continue to as more features are supported.
Logged
*

Offline randuhmm

  • *
  • 12
Re: G1X mark ii Firmware Dump
« Reply #18 on: 16 / February / 2021, 14:58:10 »
Here is the most up-to-date patch files for the code. I prepared the patch zipfile with the following BASH script:
Code: [Select]
SVN_REV="5729"
PATCH_DATE="$(date "+%Y-%m-%d")"
PATCH_ZIP="CHDK-g1x2-120a_svn-${SVN_REV}_${PATCH_DATE}.patch.zip"
rm *.patch
git format-patch -p master..HEAD
zip "${PATCH_ZIP}" *.patch
rm *.patch

It can be applied to the specified SVN trunk revision (5729) using the following script executed from the root directory of the repo:
Code: [Select]
rm *.patch
unzip CHDK-g1x2-120a_svn-5729_2021-02-16.patch.zip
for p in *.patch; do
    patch -p1 < "${p}"
done
rm *.patch
rm .gitignore

The local repo should now contain all of the new and updated added files from the "loader / g1x2" and "platform / g1x2" directories.

Regarding the current status, I am a bit stuck on what to do next. I suspect that I need to use Ghidra a bit more to search for the correct address values to use that are currently NULL_SUBs or FAKEDEFs. I may get around to digging through the decompiled original FW a bit more this week to try to locate some of these addresses
Logged
*

Offline reyalp

  • ******
  • 14082
Re: G1X mark ii Firmware Dump
« Reply #19 on: 17 / February / 2021, 17:54:07 »
Quote from: randuhmm on 16 / February / 2021, 14:58:10
Here is the most up-to-date patch files for the code.
Thanks for posting this, a very good start.

Quote
Regarding the current status, I am a bit stuck on what to do next. I suspect that I need to use Ghidra a bit more to search for the correct address values to use that are currently NULL_SUBs or FAKEDEFs. I may get around to digging through the decompiled original FW a bit more this week to try to locate some of these addresses
A few things I noticed looking at this:

in platform/main.c start() you *definitely* need the bss initialization loop. You might see some comments in other ports that sanity check is useless, but that only refers to the if statement, not the loop!

In platform/sub, entries correctly identified by the sig finder in stubs_entry.S should be just left in stubs_entry.S, not copied to stubs_min.S / stubs_entry_2.S. stubs_min.S and stubs_entry_2.s allow overriding variables and functions respectively, if the sig finder is wrong or doesn't find them. This doesn't directly affect functionality, but but it makes it easier to maintain and identify what needs to be done. (sx700 had a few things in override files that were also found by the sig finder, because the sig finder was improved after the port was done)

The  NULL_SUB address at the top of stubs_entry_2.s should point at a return instruction. It currently doesn't, which will likely crash if any of those functions are called.

Similarly if you use the RET0 or RET1, they should point at code that returns one of those values.

Also note you can't use NULL_SUB/RET0/RET1 on all functions. If you NULL_SUB a function that needs to return a real value (like GetDrive_TotalClusters) it won't immediately crash, but it will return random garbage. Returning 0 or 1 in this case would also be broken.

Attached is a patch against the trunk r5708 which cleans up some of these things:
enable bss initialization in main.c
correct NULL_SUB
remove auto-identified stubs from stubs_entry_2.S and stubs_min.S
generate firmware_crc_data.h

(note this would apply to a fresh SVN checkout of the trunk, not your source)

To find the remaining unknown functions and variables, I highly recommend the Ghidra version tracking tool: https://chdk.fandom.com/wiki/Ghidra_Version_Tracking_workflow_for_porting
Logged
Don't forget what the H stands for.
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics