CHIMP - Canon Hack Installation and Management Platform

dmitrys

Canon Hack Installation and Management Platform - a Windows utility for installing/updating CHDK (and more).

Source
Releases (Patch added)

If you encounter a bug:

Make sure it isn't a known limitation (see below) or issue.
CHIMP.exe /verbose
cd %TEMP%\CHIMP
Locate the relevant log file (should be CHIMP-YYYYMMDD.log)
Create an issue attaching the log file from the previous step.

Known limitations:

No special multiple partition handling on Windows 10 Creators Update
No maximum card size checks. Use at your own risk!

reyalp

Some people don't like to install java just to run stick, so there's certainly room for a native windows utility. You might also be able to avoid some of the confusion about running it with the correct permission.

Probably obvious, but one thing you should be very cautious about is ensuring the selected SD card is actually an SD card.

IIRC there are a couple cameras which cannot be detected correctly from exif, see http://www.zenoshrdlu.com/stick/sticknfb.html

You might be able to detect the camera by platform id when it's connected to USB.

dmitrys

Thank you for your reply.

reyalp wrote:Some people don't like to install java just to run stick, so there's certainly room for a native windows utility.

Well, not native per se, since it requires .NET Framework 4.5.2 (I'm not sure why I selected that specific version; 4.5 would probably do).

reyalp wrote:You might also be able to avoid some of the confusion about running it with the correct permission.

Check.

reyalp wrote:IIRC there are a couple cameras which cannot be detected correctly from exif, see http://www.zenoshrdlu.com/stick/sticknfb.html

Check, see attachment 😎

reyalp wrote😛robably obvious, but one thing you should be very cautious about is ensuring the selected SD card is actually an SD card.

Actually, this one is tricky. I can easily verify that the card is SD if it's in the built-in card reader, but not if it's in an external one (at least not easily).

Now, I'm going to need a couple of CHDK questions answered for this utility to be usable (Can you spot the problem with the first couple of screenshots? 😉 ) Should I post those here or in separate topic(s)?

reyalp

dmitrys
Actually, this one is tricky. I can easily verify that the card is SD if it's in the built-in card reader, but not if it's in an external one (at least not easily).

On thing you might do is sanity check the size and existing format, and put up a big warning if it's unusual e.g. 4-64 GB in FAT32 or EXFAT is unlikely to be someones main hard drive at least. Of course there's some overlap, 128 GB cards and SSDs are relatively common.

Now, I'm going to need a couple of CHDK questions answered for this utility to be usable (Can you spot the problem with those first screenshots? 😉 ) Should I post those here or in separate topic(s)?

Probably simplest to just keep it in this thread.

dmitrys

reyalp
On thing you might do is sanity check the size and existing format, and put up a big warning if it's unusual e.g. 4-64 GB in FAT32 or EXFAT is unlikely to be someones main hard drive at least. Of course there's some overlap, 128 GB cards and SSDs are relatively common.

Oh, I thought you were talking about distinguishing between SD/MMC/CF etc.

Internal hard drives are never candidates, since only Removable Media drives are enumerated.

If multiple candidates are present, the user will have to explicitly select one; see screenshot (note the radio buttons in lieu of a combo-box).

As for exFAT, isn't FAT12/FAT16/FAT32 required for autoboot to work? I'm hoping to get rid of the Firmware Update mode wherever possible, which brings me to the first set of questions...

reyalp

dmitrys
Internal hard drives are never candidates, since only Removable Media drives are enumerated.

I don't know your code would see them, but FWIW internal hard drives can show up as removable drives (e.g. http://www.overclock.net/t/974023/fix-ahci-sata-drives-showing-in-safely-remove-hardware)

As for exFAT, isn't FAT12/FAT16/FAT32 required for autoboot to work?

Yes, current cameras do not boot from exFAT. But 64 GB and larger cards are likely to come formatted exFAT.

dmitrys

So the answer to my riddle from above is that CHDK won't autoboot from a FAT32 card (1st screenshot) on IXUS 870 (2nd screenshot).

I used the ExifTool metadata and [tt]camera_list.csv[/tt] as inputs to build a mapping from Model ID + Firmware Revision to CHDK build prefixes.

My next steps are to determine for each model:

1. whether booting from FAT32 is supported. I believe that if the build includes any SD Card partition options, the answer is no. Which build option should I look for?

2. the ALT button. I found [tt]CAM_ALT_BUTTON_NAMES[/tt] and [tt]CAM_ALT_BUTTON_OPTIONS[/tt] in [tt]platform_camera.h[/tt], but what if [tt]CAM_ADJUSTABLE_ALT_BUTTON[/tt] is undefined (as in [tt]ixus870_sd880[/tt])?

3. the autoboot flag. N and N Facebook use the microSD door, which is indicated by [tt]KBD_SKIP_READONLY_BIT[/tt] and [tt]SD_DOOR_OVERRIDE[/tt] in [tt]platform_kbd.h[/tt]. Any other models/flags?

Bonus question:

Is the SD Card tool a module? I'm hoping to remove it whenever possible (i.e., for a bootable single partition).

zeno

STICK (and ASSIST) use the following method to detect if the camera supports booting from FAT32:

1. obtain the camera model ID from the photo's EXIF (Canon bump this number up with each new model, so it's in historical sequence) - this number is in the CanonMakerNoteDirectory with an offset of x'10' - see http://www.exiv2.org/tags-canon.html (Ger Vermeulen's page no longer exists)
2. compare it with the number of the first camera to support booting from FAT32 (the A3300) - the first with DRYOS2.3 Release 47 - its ID is 0x2980000

The algorithm to select a drive tries its best to avoid 'hard drives' by checking the removability and size, but as REYALP says, there are problems with 128GB. Both STICK and ASSIST present a dropdown list of drives if there's more than one suitable candidate.

I'm happy to share the Windows code that checks for and does any necessary partitioning (it's in the C DLL's containing native methods that STICK loads if run on a Windows machinep. )

zeno

I'm also happy to answer any questions you may have about how STICK works - almost all the people who have problems with STICK are Windows users - having an alternative would reduce the number of emails I get!

dmitrys

Hello, Dave.

It's great to hear from you. Thank you for replying and for being so gracious!

zeno wrote:STICK (and ASSIST) use the following method to detect if the camera supports booting from FAT32

I guess that would work, but I'm hoping for a (supposedly) more reliable method of deducing it from the CHDK repo.

zeno wrote:The algorithm to select a drive tries its best to avoid 'hard drives' by checking the removability and size, but as REYALP says, there are problems with 128GB.

I'm hoping checking drive sizes could be avoided. I believe that if someone really wants to format or corrupt their hard drive, they will find a way.

zeno wrote:I'm happy to share the Windows code that checks for and does any necessary partitioning (it's in the C DLL's containing native methods that STICK loads if run on a Windows machine.)

Yes, please.

Was the code in question distilled from CHDK, taken elsewhere, or written by you? In the latter case, would you agree to my releasing it under a liberal license (with proper attribution, of course)? Since I assumed the former, I tentatively selected GPLv2, but I'm flexible.

Speaking of which, fat32format is also GPL. Since STICK already has JNI, why not add it to the package as well?

Thanks.

zeno

The code for partitioning is (mostly) in CHDK too. I think it actually originates from Canon - certainly not written by me.

I've given the code for STICK to several people - the main reason for not making it public is its messy nature, lack of useful comments and general untidiness. Send me an email and I'll send you the 32-bit and 64-bit DLL code.

daveATzenoshrdluDOTcom

dmitrys

zenoI've given the code for STICK to several people - the main reason for not making it public is its messy nature, lack of useful comments and general untidiness.

That's precisely why I haven't yet published any of the CHDK utility code - hopefully I'll find the time and energy to clean it up.

reyalp

dmitrys
I guess that would work, but I'm hoping for a (supposedly) more reliable method of deducing it from the CHDK repo.

There is not, I would say using the PID as zeno describes would be the best option. The ability to boot from FAT32 is purely a product of the Canon firmware, so isn't explicitly defined in the CHDK source. CHDK usually supports multi-partition on cameras that don't support FAT32 booting, but there could be exceptions in either direction.

I'm hoping checking drive sizes could be avoided. I believe that if someone really wants to format or corrupt their hard drive, they will find a way.

You can't make it totally foolproof, but IIRC there have been cases of this involving CHDK tools in the past, so I would suggest a cautious approach.

2. the ALT button. I found CAM_ALT_BUTTON_NAMES and CAM_ALT_BUTTON_OPTIONS in platform_camera.h, but what if CAM_ADJUSTABLE_ALT_BUTTON is undefined (as in ixus870_sd880)?

Then it's KEY_PRINT, but this may or may not be an actual "print" button. For example, on cameras without a print button, the play button might defined as print for CHDK purposes. In some cases there might be another key defined with the same bits (e.g. KEY_PRINT and KEY_PLAYBACK defined with the same bit values kbd.c) and others might not. Unfortunately this isn't defined in any coherent way, and is very hard to fix without having all the cameras.

Is the SD Card tool a module? I'm hoping to remove it whenever possible (i.e., for a bootable single partition).

Are you asking about the "SD Card" section of the CHDK menu? It's not, but in any case I would strongly urge you to install the full, unmodified CHDK package no matter what.

3. the autoboot flag. N and N Facebook use the microSD door, which is indicated by KBD_SKIP_READONLY_BIT and SD_DOOR_OVERRIDE in platform_kbd.h. Any other models/flags?

I'm not aware of any. Canon made a few other microSD cameras, but I remember any being ported.

dmitrys

reyalp
There is not, I would say using the PID as zeno describes would be the best option. The ability to boot from FAT32 is purely a product of the Canon firmware, so isn't explicitly defined in the CHDK source. CHDK usually supports multi-partition on cameras that don't support FAT32 booting, but there could be exceptions in either direction.

Yes, I found [tt]CAM_MULTIPART[/tt]. Its use does indeed look pretty incoherent.

Then it's KEY_PRINT, but this may or may not be an actual "print" button. For example, on cameras without a print button, the play button might defined as print for CHDK purposes. In some cases there might be another key defined with the same bits (e.g. KEY_PRINT and KEY_PLAYBACK defined with the same bit values kbd.c) and others might not. Unfortunately this isn't defined in any coherent way, and is very hard to fix without having all the cameras.

I suppose maintaining code for 134 cameras with 290 distinct revisions is just as easy as it sounds 😉

dmitrys

reyalp
in any case I would strongly urge you to install the full, unmodified CHDK package no matter what.

I believe this deserves special attention, hence the separate reply.

First of all, there are two "full" CHDK packages, which will appear as Installation Type options. However, I was thinking about adding at least one other option - a "Minimal" install, which would exclude all the extras, i.e. games and Eye-Fi. I believe CHDK could do just fine without those (I wish the Games menu item could be removable as well). And how about Module Explorer, CPU info, Benchmark etc.?

There is also the issue of (more or less) useless files, i.e.:

PS.FI2 - can be removed unless absolutely necessary (or convenient, as could be the case with the N model)
vers.req - no use if installed via GUI
changelog.txt - outdated
readme.txt - little use if installed via GUI

Some subdirectories have readme.txt files with the following content:

nothing in here yet.

Maybe it's just my OCD, but I suggest removing those.

reyalp

dmitrys
First of all, there are two "full" CHDK packages, which will appear as Installation Type options. However, I was thinking about adding at least one other option - a "Minimal" install, which would exclude all the extras, i.e. games and Eye-Fi. I believe CHDK could do just fine without those (I wish the Games menu item could be removable as well). And how about Module Explorer, CPU info, Benchmark etc.?

Note the minimal package is not intended an option for new installs. It's a leftover to allow people who already had an install to update only the binary files. These days the difference between a 300k and 700k download isn't significant, but changing the autobuild is complicated and potentially breaks things that scrape the page, so we've left them.

PS.FI2 - can be removed unless absolutely necessary (or convenient, as could be the case with the N model)
vers.req - no use if installed via GUI
changelog.txt - outdated
readme.txt - little use if installed via GUI

Please don't do this. Just install all the files. 4 GB cards cost couple bucks, and the whole CHDK install uses less than a single raw from a modern cam.

If you start leaving things out, the wiki pages etc will refer to stuff that isn't there and cause headaches.

nothing in here yet./quote]
Maybe it's just my OCD, but I suggest removing those.

Yeah, they aren't helping much and should either have content or be removed.

dmitrys

I should report that there has been some progress with CHDK Utility.

I'm posting two new screenshots (the maximum allowed per post >🙁) from the (mostly completed) Introduction and Summary steps.

I was sorely missing some of the instructions included in the latter during my first encounter with CHDK.

As you may have guessed, the notes and other metadata are scraped from the repository.

dmitrys

There goes the Installation Type step. The less the merrier I suppose 😎

reyalp
Note the minimal package is not intended an option for new installs. It's a leftover to allow people who already had an install to update only the binary files. These days the difference between a 300k and 700k download isn't significant, but changing the autobuild is complicated and potentially breaks things that scrape the page, so we've left them.

Agreed, although I believe I did see something in the vein of "don't overload our server, use only the minimal build for updates".

Speaking of updates, is there any reliable way of gleaning the build number from the binaries?

If you start leaving things out, the wiki pages etc will refer to stuff that isn't there and cause headaches.

Well, the whole point of this utility is for people to not need the wiki. Seriously, have you seen the size of this page? 😉

reyalp

dmitrys
There goes the Installation Type step. The less the merrier I suppose 😎Speaking of updates, is there any reliable way of gleaning the build number from the binaries?

Once they are installed? Not that I can think of. For all modern cameras, binaries are encoded (see dancingbits and packfi2 in the source tools directory) so you can't trivially read the version from the executable.

Having the version identified on the SD card seems like it could be useful, suggestions are welcome.

Well, the whole point of this utility is for people to not need the wiki.

Yes, but my worry is that they go back later and try to do something described on the wiki, and it mysterious doesn't work. e.g. long after they installed they decide they want to run benchmarks, but original didn't install the benchmark module, and then they come on the forum asking why the benchmark item doesn't appear.

dmitrys

reyalp
Once they are installed? Not that I can think of. For all modern cameras, binaries are encoded (see dancingbits and packfi2 in the source tools directory) so you can't trivially read the version from the executable.

If I were looking for trivial, I would have given up on the entire CHDK Utility project 😀

Having the version identified on the SD card seems like it could be useful, suggestions are welcome.

Software metadata proposal

Edit: Made [tt]source[/tt] an optional property.
Edit 2017-04-18: Moved to GitHub wiki.