Signature verification prior to startup - page 6 - General Discussion and Assistance - CHDK Forum supplierdeeply
« previous next »
Pages: 1 2 3 4 5 [6] 7 8 9   Go Down

Signature verification prior to startup

  • 83 Replies
  • 32183 Views
*

Offline reyalp

  • ******
  • 13715
Re: Signature verification prior to startup
« Reply #50 on: 12 / October / 2013, 17:28:17 »
Advertisements
Quote from: srsa_4c on 12 / October / 2013, 17:14:15
Looks like I'll have to modify the signature generation code. Thanks to nafraf's recent commit (and then his research), it turned out that Debian (and its derivatives) have
- mawk 1.3.3, which is unable to handle integers over 0x7fffffff
- a shell named "dash" as the default shell, whose built-in echo command doesn't know about the -e switch.
The latter can be dealt with easily. To be able to output correct addresses, I'll either have to use GNU awk (is it available on the autobuild server?), or do the math inside the sigfinder. Unless somebody has a better idea, I'll probably modify the sigfinders.
This is probably better than relying on specific flavors of awk...
Quote
The shell issue has visible consequences btw: the autobuild's zip files' zip comment always begins with "-e " ...
It based on https://wiki.ubuntu.com/DashAsBinSh seems like bash is still available, so we could potentially force the use of /bin/bash instead of dash (though using dash would probably make builds faster)
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #51 on: 22 / October / 2013, 18:06:02 »
I went ahead and implemented the "blink whenever you can" version (it was actually suggested by ahull). This will probably not make it into CHDK, but I post it anyway.
I have compiled a table that contains an LED address, P-ID location, and the P-ID for every camera that can execute the resulting DISKBOOT. If the startup check fails, a routine will use this table to get the camera's P-ID (there is a slim chance that it will find an incorrect P-ID due to the different P-ID locations in ROM). When a value that looks like a P-ID is found, it will blink using the corresponding LED address.
DISKBOOT compatibility depends on the cameras' "dancing bits" encoding. Cameras with the same encoding can start each others diskboot files. Diskboot files with different encodings will result in the previously mentioned nonsense, which we can't control. It also looks like unencoded diskboot files are ignored by later cameras (they fail a check).
Now for the table data:
Several cameras are left out for one or more of the following reasons:
- the P-ID location is unknown (early Vx cameras with short available firmware dumps)
- no LED address can be determined ( debug_led() uses firmware functions and/or no-one has found correct LED addresses)
- strange LED operating requirements (on DIGIC 5 cams)

The rest isn't absolutely sure either: some cameras may require two GPIOs to operate certain LEDs (the print LED didn't work on the ixus30/40, s80 for example).

The patch contains the needed changes for a few cameras. check_compat() is now the first subroutine that gets executed - entry.S is easier to change (with a script), and it also avoids setting a potentially wrong GPIO (SD power).

The "sub CPU" cameras don't let the DIGIC loop indefinitely, the other CPU shuts down the camera a few seconds later.

Unrelated finding: shutdown() in platform/camera/lib.c can be declared unusable. Many ports have nonsense code here (unless I'm misunderstanding something). This should switch off the DIGIC's main power directly by using a GPIO. Some ports try to operate an LED instead, others use mis-calculated offsets.
Logged
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #52 on: 04 / December / 2015, 17:05:42 »
Update.

Attached is a version that appears to work. It's likely not the final state of things because:
- I made a script that generates the pid_leds array: tools/compatbuilder.sh
  The script is not currently part of the build system. I'm not certain if it should be. The array only needs to change when adding new ports.
  I have not yet attempted to test the whole thing on Windows (or any other system besides mine).
- I already made some educated guesses about usable LED addresses on ports without a defined "debug LED", but there still are ports without LED details. (The impact is that inappropriate builds won't blink on those cams.) I'll probably add some more guessed entries later.
- Ports with compat check will start execution with the compatibility check routine. I did this because most entry.S files contain GPIO-poking code (which in theory could be risky).
- The compat check routines do use some stack. I currently do not make any SP adjustments. I'd expect that the camera routines already set a sane SP prior to jumping to the loaded code.
- I added two columns to camera_list.csv. This could be subject of discussion, I don't know whether it affects CHDK Shell.
- Compatibility check is enabled by default, but can be disabled with OPT_DISABLE_COMPAT_CHECK

The patch does include some changes that are not strictly related.
- I finally fixed the "os-camera-lists" build action
- I have replaced the loader's blob copy routine in all ports that have the reworked loader. Turns out, all ports used the same code despite formatting differences.
- Fixed yet another bug in bin_compat.h generation

Ports with the old loader are untouched for now - that means they won't check compatibility.

I used scripts for making repetitive changes in lots of files. The result doesn't look too nice in some cases (indentation), but the original files were already looking like a mess.

edit (& reupload):
- sx280 loader fixed
- SP is set to MEMBASEADDR before the first BL in loader

edit2:
- successful batch build on Windows/Cygwin
« Last Edit: 06 / December / 2015, 13:46:07 by srsa_4c »
Logged
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #53 on: 06 / December / 2015, 12:11:54 »
Previous post was missing a tl;dr section, so here it is:

Q) What is the compatibility check good for?
A) It can prevent execution of incompatible code, thereby reducing the bricking risk somewhat.

Q) What kind of accidents can it prevent?
A)
  - attempting to run CHDK on an unsupported firmware version (diskboot, fw update)
  - attempting to run CHDK on a different model (diskboot only)

Q) When is it ineffective?
A) It can't prevent executing garbage on cameras with incompatible dancingbits encoding. It also won't help when one attempts to execute a DIGIC 6 diskboot on a DIGIC II..5 camera (or vice versa).

Q) What happens when the check fails?
A) The camera's ARM CPU enters an infinite loop. If the camera's LED details are known, one LED will blink. Otherwise, no visible activity will occur. The battery will start draining. A very few number of models will switch off due to a hardware watchdog (the so-called subcpu models: S2IS, S3IS, S80, plus maybe a few more).

Q) What to do when this happens?
A) The camera's battery needs to be removed (pushing buttons won't help).


The attached version of compatbuilder.sh needs a parameter (camera_list.csv) or it will exit with an error message.
Logged
*

Offline reyalp

  • ******
  • 13715
Re: Signature verification prior to startup
« Reply #54 on: 06 / December / 2015, 16:35:10 »
Thanks for working on this, and the very clear tl;dr :)

I think this is a good thing, and it's look pretty complete.

I tested the following with diskboot. In each case the "correct" build works.
D10 100a (build for the 100b copied sub also works)
A1100 100c blinks

A540 100b
A530 100a blinks

ixus140_elph130 (build for the 100b copied sub also works)
ixus135_elph120 100b blinks

SX160is 100a
g15 100b blinks

Quote
The script is not currently part of the build system. I'm not certain if it should be. The array only needs to change when adding new ports.
I'm not sure either. Rebuilding every time for batch builds seems like a waste, but it might be easy to forget. Leaving it manual is probably OK.

The impact of not rebuilding would be that incorrect builds loaded on that camera would not blink, but the new camera would still boot it's own builds, correct?

Quote
- I added two columns to camera_list.csv. This could be subject of discussion, I don't know whether it affects CHDK Shell.
CHDK Shell has an error with the patched csv. I lean toward keeping camera_list.csv to the minimum needed to do batch builds, just because it could get unmanageable quickly if we put a lot of stuff there, but if the alternative are too ugly I'd be OK with having there.

Quote
Q) When is it ineffective?
A) It can't prevent executing garbage on cameras with incompatible dancingbits encoding. It also won't help when one attempts to execute a DIGIC 6 diskboot on a DIGIC II..5 camera (or vice versa).
I think these cases are relatively low risk, it should be hard for them to get very far.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #55 on: 06 / December / 2015, 17:17:19 »
Quote from: reyalp on 06 / December / 2015, 16:35:10
I'm not sure either. Rebuilding every time for batch builds seems like a waste, but it might be easy to forget. Leaving it manual is probably OK.
Just had an idea: perhaps checksumming camera_list.csv (& storing the result) and only running the script when changed could help... Downside: implementing this might be trickier...

Quote
The impact of not rebuilding would be that incorrect builds loaded on that camera would not blink, but the new camera would still boot it's own builds, correct?
Yes, if the port's bin_compat.h is correct (that needs to be created manually on DIGIC6, automatic on <DIGIC6).

Quote
CHDK Shell has an error with the patched csv.
:(
Quote
I lean toward keeping camera_list.csv to the minimum needed to do batch builds, just because it could get unmanageable quickly if we put a lot of stuff there, but if the alternative are too ugly I'd be OK with having there.
The alternative is to put the required two fields in camera specific makefile.inc files (just like other pieces of information). Can be done with some work. Downside is that this detail will get a bit less apparent there.

Thanks for the tests and thoughts.
Logged
*

Offline reyalp

  • ******
  • 13715
Re: Signature verification prior to startup
« Reply #56 on: 06 / December / 2015, 17:45:25 »
Quote from: srsa_4c on 06 / December / 2015, 17:17:19
Just had an idea: perhaps checksumming camera_list.csv (& storing the result) and only running the script when changed could help... Downside: implementing this might be trickier...
That seems like a lot of extra work and complexity. Normally this kind of thing would be done with a Makefile dependency, but the way we use our makefiles I'm not sure that would actually help. I think manually re-building is OK for now.

I have thought about making a "adding a new port to svn" checklist before, maybe this is a good excuse.

Quote
The alternative is to put the required two fields in camera specific makefile.inc files (just like other pieces of information). Can be done with some work. Downside is that this detail will get a bit less apparent there.
I think that's OK, we already have a lot of critical stuff there. Have to watch out for inappropriate copy/paste, but the same is true for camera_list.csv lines.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #57 on: 06 / December / 2015, 19:03:23 »
Quote from: reyalp on 06 / December / 2015, 17:45:25
That seems like a lot of extra work and complexity. Normally this kind of thing would be done with a Makefile dependency, but the way we use our makefiles I'm not sure that would actually help.
Okay, I'm convinced. Was just an idea :)
Quote
Have to watch out for inappropriate copy/paste, but the same is true for camera_list.csv lines.
I'll get this done by a script. However, this will mean that all ports will now get a makefile.inc in platform/(cam)/ .
Logged
*

Offline reyalp

  • ******
  • 13715
Re: Signature verification prior to startup
« Reply #58 on: 06 / December / 2015, 19:42:42 »
Quote from: srsa_4c on 06 / December / 2015, 19:03:23
I'll get this done by a script. However, this will mean that all ports will now get a makefile.inc in platform/(cam)/ .
I would actually like to standardize on putting the non-sub specific information there. It's already done this way on some cameras like G12. Things like ROM start, PID, dryos ver, diskboot+fi2 encodings should never change between subs, and having some in platform/cam and others in sub is confusing.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4450
Re: Signature verification prior to startup
« Reply #59 on: 09 / December / 2015, 11:10:06 »
Next try.

LED related definitions moved from camera_list.csv to individual platform/(cam)/makefile.inc files, camera_list.csv is now untouched. I have also added a build action to the main makefile to invoke the script from there (this still has to be done manually).
As noted above, I actually tried both the script and a batch build on Windows (Cygwin, because that's what I have currently installed there).
Logged
Pages: 1 2 3 4 5 [6] 7 8 9   Go Up
« previous next »
 

Related Topics