finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

reyalp

14002

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #50 on: 06 / August / 2017, 16:49:51 »

Advertisements

Quote from: srsa_4c on 06 / August / 2017, 11:15:16

Attached patch adds the ability to identify the Zico blobs (and the function that references them). It's not finished, because I wasn't sure how the info about those blobs should be stored. There's already an array that keeps blob properties, but that seems mostly for blobs that can actually be used for identifying stuff (some utility functions use that array).
The goal would be a list of blobs (and their known properties), extending the existing list in stubs_entry.S. Some software (khm) could then read that list and let the user easily import the blobs appearing there.
Some other blobs might be added later - second ARM, audio/video codec blob(s), etc.

Nice work. I've thought about doing something along these lines but hadn't got around to it.

The adr_range stuff is currently identified by firmware_load_ng, as part of the minimal information needed to work with the firmware. I'd lean toward keeping stuff identified by finsig_thumb2 in a separate structure. It be a new structure for this purpose, or just added to misc vals.

I've also been thinking that stuff we have as comments in stubs_entry.S would be more convenient as a separate, easy to parse "firmware attributes" file, but I haven't really thought it through.

Quote

Can you take a look at it and spot if I'm doing anything incorrectly? Those two iter states are somewhat confusing (there's one in fw and there's also 'is').

Yeah. Having fw->is was probably a mistake on my part. General rule, always use the "is" passed to the match function (and use functions that take an explicit "is" argument), unless you need a second one. fw->is is basically used so functions that need to step a few instructions don't need to deal with allocating / freeing a new state. (Another mistake of mine: Using "is" as a variable name makes documentation confusing

)

I'll take a closer look at the code a bit later, but for now it seems to work correctly on all the d6 dumps I have.

Logged

Don't forget what the H stands for.

reyalp

14002

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #51 on: 08 / August / 2017, 01:48:51 »

Looking more at the code, it makes sense to me. The use of fw->is makes sense for backtracking, since you need the history from the other state.

I would probably break this up into several matches. I'm just posting this in case it's helpful, you don't have to do it.

1) Find zicokick_start. You could handle handle the firmware variations using version flags SIG_DRY_MIN/SIG_DRY_MAX and two different, simpler match functions (assuming the variation corresponds to dryos versions, which I think it does).
2) Another to find the values, with zicokick_start sig as the reference

Breaking it up like this makes it easier to handle variations, because #1 can be version specific while #2 isn't (the GraphicSystemCoreFinish is done like this).

If you identified the copy function (as another match, between #1 and #2), you could probably just use find_next_sig_call and get_call_const_args to pick up the values.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #52 on: 08 / August / 2017, 16:15:16 »

Thanks for the explanation and suggestions. I'll see what I can come up with.

Logged

reyalp

14002

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #53 on: 09 / August / 2017, 02:32:46 »

Here's my take at splitting up the matches. I only tested for g7x and sx280. I noticed sx280 only picks up two blobs, but the same is true for your earlier code.

My earlier comment "you could probably just use find_next_sig_call and get_call_const_args to pick up the values." was wrong, get_call_const_args doesn't understand things like

Code: [Select]

ldr	r0, =0xfcabcabc
ldr	r0, [r0]

It could in theory, and it would be nice if it did, but difficult since it goes backwards.

finsig_find_zico_blobs_partial-reyalp-1.diff (6.75 kB - downloaded 30 times.)

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #54 on: 12 / August / 2017, 12:07:39 »

Quote from: reyalp on 09 / August / 2017, 02:32:46

Here's my take at splitting up the matches. I only tested for g7x and sx280. I noticed sx280 only picks up two blobs, but the same is true for your earlier code.

The sx280 doesn't seem to have a valid 0xbff00000 blob, so that's ok. The code seems to be working. I did not go further, because I have a feeling you may prefer to store the blob properties in misc_vals (and that's not trivial).

I also tested the blob finder on the D7 dumps I have and found that it also works there.
The sigfinder's usability on D7 dumps can be improved a lot, just by restricting the search for the ctypes array. The reason it currently fails to find most stuff is because the D7 bootloader now has its own instance of DryOS, along with a copy of ctypes. Something like this in find_ctypes() appears to be enough:

Code: [Select]

-    if (k < (fw->size8 - sizeof(ctypes)))
+    if ((k < (fw->size8 - sizeof(ctypes))) && (k > (fw->main_offs)))

Logged

reyalp

14002

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #55 on: 13 / August / 2017, 20:37:47 »

Quote from: srsa_4c on 12 / August / 2017, 12:07:39

The sx280 doesn't seem to have a valid 0xbff00000 blob, so that's ok. The code seems to be working. I did not go further, because I have a feeling you may prefer to store the blob properties in misc_vals (and that's not trivial).

I've checked in some code for this. Right now it outputs the blobs in a similar format to the ROM / RAM regions. If you want a different format, that's fine. Right now they are just labeled zico_N, but it would probably be better to identify code, data etc.

Quote

Something like this in find_ctypes() appears to be enough:

Thanks for that. I verified it didn't affect any of the digic 6 stubs and checked it in.

« Last Edit: 19 / August / 2017, 18:45:33 by reyalp »

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #56 on: 20 / August / 2017, 12:49:30 »

Quote from: reyalp on 13 / August / 2017, 20:37:47

I've checked in some code for this. Right now it outputs the blobs in a similar format to the ROM / RAM regions. If you want a different format, that's fine.

Thanks. The format is OK, I already made a parser for it.

Quote

Right now they are just labeled zico_N, but it would probably be better to identify code, data etc.

I suspect that the two special RAM areas may have fixed role - would not be unusual for Xtensa. In known cases, 0xbff20000 is executable, 0xbff00000 is data only, 0x80a00000 is also executable.

Excerpt from one of the published overlays (cpu3400):

Code: [Select]

/*  Instruction RAM 0:  */
#define XCHAL_INSTRAM0_VADDR		0x5FFF8000
#define XCHAL_INSTRAM0_PADDR		0x5FFF8000
#define XCHAL_INSTRAM0_SIZE		32768
#define XCHAL_INSTRAM0_ECC_PARITY	0

/*  Data RAM 0:  */
#define XCHAL_DATARAM0_VADDR		0x5FFE8000
#define XCHAL_DATARAM0_PADDR		0x5FFE8000
#define XCHAL_DATARAM0_SIZE		32768
#define XCHAL_DATARAM0_ECC_PARITY	0

/*  Data RAM 1:  */
#define XCHAL_DATARAM1_VADDR		0x5FFF0000
#define XCHAL_DATARAM1_PADDR		0x5FFF0000
#define XCHAL_DATARAM1_SIZE		32768
#define XCHAL_DATARAM1_ECC_PARITY	0

I'm now trying to add a similar information block for finsig_dryos.

Logged

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #57 on: 24 / January / 2018, 14:09:13 »

I found a "lost" event procedure table while browsing for something else. Attached patch adds support for the kind of code pattern involved.
Patched sigfinder finds the ShutterTool.* event procedures on cameras that have them (approx. r54 and above).
As usual, my code is likely not the most optimal.

find_more_eventprocs_t2_180124.diff (2.96 kB - downloaded 28 times.)

Logged

reyalp

14002

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #58 on: 25 / January / 2018, 00:44:40 »

Quote from: srsa_4c on 24 / January / 2018, 14:09:13

I found a "lost" event procedure table while browsing for something else. Attached patch adds support for the kind of code pattern involved.
Patched sigfinder finds the ShutterTool.* event procedures on cameras that have them (approx. r54 and above).

Nice. Checked in r4980.

Quote

As usual, my code is likely not the most optimal.

I used adr_hist_get to go back 10 instructions rather than manipulating the address. No difference in this case but less likely to get out of sync.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #59 on: 25 / January / 2018, 16:40:42 »

Quote from: reyalp on 25 / January / 2018, 00:44:40

Checked in r4980.

Thanks.

I'm having problems with physw_bits.txt files in svn. They do not have the eol-style svn property set, and show up as changed whenever the sigfinder is executed on a D6 port. Since fixing this may involve more than just setting the properties and re-uploading the files, I'd rather not attempt doing it myself. This isn't urgent, just mentioning it.

Logged