finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

reyalp

13880

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #60 on: 25 / January / 2018, 22:07:28 »

Advertisements

Quote from: srsa_4c on 25 / January / 2018, 16:40:42

Since fixing this may involve more than just setting the properties and re-uploading the files, I'd rather not attempt doing it myself. This isn't urgent, just mentioning it.

Thanks for mentioning it. Normally all you should have to do is set the prop and commit. I've did this for physw_bits.txt, and added it to platprops.bash so future ports should get set correctly.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #61 on: 02 / February / 2018, 12:35:47 »

Attached patch lets the sigfinder find more eventproc tables. The funcs_by_*.csv diffs are part of the patch so they can be seen without rebuilding stubs.
I'd like to note that some review/work might be required before this can be checked in, because
- I reused some existing routines that required me to add two fw function aliases (j_regevproctable, j_unregevproctable). One of those aliased functions is already found as j_UnRegisterEventProcTable (and the addresses match), the other (j_RegisterEventProcTable) is not recognized by existing code for some reason.
To find j_regevproctable and j_unregevproctable, I used a pattern matching method that may or may not be supported by existing routines.
- Some of the comments I left are not intended to stay.
- I was planning to verify a blx call, but I have not been able to get the address of j_dry_memcpy (it does appear in the csv files, so I'm not sure what's going on). The routine does work without that verification, though.

find_more_eventproctables_d6_180202.diff (451.34 kB - downloaded 26 times.)

Logged

reyalp

13880

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #62 on: 04 / February / 2018, 01:14:39 »

Quote from: srsa_4c on 02 / February / 2018, 12:35:47

Attached patch lets the sigfinder find more eventproc tables. The funcs_by_*.csv diffs are part of the patch so they can be seen without rebuilding stubs.

Nice. I ended up re-working this quite a bit and checking it in as r4990

I changed the names to RegisterEventProcTable_alt / UnRegisterEventProcTable_alt, since the j_ prefix is used for auto-detected veneers.

I moved the code to find these into sig_rules_initial. I also moved the dry_memcpy match there, using a different string match that doesn't depend on anything else.

The reason for this is to allow the new tables to be found in find_generic_funcs. The original patch added a second full firmware disassembly pass, which roughly doubles the run time. Doing it in find_generic_funcs also means that normal matches (in sig_rules_main) can use any of the new eventprocs as references.

Quote

- I was planning to verify a blx call, but I have not been able to get the address of j_dry_memcpy (it does appear in the csv files, so I'm not sure what's going on).

I think there was a bug, looking for the target at "is" instead of "fw->is". I re-added this check.

Some of the remaining non-detected eventprocs are due to literal pools in the middle of long eventproc registration functions e.g.
ldr r1, some_func,
b loc_abc
...
loc_abc:
ldr r0, "some_func"
bl RegisterEventProc

I have some ideas for picking those up, but it's not simple and the number of functions affected is fairly small so I haven't done anything with it.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #63 on: 04 / February / 2018, 16:25:39 »

Quote from: reyalp on 04 / February / 2018, 01:14:39

I ended up re-working this quite a bit and checking it in as r4990

Thanks.

Quote

The original patch added a second full firmware disassembly pass, which roughly doubles the run time.

I'll try to avoid doing that in the future (now that I know that it's that expensive).

Quote

Quote
- I was planning to verify a blx call, but I have not been able to get the address of j_dry_memcpy (it does appear in the csv files, so I'm not sure what's going on).
I think there was a bug, looking for the target at "is" instead of "fw->is".

Since my idea wasn't working, I started printing the address of j_dry_memcpy to stdout. I became confused when I found that for most of the time, that address was returned as zero, and I left those two lines in the code as they were, commented.
I've had a lot more trouble with stuff like how history works, and which function does what with the iterators. At least I gathered some experience at the end.

Logged

reyalp

13880

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #64 on: 04 / February / 2018, 17:48:32 »

Quote from: srsa_4c on 04 / February / 2018, 16:25:39

I've had a lot more trouble with stuff like how history works, and which function does what with the iterators.

Yeah, it's not very clear. I'm happy to try to answer questions, though I understand sometimes you just need to work in the code to get it.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #65 on: 25 / March / 2018, 07:04:09 »

I have a suggestion for a capdis feature:
When disassembling a function partially, it would be nice if capdis could (optionally) add an LDR PC instruction at the end that would point to the next instruction at the routine's original location.

Logged

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #66 on: 02 / November / 2018, 19:45:58 »

Attached is the thumb2 version of the propcase/propset finder addition.
It seems to work, but its code quality can surely be improved.
It uses firmware debug strings to identify some known and some guessed propcases.
Both the finsig_dryos and this version seem to indicate that one of the propcases (PROPCASE_BV) is incorrect in propset8.h - blackhole's note lists the correct one as a possible candidate.

find_props_t2_v1.diff (10.14 kB - downloaded 25 times.)

Logged

reyalp

13880

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #67 on: 03 / November / 2018, 22:38:16 »

Quote from: srsa_4c on 02 / November / 2018, 19:45:58

Attached is the thumb2 version of the propcase/propset finder addition.
It seems to work, but its code quality can surely be improved.
It uses firmware debug strings to identify some known and some guessed propcases.
Both the finsig_dryos and this version seem to indicate that one of the propcases (PROPCASE_BV) is incorrect in propset8.h - blackhole's note lists the correct one as a possible candidate.

Thanks for doing this

Committed in r5110. ~~I may try to reduce the duplication with sig_match_near_str but it's fine for now~~ done.

For propset 8 BV, I think if we have a string match that's known to be right on other versions, it's safe to change it.

« Last Edit: 03 / November / 2018, 23:50:23 by reyalp »

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #68 on: 04 / November / 2018, 12:43:15 »

Quote from: reyalp on 03 / November / 2018, 22:38:16

Committed in r5110. ~~I may try to reduce the duplication with sig_match_near_str but it's fine for now~~ done.

Thanks for sanitizing the code. I found some mistakes of mine and corrected them in r5114.

Quote

For propset 8 BV, I think if we have a string match that's known to be right on other versions, it's safe to change it.

I did that in changeset 5115. I included the re-generated stubs files in the same commit due to their small number.

Logged

reyalp

13880

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #69 on: 10 / November / 2018, 17:46:38 »

Quote from: srsa_4c on 04 / November / 2018, 12:43:15

I included the re-generated stubs files in the same commit due to their small number.

That's fine with me. The main reason I like to split sig finder changes and rebuilding stubs is in case something needs to be merged back to the stable branch, but the thumb2 stuff only exists in 1.5 so it's not a concern.

Logged

Don't forget what the H stands for.