supplierdeeply

finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

  • 65 Replies
  • 23493 Views
*

Offline reyalp

  • ******
  • 11392
Advertisements
Here's a little tool I wrote using capstone. The main purpose was just to learn how to use the framework and get a better understanding of thumb2, it's not a finished porting tool and probably has significant bugs. I'm just posting it here in case anyone is interested. Once I've got a better understanding I'll work more on something that can be integrated into the CHDK development process.

It loads a firmware and disassembles a specified range of address, e.g.
Code: [Select]
./captest.exe -c=128 -o=0x20000 ../dumps/g7x/sub/100d/PRIMARY.BIN 0xfc000000
Loads the dump at 0xfc000000, and disassembles up to 128 instructions starting at  0xfc020000. Note the "up to" is because, as used in this code, capstone stops when it hits an invalid instruction. As such, it's not suitable for disassembling entire dumps, it's for extracting snippets more like code-gen.

There are a bunch of options. By default
loc_xxx labels are generated for B instructions whose target is in the dissassembled code, sub_xxx are generated for BL, and  LDR Rn, [pc, #x] converted to LDR Rn, =0x.... These can be turned off with -noloc, -nosub and -noconst, respectively.

The default output is generic asm listing. Using -f=chdk quotes the output suitably for use as CHDK inline asm.

The various -d options add commented output about the disassembly.

-f=objdump -d-const -d-addr -d-bin -noconst

outputs something similar to what is produced by thumb2dis.pl, for easier verification of the disassembly. Note, there are some fairly significant differences.

source (needs capstone library to compile)

windows binary (think it will run standalone without any additional DLLs, but not certain)

edit:
box folder with all the files:
https://app.box.com/reyalp-t2-tools

latest source is now in CHDK 1.5 svn

CHDKShell toolchain / mingw compatible capstone library https://app.box.com/s/k3r6vmm2doqbkg6mblhjxlni7sckbq12
« Last Edit: 01 / January / 2016, 16:26:07 by reyalp »
Don't forget what the H stands for.

*

Offline srsa_4c

  • ******
  • 3654
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #1 on: 23 / December / 2015, 19:48:10 »
Here's a little tool I wrote using capstone.
Interesting, thanks for doing this.

*

Offline reyalp

  • ******
  • 11392
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #2 on: 24 / December / 2015, 03:02:12 »
Quote
probably has significant bugs
As indeed it does, I got the code_size and count arguments to cs_disasm wrong. So -c was the maximum number of bytes to disassemble  ::)

Updated
edit: see top post for links
also now accepts start and end addresses instead of offset and count.
« Last Edit: 27 / December / 2015, 01:39:44 by reyalp »
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 11392
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #3 on: 27 / December / 2015, 01:54:12 »
I decided to split this into it's own thread.

I've been working on a thumb2 sig finder based on the tool posted earlier. This is VERY preliminary and not really useful yet, but it does find ~20 eventproc stubs correctly for sx280 ;)

Right now it's a mix of very butchered finsig / firmware_load code and capstone code. The fact that thumb2 breaks the word=>instruction assumptions makes it messy, and I haven't really settled on where to use firmware address, pointers, offsets...  Doing it all with capstone is also quite slow, although I think there's some room for optimization.

I have also updated the captest tool to deal with thumb ADR instructions.

I'll keep links to the latest files in the first post.

edit:
To build with the patch, you should set something like the following in your localbuildconf.inc if capstone isn't in your default compiler paths
CAPSTONE_LINK=-L/path/to/capstone-3.0.4 -lcapstone
CAPSTONE_INC=-I/path/to/capstone-3.0.4/include

You can also override the finsig location, like
FINSIG_EXE=/path/to/finsig_thumb2.exe
« Last Edit: 27 / December / 2015, 02:06:05 by reyalp »
Don't forget what the H stands for.


*

Offline srsa_4c

  • ******
  • 3654
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #4 on: 28 / December / 2015, 18:09:59 »
I've been working on a thumb2 sig finder based on the tool posted earlier. This is VERY preliminary and not really useful yet
Thanks (again) for starting this.

I tried it on some dumps, and found that a number of eventproc's are left unrecognized (I don't see a clear pattern which ones, perhaps mostly those where subw supplies an argument to ExportToEventProcedure). Tried modifying the part of find_event_procs() that appears to search in instruction history (increased the depth to 3). Instead of more hits, I got significantly less, and those hits seem to be some of those that are not found by the untouched source.
I know it's a really early state of the code, just wanted to give some feedback.

*

Offline reyalp

  • ******
  • 11392
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #5 on: 28 / December / 2015, 20:55:49 »
I know it's a really early state of the code, just wanted to give some feedback.
Thanks.

I've uploaded updated versions. It now finds a lot more eventprocs, and also many tasks. The searching / matching / backtracking code is also more general, and fixes some bugs in the previous version.

It's still big mess and probably has a lot of bugs, but I think it's to the point of being somewhat useful.

captest is updated to handle some additional ADR-like instructions (e.g. subw/addw...). I use captest -d a lot to see how capstone disassembles the instructions.
Don't forget what the H stands for.

*

Offline srsa_4c

  • ******
  • 3654
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #6 on: 29 / December / 2015, 20:43:19 »
I've uploaded updated versions. It now finds a lot more eventprocs, and also many tasks. The searching / matching / backtracking code is also more general, and fixes some bugs in the previous version.
Very nice. 894 entries in the csv for the sx280 after fixing 2 c&p bugs in B_BL_target and B_BL_BLXimm_target.

*

Offline reyalp

  • ******
  • 11392
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #7 on: 29 / December / 2015, 22:21:32 »
I've uploaded updated versions. It now finds a lot more eventprocs, and also many tasks. The searching / matching / backtracking code is also more general, and fixes some bugs in the previous version.
Very nice. 894 entries in the csv for the sx280 after fixing 2 c&p bugs in B_BL_target and B_BL_BLXimm_target.
oops, thanks for catching those :)

I've uploaded a new version. This doesn't change matching much (other than the bugs you caught), but the more generic code is now split into firmware_load_ng.c and better organized / documented. This isn't anywhere close to "done" but it's a step in the right direction.

I called it firmware_load_ng ("next generation", or maybe "no good" ;)) rather than thumb2, because I plan make some of it usable with old style firmwares too. Specifically, I'm going to rework captest to use it, so it can use more of the same code.

Unless there are objections, I will put this stuff in the trunk (minus the makefile changes to make it run for thumb2 ports) soon to make it easier to collaborate.

I'm not sure what to do about the capstone dependency. I'm hesitant to dump that much code into our tools source, although it's BSD licensed so technically we could.
Don't forget what the H stands for.


*

Offline srsa_4c

  • ******
  • 3654
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #8 on: 30 / December / 2015, 12:34:25 »
I'm not sure what to do about the capstone dependency. I'm hesitant to dump that much code into our tools source, although it's BSD licensed so technically we could.
That library is huge and has lots of unrelated parts (non-ARM).
I'd consider making it an optional requirement: if it can't be found, thumb-2 tools won't be built.
I don't know if we'll ever need to patch it. If we do, we could host either a patch or a forked version - off-tree.

Here's a case that the current sigfinder can't handle (sx280 102b):
(LDR instructions are used instead of ADR)
Code: [Select]
reconstructed:

fc3b81e0:   483f        ldr r0, =0xfc71a610
fc3b81e2:   f6e2 bcf0   b.w loc_fc29abc6    ; <register_eventproctable>
loc_fc3b81e6:
fc3b81e6:   483e        ldr r0, =0xfc71a610
fc3b81e8:   f6e2 bcf8   b.w loc_fc29abdc    ; <unregister_eventproctable>

disassemblev7

fc3b81d8:   2000        movs    r0, #0
fc3b81da:   bd10        pop {r4, pc}
fc3b81dc:   a5f8        add r5, pc, #992    ; 0xfc3b85c0: (f5e6d105)...sub?
fc3b81de:   fc71 483f   ldc2l   8, cr4, [r1], #-252
fc3b81e2:   f6e2 bcf0   b.w loc_fc29abc6    ; <register_eventproctable>
loc_fc3b81e6:
fc3b81e6:   483e        ldr r0, [pc, #248]  ; 0xfc3b82e0: (fc71a610)
fc3b81e8:   f6e2 bcf8   b.w loc_fc29abdc    ; <unregister_eventproctable>

capstone (captest -adrldr)

fc3b81d8:   00 20        movs    r0, #0
fc3b81da:   10 bd        pop     {r4, pc}
fc3b81dc:   f8 a5        ldr     r5, =0xfc3b85c0
fc3b81de:   71 fc 3f 48  ldc2l   p8, c4, [r1], #-0xfc
fc3b81e2:   e2 f6 f0 bc  b.w     loc_fc29abc6
fc3b81e6:   3e 48        ldr     r0, =0xfc71a610
fc3b81e8:   e2 f6 f8 bc  b.w     loc_fc29abdc
We may need a way to recognize bogus instructions (ldc2l here) and restart parsing upon encountering them.
Or, should we just ignore these cases and track "unregister_eventproctable" calls instead?

Attached is a modified version of "thumb2_tools-src-2015-12-29_1", finds 1412 functions in the sx280 dump
Changes:
- CreateTask backtrack depth 8 -> 10, finds one more task in the sx280 fw
- handle preinited data in RAM
- handle ADD in get_call_const_args() - you may not like the way it's done
« Last Edit: 30 / December / 2015, 12:39:58 by srsa_4c »

*

Offline reyalp

  • ******
  • 11392
Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)
« Reply #9 on: 30 / December / 2015, 14:34:28 »
That library is huge and has lots of unrelated parts (non-ARM).
I'd consider making it an optional requirement: if it can't be found, thumb-2 tools won't be built.
I don't know if we'll ever need to patch it. If we do, we could host either a patch or a forked version - off-tree.
Yeah, that seems like the way to go. I can post a pre-built windows lib  for use chdkshell.

Quote
Here's a case that the current sigfinder can't handle (sx280 102b):
(LDR instructions are used instead of ADR)
Code: [Select]
reconstructed:

fc3b81e0:   483f        ldr r0, =0xfc71a610
fc3b81e2:   f6e2 bcf0   b.w loc_fc29abc6    ; <register_eventproctable>
loc_fc3b81e6:
fc3b81e6:   483e        ldr r0, =0xfc71a610
fc3b81e8:   f6e2 bcf8   b.w loc_fc29abdc    ; <unregister_eventproctable>

disassemblev7

fc3b81d8:   2000        movs    r0, #0
fc3b81da:   bd10        pop {r4, pc}
fc3b81dc:   a5f8        add r5, pc, #992    ; 0xfc3b85c0: (f5e6d105)...sub?
fc3b81de:   fc71 483f   ldc2l   8, cr4, [r1], #-252
fc3b81e2:   f6e2 bcf0   b.w loc_fc29abc6    ; <register_eventproctable>
loc_fc3b81e6:
fc3b81e6:   483e        ldr r0, [pc, #248]  ; 0xfc3b82e0: (fc71a610)
fc3b81e8:   f6e2 bcf8   b.w loc_fc29abdc    ; <unregister_eventproctable>

capstone (captest -adrldr)

fc3b81d8:   00 20        movs    r0, #0
fc3b81da:   10 bd        pop     {r4, pc}
fc3b81dc:   f8 a5        ldr     r5, =0xfc3b85c0
fc3b81de:   71 fc 3f 48  ldc2l   p8, c4, [r1], #-0xfc
fc3b81e2:   e2 f6 f0 bc  b.w     loc_fc29abc6
fc3b81e6:   3e 48        ldr     r0, =0xfc71a610
fc3b81e8:   e2 f6 f8 bc  b.w     loc_fc29abdc
We may need a way to recognize bogus instructions (ldc2l here) and restart parsing upon encountering them.
Or, should we just ignore these cases and track "unregister_eventproctable" calls instead?
Yeah, I've been thinking about ways to recognize instructions that disassemble but aren't "reasonable".

OTOH, maybe it's not so important to worry about finding every case. We can still do other kinds of matches for the functions we actually need.

edit:
The other part of this is recognizing branches that land somewhere in the backtracking range. I actually added code to stop backtracking when an unconditional branch is hit, on the assumption that it means execution when somewhere else. However, in cases like this, you would get the right result ignoring the branch.

Quote
Attached is a modified version of "thumb2_tools-src-2015-12-29_1", finds 1412 functions in the sx280 dump
Changes:
- CreateTask backtrack depth 8 -> 10, finds one more task in the sx280 fw
- handle preinited data in RAM
Thanks. I added ram data support to my source last night but didn't bother to post again. Good reason to have it in svn.

Quote
- handle ADD in get_call_const_args() - you may not like the way it's done
Hmm, that's clever, I was thinking I would have to iterate forward, which would be better in some other ways too, but needs more re-writing.

I've been thinking about making a general "track the register contents we can figure out from <address> forward". This could be a flag in disasm_iter... though conditional execution makes it very hard to be confident.
« Last Edit: 30 / December / 2015, 16:18:52 by reyalp »
Don't forget what the H stands for.

 

Related Topics