finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #10 on: 30 / December / 2015, 17:05:02 »

Advertisements

Posted updated code with srsa's changes above, plus some other changes.

I took a slightly different approach to handling the RAM data, there are separate adr_range_with_data / adr2ptr_with_data. Not sure if this is needed, but the data range doesn't make sense in contexts like fw_search_insn

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #11 on: 30 / December / 2015, 20:42:46 »

I wouldn't worry much about those possible jump destinations at this point. AFAIK the "old" sigfinder does not handle them either.

Small change to support cams which use yet another special "RAM" region (M3, G5X):

Code: [Select]

--- firmware_load_ng.c.orig
+++ firmware_load_ng.c
@@ -1066,7 +1066,7 @@
                 fptr=pv;
             }
         } else if(!dptr) {
-            if(*pv < fw->base) {
+            if((*pv < fw->base) && (*pv < 0xb0000000)) { // TODO some cams have yet another area with code, avoid that
                 dptr=pv;
             } else {
                 fptr=NULL; // dest address in ROM, reset
@@ -1087,6 +1087,7 @@
         count++;
     }
     // look for BSS init
+    count=0;
     if(fw->data_start) {
         while(fw_disasm_iter(fw) && count < 20) {
             uint32_t *pv=LDR_PC2valptr(fw,fw->is->insn);

Of course this could be done in a more organized way, and perhaps that area (~bfe10800...) could be made supported too.

Logged

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #12 on: 31 / December / 2015, 20:27:11 »

Updated, links in first post. This fixes a nasty stack corruption bug in get_call_const_args. Strangely, this only caused crashes for me with -O2, so I didn't notice until I started testing builds from the CHDK tools directory.

Also added capdis, which is a conversion of captest to the firmware_load_ng code. It currently works pretty much the same, but instead of stopping when it encounters an invalid instruction, it will output the hex values as comments and continue.

Logged

Don't forget what the H stands for.

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #13 on: 01 / January / 2016, 16:19:55 »

I've checked this in. It's far from finished, but having it in the main svn should make development easier.

It's not enabled by default, but the pieces are there to make it run for the same targets as the old sig finder.

If it's not enabled, the rebuild-stubs and batch-rebuild-stubs targets will just skip the thumb2 platforms.

To enable it, you need
* The capstone library + headers
* Set OPT_CAPSTONE_TOOLS=1 in your buildconf or command line
* Set CAPSTONE_TOOLS_INC and CAPSTONE_TOOLS_LINK to specify any compile and linker options needed to use locate the capstone headers and library.
* Make sure there aren't any overrides in the platform makefile.inc disabling stubs generation

I've uploaded a windows mingw build of the library to https://app.box.com/s/sshu7dv0mebvnee6gvpsex46y2m18w51
(edit: updated Jan 11 2016)

This works with the chdkshell toolchain, just extract the files somewhere (e.g. c:/capstone) and set:
CAPSTONE_TOOLS_INC=-Ic:/capstone/include
CAPSTONE_TOOLS_LINK=-Lc:/capstone/lib -lcapstone

From my limited testing, makefile changes shouldn't interfere with chdkshell. The OPT_CAPSTONE_TOOLS option shows up as a checkbox. However, the INC/LINK variables need to be set manually to use it (I avoided naming them OPT_* because chdkshell turns those into checkboxes). If someone wants to include the above library package with chdkshell, this could be automated.

For linux, you should be able to just download the source and configure/make/make install. Some distros also include capstone packages, but the 3.0.4 changelog says "ARM * Fixed a memory corruption bug on IT instruction." so you might want to make sure it's the latest version if you use a distro package.

I have not tested any of this on linux or with 64 bit toolchains. I wouldn't be surprised if there are some 64-bit related issues in the code I've written.

« Last Edit: 12 / January / 2016, 02:21:44 by reyalp »

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #14 on: 01 / January / 2016, 18:30:13 »

Quote from: reyalp on 01 / January / 2016, 16:19:55

I have not tested any of this on linux or with 64 bit toolchains.

Can't test 64 bit, but 32 bit linux is OK.
A suggestion: the convention used by the other sigfinders is to append "_FW" to the eventproc names. I'd consider doing the same to avoid confusion caused by the 2 different parameter passing methods.
Would it be possible to add a framework to support string-based sigfinder methods - something similar to "Signature matching data" and the related functions in finsig_dryos?

Logged

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #15 on: 01 / January / 2016, 18:54:20 »

Quote from: srsa_4c on 01 / January / 2016, 18:30:13

Can't test 64 bit, but 32 bit linux is OK.

Thanks.

Quote

A suggestion: the convention used by the other sigfinders is to append "_FW" to the eventproc names. I'd consider doing the same to avoid confusion caused by the 2 different parameter passing methods.

Agreed, I plan to do that.

Quote

Would it be possible to add a framework to support string-based sigfinder methods - something similar to "Signature matching data" and the related functions in finsig_dryos?

Yeah, this is the plan, I just haven't got to it yet, still trying to understand the old code and figure out which parts I can re-use. I don't plan to implement the old style (gensig) matches, but some kind of table of names and methods like string_sig

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #16 on: 01 / January / 2016, 19:05:11 »

Quote from: srsa_4c on 01 / January / 2016, 18:30:13

Quote from: reyalp on 01 / January / 2016, 16:19:55
I have not tested any of this on linux or with 64 bit toolchains.
Can't test 64 bit, but 32 bit linux is OK.

Builds okay on Fedora 23 64 bit. Can't test if the result runs on a camera though.

Seems to build okay for the SX280HS 102b with the PRIMARY.BIN file installed and the override lines in platform/sx280hs/sub/102b/makefile.inc removed.

I just used the distro's package for the capstone stuff as it seems to be the latest version.

Generated stubs_entry.S and stubs_auto.S attached.

I noted this in the build output - not sure if it matters :

Code: [Select]

>> Entering to ./platform/sx280hs/sub/102b
boot.c -> boot.o
stubs_min.S -> stubs_min.o
stubs_auto.S -> stubs_auto.o
-> stubs_entry.S
failed to get export/register eventproc args at 0xfc1a7ef6
failed to get CreateTask args at 0xfc1abec8
failed to get export/register eventproc args at 0xfc29abce
failed to get RegisterEventProcTable arg 0x00000000 at 0xfc29b0e8
task name name not string at 0xfc29ea54
task name name not string at 0xfc29f0ac
failed to get RegisterEventProcTable arg 0x00000000 at 0xfc3b81e2
Time to generate stubs 4.13 seconds
stubs_entry.S -> stubs_entry.o
lib.c -> lib.o
stubs_entry_2.S -> stubs_entry_2.o

stubs_entry.S (13.86 kB - downloaded 47 times.)

stubs_auto.S (2.79 kB - downloaded 45 times.)

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #17 on: 01 / January / 2016, 19:14:39 »

Quote from: waterwingz on 01 / January / 2016, 19:05:11

Builds okay on Fedora 23 64 bit. Can't test if the result runs on a camera though.

Thanks. stubs_entry is identical to what I get, so it should be fine.

Quote

I noted this in the build output - not sure if it matters :

Those are normal, debug coding noting calls where the function names/addresses couldn't be found.

Quote

Time to generate stubs 4.13 seconds

Out of curiosity, what kind of CPU was this on? It takes almost 10 sec on my i5-2400 windows machine, but I suspect capstone may be significantly slower when built as 32 bit.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #18 on: 01 / January / 2016, 23:43:50 »

Quote from: reyalp on 01 / January / 2016, 19:14:39

Out of curiosity, what kind of CPU was this on? It takes almost 10 sec on my i5-2400 windows machine, but I suspect capstone may be significantly slower when built as 32 bit.

It's a six year old AMD Athlon II X4 640 Processor running at 3G. FWIW all the Linux stuff is on a SSD - just not the build directory where the CHDK source and toolchain reside.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: finsig and other tools for thumb2 (was Re: chdk in the DIGIC6 world)

« Reply #19 on: 02 / January / 2016, 04:34:31 »

In trunk 4323
- Automatically found eventrpocs have _FW at the end of the name.
- If the address registered for the eventproc just jumps to another function, it's added with j_ and the target is used for the _FW function.
- There is a now a structure sig_rules somewhat similar to string_sigs in finsig_dryos. Right now it's all eventproc and task lookups, but adding other functions should be straightforward.

I tried to verify all the functions in sig_rules against what is actually being used in the sx280 102b port, although I might have missed some. The remaining differences are mostly
1) Sig finder using a RAM address while existing stubs go through a veneer
2) Existing stubs using what sig finder turns into a j_...
~~There should be quite a few more functions that can be added just through renaming (e.g. things like SS.MFOn)~~ edit: I went through and did some of these in 4324 and 4326

This is all still very rough.

I have only been testing with sx280 and g7x dumps at the moment.

@waterwingz:
Thanks. I'll have to try compiling a 64 bit version. SSD shouldn't make too much difference for the time reported by the sig finder, all the work is done in RAM. Running on the G7x dump takes 14 seconds.

« Last Edit: 03 / January / 2016, 16:23:58 by reyalp »

Logged

Don't forget what the H stands for.