New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

srsa_4c

4451

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #80 on: 11 / July / 2013, 17:11:52 »

Advertisements

Quote from: philmoz on 09 / July / 2013, 06:37:19

I've merged your new match_strsig115 back into match_strsig15

That was my intention, but I thought it would be better to let you decide.

Quote

The code will try to find a dependency if it hasn't already been found; but only with the new string matching functions - if the dependency requires matching using the old gensig matching then it will fail unless the order is correct in func_names.

I see. In that case, my added comments about strict order are not needed, and those functions could be better ordered in the output (alphabetical or functional).

Logged

srsa_4c

4451

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #81 on: 18 / July / 2013, 19:13:29 »

Looks like I'm not done yet

Interrupt related functions: RegisterInterruptHandler, UnregisterInterruptHandler, GetSRAndDisableInterrupt, SetSR, EnableInterrupt. All names are based on the usual ixus30 naming scheme. The first 2 functions are wrappers around their "lower" equivalents (which are also used by some fw routines) - "lower" is also part of that old naming scheme. The last 3 functions are thunks (?), all 3 consists of a single branch to the actual function.

dryos_interrupt_related_funcs_.diff (2.24 kB - downloaded 26 times.)

Logged

reyalp

14110

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #82 on: 05 / August / 2013, 16:32:54 »

Phil asked about alt raw buffer on the the d20:

I searched for all instances of the known raw buffer. From that I found a table similar to what the sig finder finds as raw_buffers for sx260hs.

To active_raw_buffer, I look for a function with asserts using the same file name and similar line numbers to the corresponding sx260 function noted by the sig finder. Since the code was very similar, I assumed the values of the variable were equivalent.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #83 on: 19 / August / 2013, 18:45:22 »

PTM property (we may need a better naming scheme) functions for DryOS, plus the more or less related CreateController, DeleteController. "Controllers" are handling input from the user interface (buttons, etc), and are probably responsible for their visual representation too (I guess).

A question:
Would it be possible for the sigfinder to output constants which then can be used in the loader?

edit:
@philmoz
Please don't add this patch yet as it finds the real functions (i.e. it follows the first branch instruction). The firmware is using their 'wrapped' version, it would probably be better to identify those instead.

ptmprop_and_controller_functions.diff (3.56 kB - downloaded 22 times.)

« Last Edit: 21 / August / 2013, 16:48:11 by srsa_4c »

Logged

philmoz

3450

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #84 on: 23 / August / 2013, 20:36:24 »

Quote from: srsa_4c on 19 / August / 2013, 18:45:22

PTM property (we may need a better naming scheme) functions for DryOS, plus the more or less related CreateController, DeleteController. "Controllers" are handling input from the user interface (buttons, etc), and are probably responsible for their visual representation too (I guess).

A question:
Would it be possible for the sigfinder to output constants which then can be used in the loader?

You could add stubs_entry.S to the loader build with a different version of stubs_asm.h that did not generate any code for the NHSTUB and STUB macros. This would just leave you with the DEF values you can reference in the loader code. DEF values could be added for the id and version string addresses.

However there is only one loader directory for each camera so you would have to make sure the autobuild worked correctly for all firmware versions.

I'm also not sure how you would make this work for the cases where one CHDK version is used on multiple firmware versions - the firmware version string check would fail.

Quote

edit:
@philmoz
Please don't add this patch yet as it finds the real functions (i.e. it follows the first branch instruction). The firmware is using their 'wrapped' version, it would probably be better to identify those instead.

Not sure why this is an issues - the point of following the branch is to bypass an unnecessary instruction when calling the code from CHDK.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

srsa_4c

4451

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #85 on: 24 / August / 2013, 09:08:46 »

Quote from: philmoz on 23 / August / 2013, 20:36:24

Not sure why this is an issues - the point of following the branch is to bypass an unnecessary instruction when calling the code from CHDK.

To be honest, most of my additions to finsig_dryos are merely meant to identify functions that help understanding parts of the disassembled firmware. The firmware references the event procedure (="wrapped") versions of the PTM functions, the unwrapped versions are barely used.

Another question:
There are even more fw functions that could be useful for reverse engineering / identification purposes, but won't ever be used by CHDK. The sigfinder and stubs_entry.S starts to get a bit cluttered by these - do you think it's OK to add more of these 'UNUSED' stubs? If not, can you recommend a better solution?

Logged

philmoz

3450

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #86 on: 24 / August / 2013, 17:47:35 »

Quote from: srsa_4c on 24 / August / 2013, 09:08:46

Quote from: philmoz on 23 / August / 2013, 20:36:24
Not sure why this is an issues - the point of following the branch is to bypass an unnecessary instruction when calling the code from CHDK.
To be honest, most of my additions to finsig_dryos are merely meant to identify functions that help understanding parts of the disassembled firmware. The firmware references the event procedure (="wrapped") versions of the PTM functions, the unwrapped versions are barely used.

Another question:
There are even more fw functions that could be useful for reverse engineering / identification purposes, but won't ever be used by CHDK. The sigfinder and stubs_entry.S starts to get a bit cluttered by these - do you think it's OK to add more of these 'UNUSED' stubs? If not, can you recommend a better solution?

It's probably worth generating a new file with all the function names and addresses (and without the macros) for reverse engineering. The unused functions can then be left out of stubs_entry.S.

Phil.

Edit: Try the attached patch.
This has the following changes:
- Unused functions are not written to stubs_entry.S
- Two new files are generated that contain all function names & addresses found, 'funcs_by_name.txt' is sorted by function name and 'funcs_by_address.txt' is sorted by ROM address.
- code_gen.c updated to use 'funcs_by_name.txt' as well as the stubs files.
- All eventproc functions found and written to the output files.

Phil.

Finsig.patch (24.23 kB - downloaded 22 times.)

« Last Edit: 25 / August / 2013, 06:15:33 by philmoz »

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

philmoz

3450

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #87 on: 30 / August / 2013, 05:22:30 »

Revision 3049 (trunk) includes these updates to finsig_dryos.c:
- Unused functions not written to stubs_entry.S
- Two new files created:
- funcs_by_name.csv = all detected function names and addresses sorted by name
- funcs_by_address.csv = all detected function names and addresses sorted by name
- A lot more functions detected (searches for all calls to ExportToEventProcedure and RegisterEventProcedure)

Functions found with the new detection code are written with '_FW' at the end of the name. This is to avoid clashes with the older detection which may find a different function; but use the same name- e.g. EngDrvIn and EngDrvIn_FW.

In some cases the address passed to ExportToEventProcedure or RegisterEventProcedure will just be a branch instruction to another address. In this case two entries are written, the first one to the branch will have 'j_' added at the start of the name.

Revision 3049 only includes the code changes, the new funcs_by_XXX.csv files will get added in the next commit.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

srsa_4c

4451

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #88 on: 30 / August / 2013, 17:25:18 »

Quote from: philmoz on 30 / August / 2013, 05:22:30

Revision 3049 (trunk) includes these updates to finsig_dryos.c:
- Unused functions not written to stubs_entry.S
- Two new files created:
- funcs_by_name.csv = all detected function names and addresses sorted by name
- funcs_by_address.csv = all detected function names and addresses sorted by name
- A lot more functions detected (searches for all calls to ExportToEventProcedure and RegisterEventProcedure)

Functions found with the new detection code are written with '_FW' at the end of the name. This is to avoid clashes with the older detection which may find a different function; but use the same name- e.g. EngDrvIn and EngDrvIn_FW.

In some cases the address passed to ExportToEventProcedure or RegisterEventProcedure will just be a branch instruction to another address. In this case two entries are written, the first one to the branch will have 'j_' added at the start of the name.

Thanks, and sorry about not noticing that patch.

There are several functions whose names are not known (for example many members of the EngDrv family), and it isn't always easy to work out what they do exactly. Can you suggest a naming scheme for these?

Logged

reyalp

14110

Re: New stubs/signature finder (gensig2/finsig2) for DryOS cameras.

« Reply #89 on: 30 / August / 2013, 18:51:34 »

Thanks phil, this is excellent.

Just a note that these could easily be used in lua if you don't want the overhead of registering an entire eventproc group, or want to call functions that aren't evenprocs.

Code: [Select]

fh=io.open('A/funcs_by_name.csv','rb')
funcs={} 
for l in fh:lines() do 
 local ptr,name = string.match(l,'^([^,]+),([^,%s]+)') 
 funcs[name]=tonumber(ptr) 
end
fh:close()

call_func_ptr(funcs.Printf_FW,"hello world")

Or if you want to get fancy

Code: [Select]

fh=io.open('A/funcs_by_name.csv','rb')
funcs={} 
for l in fh:lines() do 
  local ptr,name = string.match(l,'^([^,]+),([^,%s]+)') 
  funcs[name]=function(...)
     call_func_ptr(tonumber(ptr),...)
  end
end
fh:close()

funcs.Printf_FW("hello world")

Logged

Don't forget what the H stands for.