New tool for porting. - General Discussion and Assistance - CHDK Forum
supplierdeeply
« previous next »
Pages: [1] 2 3 4 5   Go Down

New tool for porting.

  • 40 Replies
  • 14456 Views
*

Online philmoz

  • *****
  • 3449
    • Photos
New tool for porting.
« on: 04 / June / 2013, 02:15:20 »
Advertisements
Based on the excellent work done by waterwingz with CHDK-PT, I've created a new tool that may be of some use in porting new cameras and firmware versions.

I've used it on all the ports I have done - and it found quite a few mistakes (thankfully nothing too serious). These were mostly copy/paste errors when I based one port on a previous one.

The idea is to use a set of instructions to generate the boot.c, capt_seq.c and movie_rec.c files by disassembling the firmware code to the output file, and patching it along the way with the relevant CHDK patches.

This is not magic - it still requires someone to reverse engineer the firmware and find things like function lengths; but it will automate the file generation - hopefully eliminating simple mistakes.

Where I have it very useful if across firmware versions - where little if any change is needed to the instructions file in order to generate fully working source files for CHDK.

Attached is the source code for 'code_gen.c' along with a patch file against the current trunk that integrates it into the build (make PLATFORM=x PLATFORMSUB=y run-code-gen).

The patch includes the instructions files for the G12 ports for reference.

Documentation on the format of the 'code_gen.txt' input file are in the source code.

There is a good chance there are lots of bugs in the program - it works on my ports; but your mileage may vary. Also, it has only been tested on Windows so far.

Phil.

Note - creating the 'code_gen.txt' file that drive the process can be complex. It probably needs a good GUI to help; but that's a project for another day.

Edit: Files replaced - I missed one feature in the documentation.

« Last Edit: 04 / June / 2013, 02:24:59 by philmoz »
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Online philmoz

  • *****
  • 3449
    • Photos
Re: New tool for porting.
« Reply #1 on: 04 / June / 2013, 06:02:23 »
Updated source file that should work with 64 bit Linux (thx nafraf) - also works on Windows and OS/X.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: New tool for porting.
« Reply #2 on: 04 / June / 2013, 19:05:21 »
This is very nice.  Sort of like CHDK-PT's addresses.txt file function on mega-sterioids.

One thing that I don't think I see though is CHDK-PT's  ability to use an addresses.txt file for an existing port to search for the equivalent code is a different firmware version.  Typically not a big deal for boot.c but sometimes video_rec.c and capt_seq.c get offset somewhat it memory so you would need to hand edit your instructions_file to create a new port.

Thinking about possible GUI functionality to roll this into CHDK-PT :
  • Enable CHDK-PT read the instructions_file and ROMfile and produce a dissassembly in the text window. 
  • Add color coding to dissassembly lines that have been edited or added by the instructions_file.
  • Enable CHDK-PT to read the instructions_file and create one for a new firmware version.
  • Create an edit mode where the tag lines in the instructions_file can be created in the dissassemly window.  Right click on a line in the raw dissassembly and you get a pop-menu that allows you to tag the line or add a new tagged line.  Color code the edit as you go.  Dump the final result to a file (overwrite or append options).

I'm sure there are other things - these are just my first thoughts.
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Online philmoz

  • *****
  • 3449
    • Photos
Re: New tool for porting.
« Reply #3 on: 04 / June / 2013, 19:32:25 »
Quote from: waterwingz on 04 / June / 2013, 19:05:21
One thing that I don't think I see though is CHDK-PT's  ability to use an addresses.txt file for an existing port to search for the equivalent code is a different firmware version.  Typically not a big deal for boot.c but sometimes video_rec.c and capt_seq.c get offset somewhat it memory so you would need to hand edit your instructions_file to create a new port.

That's why I added the ability to lookup function addresses using the names found in the stubs_entry.S and stubs_entry_2.S files.
The line
      FUNC sig=task_MovieRecord length=85 name=movie_record_task
will work for any firmware version unless the function length changes.

The ability to search for the end of the function and get the length would be useful though.

Quote
Thinking about possible GUI functionality to roll this into CHDK-PT :
  • Enable CHDK-PT read the instructions_file and ROMfile and produce a dissassembly in the text window. 
  • Add color coding to dissassembly lines that have been edited or added by the instructions_file.
  • Enable CHDK-PT to read the instructions_file and create one for a new firmware version.
  • Create an edit mode where the tag lines in the instructions_file can be created in the dissassemly window.  Right click on a line in the raw dissassembly and you get a pop-menu that allows you to tag the line or add a new tagged line.  Color code the edit as you go.  Dump the final result to a file (overwrite or append options).

I'm sure there are other things - these are just my first thoughts.

That's what I was thinking too - sort of a CHDK mini version of IDA with the ability to generate the source code files :)

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: New tool for porting.
« Reply #4 on: 04 / June / 2013, 20:05:16 »
Quote from: philmoz on 04 / June / 2013, 19:32:25
That's why I added the ability to lookup function addresses using the names found in the stubs_entry.S and stubs_entry_2.S files.
That works for the "top level" function but what about the subroutine chains? How does each routine in the subfunction chain get its start address if its not contiguous with the calling function?
Never mind - see it now.  It gets it start address from the call in the calling function.  Very elegant.
« Last Edit: 04 / June / 2013, 20:07:43 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline nafraf

  • *****
  • 1308
Re: New tool for porting.
« Reply #5 on: 04 / June / 2013, 20:11:15 »
In attachment an example of PATCHSUB use, I'm getting unexpected results, maybe a bug, maybe an error in input file:

file0: patch to task_FileWrite, last instruction patched.
         second function is disassembled without problems.

file2: patch to task_FileWrite, use of multiple patch references
         second function is disassembled wrong, function start some instructions later.

file4: patch to task_FileWrite, use of multiple patch without references
         second function is disassembled wrong, function start some instructions later.

I'm using remote capture v3 branch, sx240/100c files as input.
Thanks
« Last Edit: 04 / June / 2013, 20:28:52 by nafraf »
Logged
*

Online philmoz

  • *****
  • 3449
    • Photos
Re: New tool for porting.
« Reply #6 on: 04 / June / 2013, 20:51:31 »
Quote from: nafraf on 04 / June / 2013, 20:11:15
In attachment an example of PATCHSUB use, I'm getting unexpected results, maybe a bug, maybe an error in input file:

file0: patch to task_FileWrite, last instruction patched.
         second function is disassembled without problems.

file2: patch to task_FileWrite, use of multiple patch references
         second function is disassembled wrong, function start some instructions later.

file4: patch to task_FileWrite, use of multiple patch without references
         second function is disassembled wrong, function start some instructions later.

I'm using remote capture v3 branch, sx240/100c files as input.
Thanks

Thanks for trying it out.

In file0.txt I think you need 'FW $' at the end of the 'filewritetask' instructions to finish the disassembly.

File2.txt & file4.txt is a bug - attached source should fix it.
Also the length the second function looks too long - try 91 instead of 104.

Phil.

Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline nafraf

  • *****
  • 1308
Re: New tool for porting.
« Reply #7 on: 05 / June / 2013, 01:44:14 »
Quote from: philmoz on 04 / June / 2013, 20:51:31
File2.txt & file4.txt is a bug - attached source should fix it.
Thanks, it worked.

In attachment, the commands to port remote capture functions for SX260, the same file seems to work with R50 Axxxx cameras. I have not tested it in any camera yet, it was only a code_gen command exercise.

remote capture function uses an uncommon patch, a branch instruction is added. 
a810.100b capt_seq.c:
Code: [Select]
//mod start
      "LDR R3, =current_write_ignored\n"
      "LDR R3, [R3]\n"
      "CMP R3, #0\n"
      "BNE loc_FFA5229C\n" // jump over the next block          <==== BRANCH
//mod end
      "    BL      sub_FF83015C \n".
      "    B       loc_FFA522A0 \n".
"loc_FFA5229C:\n"
      "    BL      fwt_close\n"         // mod! sub_FF830158 Close
 Maybe a new command for a future release? ;)
Logged
*

Online philmoz

  • *****
  • 3449
    • Photos
Re: New tool for porting.
« Reply #8 on: 05 / June / 2013, 02:14:45 »
Quote from: nafraf on 05 / June / 2013, 01:44:14
Quote from: philmoz on 04 / June / 2013, 20:51:31
File2.txt & file4.txt is a bug - attached source should fix it.
Thanks, it worked.

In attachment, the commands to port remote capture functions for SX260, the same file seems to work with R50 Axxxx cameras. I have not tested it in any camera yet, it was only a code_gen command exercise.

remote capture function uses an uncommon patch, a branch instruction is added. 
a810.100b capt_seq.c:
Code: [Select]
//mod start
      "LDR R3, =current_write_ignored\n"
      "LDR R3, [R3]\n"
      "CMP R3, #0\n"
      "BNE loc_FFA5229C\n" // jump over the next block          <==== BRANCH
//mod end
      "    BL      sub_FF83015C \n".
      "    B       loc_FFA522A0 \n".
"loc_FFA5229C:\n"
      "    BL      fwt_close\n"         // mod! sub_FF830158 Close
 Maybe a new command for a future release? ;)

Interesting case - you still need to know the firmware address for the branch, and the label isn't guaranteed to be generated automatically for you (unless some other branch used the same location).

I'll have to think about that one.

With the current version you could probably use this to achieve the result, it moves the skipped instructions and branch label into the inserted code, then skips generation of the firmware instructions. It should work across firmware instructions unless the number of instructions to skip needs to be changed.

Code: [Select]
FUNC ref=2 length=102
ASM
FW 14
>>>
//mod start
"    LDR R3, =current_write_ignored\n"
"    LDR R3, [R3]\n"
"    CMP R3, #0\n"
"    BNE loc_skip\n" // jump over the next block
"    BL      sub_FF83015C \n".
"    B       loc_FFA522A0 \n".
"loc_skip:\n"
//mod end
<<<
SKIP 2
PATCHSUB name=fwt_close
FW $
ENDASM
ENDFUNC

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline nafraf

  • *****
  • 1308
Re: New tool for porting.
« Reply #9 on: 06 / June / 2013, 19:03:00 »
Thanks philmoz, nice solution!!  I used your second suggestion, that post is missing, so I copied the code here:
Code: [Select]
FUNC ref=2 length=102
ASM
FW 14
>>>
//mod start
"    LDR     R3, =current_write_ignored\n"
"    LDR     R3, [R3]\n"
"    CMP     R3, #0\n"
"    BNE     loc_B\n" // jump over the next block
//mod end
<<<
FW 2
>>>
"loc_B:\n"
<<<
PATCHSUB name=fwt_close
FW $
ENDASM
ENDFUNC
Logged
Pages: [1] 2 3 4 5   Go Up
« previous next »
 

Related Topics