Ghidra reverse engineering tool - page 4 - General Discussion and Assistance - CHDK Forum supplierdeeply

Ghidra reverse engineering tool

  • 36 Replies
  • 19572 Views
*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #30 on: 12 / December / 2020, 00:48:38 »
Advertisements
Ghidra 9.2 was released in November: https://ghidra-sre.org/releaseNotes_9.2.html

There are a lot of changes, but nothing jumps out as being especially important for CHDK development.

It is not compatible with some of my scripts ( get_pinsn_at fails, which breaks ImportCHDKStubs and others). I'll look into fixing this, but for the moment upgrading is not recommended.

While both can be installed at the same time, opening programs with the new version will cause them to be upgraded, which may make them not open in the old version.

Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #31 on: 13 / December / 2020, 23:34:05 »
I checked in fixes for the scripts in r5674. However, I noticed that ghidra 9.2 incorrectly disassembles some bl instructions (interpreting them as vst4.* with a warning "Instruction pcode is unimplemented") so I'd recommend sticking with 9.1.2 for now.
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #32 on: 23 / January / 2021, 19:09:24 »
I checked in fixes for the scripts in r5674. However, I noticed that ghidra 9.2 incorrectly disassembles some bl instructions (interpreting them as vst4.* with a warning "Instruction pcode is unimplemented") so I'd recommend sticking with 9.1.2 for now.
9.2.2 was released on Dec 29 and appears to fix this issue. The CHDK scripts work correctly, so I'd say this is the recommended version for new installs now. If you're already on 9.1.x, I haven't seen any really compelling reasons to upgrade, though I think the disassembly may be a bit improved.
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #33 on: 16 / February / 2021, 23:11:44 »
In trunk r5733, I added some files in tools/ghidra_scripts/datatypes that can be used to make Ghidra aware of function prototypes and structure definitions. This can significantly improve decompiler output, both in the identified functions themselves, and in other code that calls them.

The header files are manually created. I thought about trying make the normal CHDK files usable directly in Ghidra, but it seemed pretty impractical so I just started with a copy of lowlevel.h instead. I've added some additional functions that were used elsewhere, as well as some common ones from the stubs that aren't currently used in CHDK. IMO, even outside the direct benefits to analysis, this is a useful place to document functions we have named and understood.

Basic usage from the README.TXT below (I'll add it to the wiki later done ). Ghidra supports several different workflows, but this was what seemed to work best for me after playing around with it for a while.

The program should already be analyzed.

Go to File -> Parse C Source

* Use the small disk icon with ... under it to copy an existing parse configuration, e.g clib.prf
* Name your copy something obviously related to CHDK and camera configuration, e.g. chdk-dryos31
* Select all the header file entries, and use the red X button to delete them
* Use the green + button to add chdk source/tools/ghidra_scripts/fw_functions.h
* Adjust the parse options section to match your platform:
  Remove all entries except
   -D__builtin_va_list=void *
  If your camera uses dryos, add the PLATFORMOSVER value from makefile.inc, like
   -DCAM_DRYOS_REL=31
  If your camera uses 3 argument DebugAssert (see platform_camera.h) add
   -DCAM_3ARG_DebugAssert=1
  This applies to some early vxworks, all digic 6, and some other DryOS 52 and later.
 * Save your parse configuration with the big floppy icon. Note: Parse configurations are global
   within Ghidra, not specific to a particular project or program.
 * Click "Parse to Program", and continue when prompted
 * If a prompt about "Use Open Archives" appears, click continue. It may be covered by a dialog titled "Parsing C Files". If so, move the "Parsing C files" dialog out of the way.
 * If parsing is unsuccessful, the pre-processed output will appear in your system home directory in a file named CParserPlugin.out
 * If parsing succeeds, dismiss the Parse C Source dialog.

In the types manager window, right click on your program, and choose "Apply Function Data Types"

If you update the header files, re-run File -> Parse C Source, select the parse configuration
created earlier, and re-run "Apply Function Data Types".
« Last Edit: 18 / February / 2021, 11:14:31 by reyalp »
Don't forget what the H stands for.


*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #34 on: 06 / April / 2021, 02:47:32 »
In trunk 5812, I added a script CommentLeventCalls.py to comment calls to "logical event" functions with the names from levent_table

edit:
In 5814, I added ListLeventCalls.py, which lists calls referencing events, specified by name or ID.

edit:
A few more scripts:
CleanEmptyFuncs.py - Ghidra auto-analysis seems to sometimes create zero-length functions. This script removes them, and re-creates them if they are in valid code. This solves many cases where the comment / list calls scripts say "not in a function" when it looks like a function is present.

LabelsToFuncs.py - create functions from labels if they look like valid function starts. It might get a few cases wrong (particularly where the original Canon code was doing weird things in ASM) but seems seems seems to give good results in the vast majority of cases.

CommentMzrmCreateCalls.py - for thumb2 firmware, add comments with the name of mzrm messages, using list generated from tools  in https://chdk.setepontos.com/index.php?topic=11316.msg129104#msg129104

ListMzrmCreateCalls.py - as above, but list address where specific messages are created

NameMzrmFunctions.py - As above, but name the calling function with the name of the message, if it only creates one.

There's getting to be enough of these that I'm starting to want one master script to run them all in order. FWIW, my recommended sequence for a new dump is
InitCHDKMemMap.py
ImportCHDKStubs.py (Entry points only)
... auto analyze ...
... load and apply function data types ...
CleanThumb2BookmarkErrors.py (thumb2 only)
CleanFuncBookmarks.py
CleanEmptyFuncs.py
ImportCHDKStubs.py (load and disassemble)
LabelsToFuncs.py
CommentLeventCalls.py
CommentPropCalls.py
CommentMzrmCreateCalls.py (thumb2 only)
NameMzrmFunctions.py (thumb2 only)

Except for InitCHDKMemMap.py, they can safely be run repeatedly, for example if new stubs have been added, or more of the firmware has been analyzed.
« Last Edit: 18 / April / 2021, 15:15:00 by reyalp »
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #35 on: 29 / May / 2021, 01:45:01 »
Ghidra 9.2.4 was release at the end of April. It appears to work OK with the current scripts. I don't see any obvious CHDK relevant changes.

I added a script called ListCallsWithArgs.py a while back (r5878). It searches for calls to a function with specified r0 through r4 arguments. The function can be a name or address. The arguments are given as a list of numbers or - to ignore. So to find all the asserts referring to line 123 (on a 3 arg assert cam), you could do
DebugAssert
- - 123

If multiple values are given, they must all match.

Note that it does not currently handle veneers/thunks, so you'd have to search them individually.

Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14080
Re: Ghidra reverse engineering tool
« Reply #36 on: 26 / June / 2021, 02:23:43 »
Ghidra 10.0 was released recently. Testing CHDK scripts on fresh loads of digic 2, 4, 6 and 7 dumps worked and gave similar results to 9.2.4

Most of the major changes don't seem relevant to us, but it does seem to analyze a bit faster, and fixes at least one ARM issue related to thumb instructions.
Don't forget what the H stands for.

 

Related Topics