Hello from a Magic Lantern dev - General Chat - CHDK Forum supplierdeeply

Hello from a Magic Lantern dev

  • 4 Replies
Hello from a Magic Lantern dev
« on: 21 / November / 2022, 08:55:22 »
Hi, just wanted to give a quick intro.  I'm a Magic Lantern dev (probably the most active current dev?), and I've finally got around to making an account here.  I'll try not to be annoying, please let me know if I'm breaking community rules etc.  I have I suppose a medium level of experience, I started on ML about 3 years ago.  I have a strong background in C, x86 asm, and general reverse engineering, but ARM was new to me.

I'll probably mostly lurk and sometimes borrow code for ML.  I wanted to be able to post relevant ML breakthroughs here, when I guess there might be stuff CHDK could use in what we find or make.

=== ML Digic 6, 7, 8 (X) support ===
My repo: https://github.com/reticulatedpines/magiclantern_simplified/commits/dev
Most of my work is trying to get ML working on more recent cams - without breaking old cams.  Quite a lot has changed and some old ML code assumes behaviour that is universal on Digic 4 and 5 but doesn't work on 6 and up.  Still, we are making progress and have working ML GUI on at least 6 different models (we understand the boot process well enough on D678X cams, and these all use XIMR rendering too).  I made a stable, but very feature restricted release for 200D recently.

I have integrated srsa's MMU proof of concept code and made the necessary changes to support the existing ML APIs for patching (not unpatching yet, that's more work).  Thanks srsa!  It's as thread / core safe as I know how to make it (cpu0 commands cpu1 to wfi sleep, does the MMU table and mem patches, then uses a custom interrupt handler to wake cpu1 which updates its tables).

=== Qemu ===
I pulled qemu building out of the ML repo, and made a separate repo.  This makes contributing changes much simpler, and should make it easier for CHDK to use, I guess?  Qemu-eos should be independent of CHDK or ML, it should be for the hardware layer.  Feel free to raise PRs, issues etc if anyone is interested in adding powershot etc support for additional cams.

I updated from qemu 2 to 4.  Qemu 2 is unsupported by upstream.  It's difficult to build purely because of its age.  If you hit problems, fixes may exist but you'd have to backport.  Qemu 4 is a lot more recent and uses their modern QOM.  It's more efficient for per-instruction monitoring / tampering and fixes some general ARM bugs.  Sadly it became unsupported while I slowly got it closer to feature parity with qemu-eos 2 (lots of internal Qemu changes that needed fixes).  I'm hoping another update to 7 will be not as painful (7 is closer to 4 than 4 is to 2: they changed version numbering scheme).

General tooling lives here (if CHDK people are interested in making qemu-eos more "cross platform" I should do some re-naming):

I'm currently working on making a better regression testing system via qemu, so I can make more aggressive changes to ML and qemu code, with reasonable confidence I'm not breaking support for old cams.

=== general stuff ===
In case CHDK project isn't aware, a1ex has been absent from ML world for over a year now.  Last I heard he's doing fine but is taking a possibly permanent break from ML work.  This is bad, we've lost a lot of experience and knowledge, and the official repo is de facto unmaintained now.  I want to get my repo good enough re support for old cams and new that it can become a replacement.  There's a long way to go, and lots for me to learn, but it's improving.

If CHDK world has ideas or suggestions for how I could help you, perhaps changes to ML or qemu-eos repo structure, I'd be interested.  No problems if not, I am quite limited in available time anyway :)  But the offer is there!

Cam list: M, 200D, 750D, 850D, G15


Offline Caefix

  • *****
  • 946
  • Sorry, busy deleting test shots...
Re: Hello from a Magic Lantern dev
« Reply #1 on: 21 / November / 2022, 12:29:11 »
Asking without any ML experience at all... :(
Was it possible to merge their code to the CHDK compiler GUI?  ???
All lifetime is a loan from eternity.

Re: Hello from a Magic Lantern dev
« Reply #2 on: 21 / November / 2022, 12:42:22 »
That's not something I know anything about.  It looks like a standalone tool to build CHDK?  If you're asking whether that tool can currently build ML, I would expect the answer is no.


Offline reyalp

  • ******
  • 14098
Re: Hello from a Magic Lantern dev
« Reply #3 on: 21 / November / 2022, 19:30:17 »
Welcome. We're pretty low activity here too this days.

I do plan to spend some more time with the qemu stuff, and will probably have some feedback how to support P&S cams after that. If we can get to the point where adding a useful level of functionality for CHDK cams is straightforward (perhaps by detecting the required values in our sig finders), it could reduce the pain of supporting nearly 400 unique model / firmware combos  :-[
Don't forget what the H stands for.

Re: Hello from a Magic Lantern dev
« Reply #4 on: 21 / November / 2022, 20:49:26 »
Heh, yeah, you do support an impressive number of cams.  I've looked a little at the auto-stub finding and really like the approach, it's on my list of things to copy / steal / port :)  I'd also like to auto-generate ghidra projects given a rom, stubs, logs from qemu etc.

I am currently painfully learning how to add better support for D7 to qemu.  MPU messages during early boot are important, and currently hard for me to work with.  I think I'll need to do some more logging with code hooks in init code on a phys cam.  It feels hard to avoid this without being able to emulate MPU code.  Then you get a bunch of situations where the camera fails some assert and hangs, because it didn't get an expected response from hardware.  You could possibly fuzz your way through this, or maybe something like angr, but both of those are quite a lot of effort for unpredictable gain.  At least in DSLR land, if you have a cam working well from one digic gen, it's easier adding the next one.

Does CHDK not use qemu at all currently?  I'd guess that if you had one cam well supported from each major generation (not sure how that would be defined...), you'd gain quite a lot of confidence about not breaking stuff.  Automating ensuring a cam still makes it to Canon GUI, CHDK / ML menus, and can log custom messages, is fairly simple, and takes maybe 120 seconds from build start to test end.  And you can run in parallel.  Wouldn't find subtle bugs but would ensure you get good feedback from users.


Related Topics

SimplePortal 2.3.6 © 2008-2014, SimplePortal