CHDK PTP interface

[mweerden]

Hi all,

The last couple of days I've been working on a PTP extension for the CHDK that allows various remote operations (e.g. memory inspection/manipulation, function calls, uploading/downloading, switching to record mode and LUA script execution). I've also made an ad-hoc command-line tool to use it (based on ptpcam).

Have a look here for patches and documentation of what I did: http://www.mweerden.net/chdk_ptp.html

[HarpoMa]

wow - I will be checking this out tonight.  Wonderful work.

[fe50]

Great stuff !
...will have a look to this, too - but first i need to apply the LongDay patch to extend my available CHDK-ing time from Day24h to Day32h   

[hudson]

I've added some similar functions to Magic Lantern for the 5D Mark II.  It looks like we duplicated each others efforts in reverse engineering the PTP context structure and callback interface.  I added fine grained focus control for interfacing with remote focus controllers like the Impero.

One of my other goals was to build a command console that would work over the PTP interface.  Perhaps it would allow arbitrary Lua commands to be sent, and use the USB as the stdout.  And as an outside goal, I wanted to add a gdb interface so that extensions could be more easily debugged.

[mweerden]

I've added some similar functions to Magic Lantern for the 5D Mark II.  It looks like we duplicated each others efforts in reverse engineering the PTP context structure and callback interface.

First time I even hear of Magic Lantern. Fortunately the PTP part wasn't much work, but it would be nice if we can avoid duplication in the future. As I've said before, there is a lot of knowledge floating around these project, but most of it is inaccessible.

By the way, I just tried to find your functions (for comparison) but had no luck; can you give a me pointer?

One of my other goals was to build a command console that would work over the PTP interface.  Perhaps it would allow arbitrary Lua commands to be sent, and use the USB as the stdout.

This shouldn't be hard. I'm not sure if it's clear, but my "LUA script execution" means you can already execute arbitrary commands (e.g. "lua shoot();" with "ptpcam --chdk"). What I haven't looked at is something to preserve context between executions. Sending of output should be fairly simple (e.g. with a simple hook).

And as an outside goal, I wanted to add a gdb interface so that extensions could be more easily debugged.

Do you think this is possible? It seems to me that this requires support from the OS to provide tracing and breakpoints and what not. Has anything like this been found?

[ewavr]

Great!
How about developers under Windows, can they use libptp or something else?

[mweerden]

How about developers under Windows, can they use libptp or something else?

A quick search gave me the following: http://code.assembla.com/CuteCanonCapture/subversion/nodes/libptp2?rev=30 This seems to be the libptp2-1.1.10 that I used with some minor changes to make it work on Windows. I don't think my code uses anything Linux specific. I currently don't have the right configuration around to try it out, however.

[hudson]

First time I even hear of Magic Lantern. Fortunately the PTP part wasn't much work, but it would be nice if we can avoid duplication in the future. As I've said before, there is a lot of knowledge floating around these project, but most of it is inaccessible.

Since I'm working solely on the 5D Mark II and 7D cameras and focusing almost entirely on the cinematography features, there isn't a whole lot of overlap with the CHDK dev team.  I hope there will be more cross-pollination in the future, however.

By the way, I just tried to find your functions (for comparison) but had no luck; can you give a me pointer?

ptp.h.  It isn't near as complete as your implementation; I plan to hack your hacked ptpcam program to work for me, as well work on a AVR PTP library so that the Impero can talk to the 5D.

I'm not sure if it's clear, but my "LUA script execution" means you can already execute arbitrary commands (e.g. "lua shoot();" with "ptpcam --chdk"). What I haven't looked at is something to preserve context between executions. Sending of output should be fairly simple (e.g. with a simple hook).

Neat.  I hadn't realized that.  ML doesn't have Lua yet, although we do have a simple embedded Python.  I've been meaning to port the Lua library patches so that we can put Lua into it.

And as an outside goal, I wanted to add a gdb interface so that extensions could be more easily debugged.

Do you think this is possible? It seems to me that this requires support from the OS to provide tracing and breakpoints and what not. Has anything like this been found?

I see things that look like a debugger interface in the command shell inside DryOS.  As you had found, there are equivalents to ps, top, and a few other programs.  We also have control of the scheduler, so we might be able to do some simple profiling.

[Microfunguy]

Have a look here for patches and documentation

This is a bit too advanced for my basic knowledge.

Could you list the steps that I have to take to compile and patch the various softwares (under Windows) and run the command line for testing ?

Assume that I do not know anything.

If I manage to get that working, I will see if I can support my A620.

It would be VERY tedious to support all cameras.


David

[mweerden]

By the way, I just tried to find your functions (for comparison) but had no luck; can you give a me pointer?

ptp.h.  It isn't near as complete as your implementation; I plan to hack your hacked ptpcam program to work for me, as well work on a AVR PTP library so that the Impero can talk to the 5D.

Thanks, I somehow completely missed that file.

I see things that look like a debugger interface in the command shell inside DryOS.  As you had found, there are equivalents to ps, top, and a few other programs.  We also have control of the scheduler, so we might be able to do some simple profiling.

Hmmm, I'm not sure I'm as positive about the possibilities, but who knows?

[mweerden]

This is a bit too advanced for my basic knowledge.

Sorry about that.

It would be VERY tedious to support all cameras.

I don't know about that. Of course, every feature that relies on previously unused functions from the firmware requires each and every firmware to be inspected, but that is inherent to the CHDK. Perhaps it is possible to find a few generic signatures that can be used instead. Otherwise you can just give empty definitions by default and override them only for some cameras (as is done with other "optional" functionality).

The only thing that is really needed is the address of add_ptp_handler. For specific features (shutdown/reboot and mode switching) you need a few more.

Could you list the steps that I have to take to compile and patch the various softwares (under Windows) and run the command line for testing ?

Sure, but I'm not sure about the Windows part as I can't test it. Also, I haven't looked at VxWorks, so that might also be a bit different.

For the CHDK you need the following:

• NHSTUB for add_ptp_handler (found as discussed in my documentation)
• Platform functions shutdown_soft, reboot and switch_mode (I've put them in lib.c)
• Make sure init_chdk_ptp is started as task (see boot.c of IXUS 870 IS 1.01a)

The platform function need not have anything as body, unless you want the appropriate functionality to work. In the latter case you also need the following:

• Address for power button (I've found it, together with the playback button, in taskcreate_Startup, most clearly in function 0xFF845D18)
• NHSTUB for reboot_fw_update (can be found by searching for "FirmUpgrade.c")
• NHSTUB for set_control_event (discussed in documentation; for VxWorks you seem to need DisableNotificationPTP)
• NHSTUB for PB2Rec and Rec2PB functions (searchable as strings)
• Values to pass to set_control_event (can be found as discussed in documentation; not needed for VxWorks)

To get libptp/ptpcam you should download the latest version (1.1.10) or, in case of Windows, I'd download the sources linked to before in this thread. Then apply my patch (I think it should work for the Windows variant as well) and compile it. I'm not sure which compilers work for Windows and whether or not you need libusb as well.

I think that should be it. Let me know how things work out.

Edit: added creation of init_chdk_ptp as task to todo list.

[Microfunguy]

Many thanks for that.

I will do a few chores and start this later.

I am sure I will have problems, I will contact you directly 


David

[whim]

@mweerden

many thanks for this, great project !

I'm not sure which compilers work for Windows

are you using the same compiler setup for this as for compiling CHDK ?  and which native/ARM GCC versions do you use ?

TIA,

wim
 

[mweerden]

I'm not sure which compilers work for Windows

are you using the same compiler setup for this as for compiling CHDK ?  and which native/ARM GCC versions do you use ?

Yes. For CHDK I'm using 4.3.3 and for libptp/ptpcam 4.3.3-12 (Debian).

[mweerden]

In conversation with Microfunguy I realised that getting things done on Windows isn't always as easy. I've created a zip with sources for Windows (here). This works for me with MinGW and libusb for Windows (filter binary). You'll probably have to adjust some paths in the Makefile to make things work for your system.

One problem I noticed on Windows is that with certain request (as the ones Windows automatically makes) the camera goes into a deeper mode of sleep (if you will). In this mode not everything works as otherwise (e.g. shutdown and mode switching; edit: you shouldn't test another function if the previous has already crashed the camera). Edit: it seems that mode switching first makes the camera stay in the current mode. Perhaps a wrapper around PTP_OC_OpenSession that calls set_control_event(0x902) should be used.

I also noticed that my ptpcam patch had a small bug (the last #endif should have been just before the preceding else). This has been fixed.