I've added some similar functions to Magic Lantern (http://magiclantern.wikia.com/) for the 5D Mark II. It looks like we duplicated each others efforts in reverse engineering the PTP context structure and callback interface.First time I even hear of Magic Lantern. ;) Fortunately the PTP part wasn't much work, but it would be nice if we can avoid duplication in the future. As I've said before, there is a lot of knowledge floating around these project, but most of it is inaccessible.
One of my other goals was to build a command console that would work over the PTP interface. Perhaps it would allow arbitrary Lua commands to be sent, and use the USB as the stdout.This shouldn't be hard. I'm not sure if it's clear, but my "LUA script execution" means you can already execute arbitrary commands (e.g. "lua shoot();" with "ptpcam --chdk"). What I haven't looked at is something to preserve context between executions. Sending of output should be fairly simple (e.g. with a simple hook).
And as an outside goal, I wanted to add a gdb interface so that extensions could be more easily debugged.Do you think this is possible? It seems to me that this requires support from the OS to provide tracing and breakpoints and what not. Has anything like this been found?
How about developers under Windows, can they use libptp or something else?A quick search gave me the following: http://code.assembla.com/CuteCanonCapture/subversion/nodes/libptp2?rev=30 (http://code.assembla.com/CuteCanonCapture/subversion/nodes/libptp2?rev=30) This seems to be the libptp2-1.1.10 that I used with some minor changes to make it work on Windows. I don't think my code uses anything Linux specific. I currently don't have the right configuration around to try it out, however.
First time I even hear of Magic Lantern. ;) Fortunately the PTP part wasn't much work, but it would be nice if we can avoid duplication in the future. As I've said before, there is a lot of knowledge floating around these project, but most of it is inaccessible.Since I'm working solely on the 5D Mark II and 7D cameras and focusing almost entirely on the cinematography features, there isn't a whole lot of overlap with the CHDK dev team. I hope there will be more cross-pollination in the future, however.
By the way, I just tried to find your functions (for comparison) but had no luck; can you give a me pointer?ptp.h (http://bitbucket.org/hudson/magic-lantern/src/tip/ptp.h). It isn't near as complete as your implementation; I plan to hack your hacked ptpcam program to work for me, as well work on a AVR PTP library so that the Impero can talk to the 5D.
I'm not sure if it's clear, but my "LUA script execution" means you can already execute arbitrary commands (e.g. "lua shoot();" with "ptpcam --chdk"). What I haven't looked at is something to preserve context between executions. Sending of output should be fairly simple (e.g. with a simple hook).Neat. I hadn't realized that. ML doesn't have Lua yet, although we do have a simple embedded Python. I've been meaning to port the Lua library patches so that we can put Lua into it.
I see things that look like a debugger interface in the command shell inside DryOS. As you had found, there are equivalents to ps, top, and a few other programs. We also have control of the scheduler, so we might be able to do some simple profiling.QuoteAnd as an outside goal, I wanted to add a gdb interface so that extensions could be more easily debugged.Do you think this is possible? It seems to me that this requires support from the OS to provide tracing and breakpoints and what not. Has anything like this been found?
Have a look here for patches and documentation
Thanks, I somehow completely missed that file.QuoteBy the way, I just tried to find your functions (for comparison) but had no luck; can you give a me pointer?ptp.h (http://bitbucket.org/hudson/magic-lantern/src/tip/ptp.h). It isn't near as complete as your implementation; I plan to hack your hacked ptpcam program to work for me, as well work on a AVR PTP library so that the Impero can talk to the 5D.
I see things that look like a debugger interface in the command shell inside DryOS. As you had found, there are equivalents to ps, top, and a few other programs. We also have control of the scheduler, so we might be able to do some simple profiling.Hmmm, I'm not sure I'm as positive about the possibilities, but who knows? ;)
This is a bit too advanced for my basic knowledge.Sorry about that.
It would be VERY tedious to support all cameras.I don't know about that. Of course, every feature that relies on previously unused functions from the firmware requires each and every firmware to be inspected, but that is inherent to the CHDK. Perhaps it is possible to find a few generic signatures that can be used instead. Otherwise you can just give empty definitions by default and override them only for some cameras (as is done with other "optional" functionality).
Could you list the steps that I have to take to compile and patch the various softwares (under Windows) and run the command line for testing ?Sure, but I'm not sure about the Windows part as I can't test it. Also, I haven't looked at VxWorks, so that might also be a bit different.
I'm not sure which compilers work for Windowsare you using the same compiler setup for this as for compiling CHDK ? and which native/ARM GCC versions do you use ?
Yes. For CHDK I'm using 4.3.3 and for libptp/ptpcam 4.3.3-12 (Debian).QuoteI'm not sure which compilers work for Windowsare you using the same compiler setup for this as for compiling CHDK ? and which native/ARM GCC versions do you use ?
I've created a zip with sources for Windows (here (http://www.mweerden.net/download/chdk-ptp/libptp2-chdk-win32.zip)). This works for me with MinGW and libusb for Windows (http://libusb-win32.sourceforge.net/) (filter binary).
I also noticed that my ptpcam patch had a small bug (the last #endif should have been just before the preceding else). This has been fixed.
now ptpcam.exe works
Is that with A710IS ?Yes.
ptpcam -L just waits forever.How about disable "Canon Camera Access Library" service (and in extreme case, WIA service) in services applet?
How about disable "Canon Camera Access Library" service (and in extreme case, WIA service) in services applet?
Is there any way the ptpcam code can do that ?
net stop CCALib8
ptpcam --chdk
net start CCALib8
I think so (see here (http://msdn.microsoft.com/en-us/library/ms686335(VS.85).aspx)), but I'm not sure it's the right place. Why not just use a command like 'sc' to do it?How about disable "Canon Camera Access Library" service (and in extreme case, WIA service) in services applet?Is there any way the ptpcam code can do that ?
p.s. For a710 NHSTUB(add_ptp_handler, 0xFFE59FE4) - and transfer works!Great!
But uploading/downloading of files fails - ptpcam reports about unexpected result code, then camera shuts down.Hmm.. If the code is something like 0x2ff it usually means the camera has crashed; 0x2002 means a less critical error (e.g. alloc failed). Perhaps the functions passed to the handler work a bit different for VxWorks. I deduced their workings by looking at how they were used by handlers for things like PTP_OC_GetObject.
Hmm.. If the code is something like 0x2ff it usually means the camera has crashed; 0x2002 means a less critical error (e.g. alloc failed). Perhaps the functions passed to the handler work a bit different for VxWorks. I deduced their workings by looking at how they were used by handlers for things like PTP_OC_GetObject.
It was my stupid error - remote filename must begin with 'A/' (not with 'a/') ;)Yeah, I fell for that one too... However, it shouldn't really cause a crash. It is likely a problem caused by the fact that one side still expects to send/receive data. Interestingly enough I found that in these cases another function (that comes after get_data_size in ptp_data) than send_resp is used in the A620 firmware (and not in the IXUS 870 IS firmware, as far as I can tell). This might be a solution, but I haven't tried anything out yet.
Yeah, I fell for that one too... However, it shouldn't really cause a crash. It is likely a problem caused by the fact that one side still expects to send/receive data.
No, in my a710Ah.. I guess that is the same with my camera then. I think I assumed adding NULL checks would fix it and didn't actually check. I didn't consider the possibility that fopen actually crashes stuff. Perhaps it would be a good idea to add some check in its definition so it behaves more naturally.
FILE* f=fopen("a/1.txt","wb");
really crashes the camera (and "A/1.txt" not).
Yeah, I fell for that one too... However, it shouldn't really cause a crash. It is likely a problem caused by the fact that one side still expects to send/receive data.
No, in my a710
FILE* f=fopen("a/1.txt","wb");
really crashes the camera (and "A/1.txt" not).
int open (const char *name, int flags, int mode )
{
if(name[0]!='A')return -1;//on the SD980, it will crash if a file path doesn't start with A
return _Open(name, flags, mode);//hmm
}
It was my stupid error - remote filename must begin with 'A/' (not with 'a/') ;)
Does ptpcam work in Record mode on your A710 ?I'll try this evening.
I'll try this evening.
For cameras that have a playback pushbutton, I have used the following for PostLogicalEventForNotPowerType :-I like the use of PostLogicalEventForNotPowerType (also for soft shutdown with 0x1005); it's seems a lot safer than directly calling functions that hopefully do the same. Unfortunately switching to playback this way causes the PTP connection to be closed (just like really pressing the button), while calling Rec2PB doesn't. On the other hand, it is just a minor inconvenience.
RECORD 0x1001
PLAYBACK 0x1003
So, on the ixus870, does that switch to record mode and does ptp continue to work ?Yes. That is, together with the call to set_control_event.
together with the call to set_control_event.
I don't know. I tried several things (including PostLogicalEventForNotPowerType(0x10A6); for you it seems 0x1086 is needed), but they typically resulted in actually disconnecting USB connections.together with the call to set_control_event.
As I do not seem to have that on the A620, I wonder if I should replace with connect and disconnect USB functions ?
void switch_mode(int mode) {
if ( mode == 0 ) {
_PostLogicalEventForNotPowerType(0x1003, 0); //PressPBButton
}
else if ( mode == 1 ) {
_PostLogicalEventForNotPowerType(0x1001, 0); //PressRecButton
}
}
works very well. No crashes, no USB disconnect when camera toggles record<->playback.Ok, it seemt that for A710 this simple code:
works very well. No crashes, no USB disconnect when camera toggles record<->playback.
I guess _ExitTask needs declaring in lolevel.h.According to lolevel.h it should only be used in platform code. To fix it it would be best to add a wrapper ExitTask to platform/generic/wrappers.h (and declaration to include/platform.h) and use that function.
Edit: it is declared but still get implicit declaration warning.
I wonder if you can (in principle) upload files that the Canon firmware will not report over PTP ?Sure you can, the normal PTP and CHDK PTP functionalities are completely independent.
Is there any way ptpcam can talk to two cameras ?I think it should be possible with two instances of ptpcam. You can use the options --bus and --dev when starting an instance to indicate which device you want to connect to. You should be able to see the right values with 'ptpcam -l'.
transfer of live image over USB 2.0 is quite fast
void CreateTask_init_chdk_ptp() {
_CreateTask("InitCHDKPTP", 0x19, 0x2000, init_chdk_ptp, 0);
};
static void task_start_hook(
long p0, long p1, long p2, long p3, long p4,
long p5, long p6, long p7, long p8, long p9)
{
_CreateTask("SpyTask", 0x19, 0x2000, spytask, 0);
task_prev(p0, p1, p2, p3, p4, p5, p6, p7, p8, p9 );
CreateTask_init_chdk_ptp();
}
hald in Ubuntu causes a little bit of trouble (first invocation if ptpcam always fails to connect but it works on the second attempt...I didn't try without hald yet):the REC switch actually shuts down the camera if USB cable is plugged in (even without CHDK).The same for a710.
lua set_levent_script_mode(1); post_levent("PressRecButton"); post_levent("UnpressRecButton"); set_levent_script_mode(0)post_levent_to_ui("PressRecButton")
but that didn't have any effect.
edit: If someone interested, I write simple GUI program for Windows for remote shooting (see screenshot), using slightly modified mweerden's code....very interested 8)
Could you publish the code ?Yes, I'll do a few days (currently only a710 is supported, but I'll try tomorrow port PTP extension for SX10).
edit: If someone interested, I write simple GUI program for Windows for remote shooting (see screenshot), using slightly modified mweerden's code.Hi ewavr, good work, I too am interested, but I have A720 - possible for A720? With code maybe I could compile
Great :xmasCould you publish the code ?Yes, I'll do a few days (currently only a710 is supported, but I'll try tomorrow port PTP extension for SX10).
How about ConnectUSBCable, DisconnectUSBCable ? Or possibly the Cradle ones (or is cradle something different than USB ?)
Or perhaps, using the eventproc code UiEvnt_StartDisguiseCradleStatus UiEvnt_StopDisguiseCradleStatus ?
Take it easy, i'll need also some more days to test & play with all the new, nice stuff, PTP & reyalP's events [
src&bin for Win, a710 and sx10 1.01a (pre-alpha) here - http://ewavr.nm.ru/chdk/for_test/ (http://ewavr.nm.ru/chdk/for_test/)Wow, that was fast !
Started with testing, playback connection is ok, ~ 4 to 5 fps.This depends on computer speed and camera model (I have 12 fps for SX10 and 7 fps for A710 with overlays) - GDI drawing is not too fast.
BTW: atm i use an unmodified libusb-win32-filter-bin-0.1.12.2 package; are the patches from mweerden still needed ?No, libusb and ptpcam are different things. To recompile libusb-win32, DDK is required.
set_levent_script_mode(1)
post_levent_to_ui("PressRecButton")
post_levent_to_ui("UnpressRecButton")
sleep(2000)
post_levent_to_ui("ModeDialToP")
set_levent_script_mode(0)
in the script box, which switches to "manual" mode, and then take a shot with the "Shoot" button!Has anyone succeeded in switching to movie without disconnecting PTP?
Or maybe exec_event_proc("UIFS_SetDialMovieRec")Has anyone succeeded in switching to movie without disconnecting PTP?
May be post_levent_to_ui("ModeLeverMovieRec") ?
May be post_levent_to_ui("ModeLeverMovieRec") ?
When camera was booted to REC mode, I can't get any mode change (including movie) to stick (camera announces the change on LCD, but then it changes back to whatever it was). In the mode dial thread http://chdk.setepontos.com/index.php/topic,3228.45.html (http://chdk.setepontos.com/index.php/topic,3228.45.html) reyalp's solution for this particular problem is to set set_levent_script_mode(1) first but it doesn't seem to help when PTP is active.set_levent_script_mode() is only half the solution. The other half is set_levent_active("ModeDailTo...",false) on all the mode dial position events *except* the one you want to use, and set_levent_active("ModeDailToMovie",true)
post_levent_to_ui("ModeLeverMovieRec") is ignored.Or maybe exec_event_proc("UIFS_SetDialMovieRec")Has anyone succeeded in switching to movie without disconnecting PTP?May be post_levent_to_ui("ModeLeverMovieRec") ?
(you probably have to exec_event_proc("UI_RegistDebugEventProc") first)
Almost every camera is somehow different in this respect, it seems. Funny. Why would they bother to change the logic every time?
set_levent_script_mode() is only half the solution. The other half is set_levent_active("ModeDailTo...",false) on all the mode dial position events *except* the one you want to use, and set_levent_active("ModeDailToMovie",true)
set_levent_script_mode(true)
set_mode_exclusive(modename)
post_levent_to_ui(modename)
sleep(ModeSwDelay)
set_levent_script_mode(false)src&bin for Win, a710 and sx10 1.01a (pre-alpha) here - http://ewavr.nm.ru/chdk/for_test/ (http://ewavr.nm.ru/chdk/for_test/)Support for sx200is 100c, wrong aspect for overlay...
Support for sx200is 100c, wrong aspect for overlay...
and try to make corrections in my code tomorrow.Done. But I can't test how it works on SX200 :).
>we have a 720x240 efective memory buffer (in a 960x270 physical memory buffer)
Does that mean we have to write a pixel to the bitmap buffer twice in the 'x' direction to memory locations (y*960)+(x*2) and (y*960)+(x*2)+1 ?Maybe this question is for 'SX200 porting' thread. Maybe yes, but drawing functions for sx200 in dui_draw.c are not too easy to understang :).
Is the size of the live viewport buffer 360x240x3 bytes ?More precisely, 720*240*6/4 (in playback mode).
drawing functions for sx200 in dui_draw.c are not too easy to understang
Tried to patch source, but ptp.c is missing in your patch.
but there is no improvement...It seems this was error in PC client program. Corrected now.
Could you upload ptp.c from my previous patch?Your first (I think) ptp_patch.zip attached.
Your first (I think) ptp_patch.zip attached.Many thanks! I forgot about this (http://my-trac.assembla.com/chdkde/browser/branches/fe50/core/ptp.c).
It would be nice if you could also create a version for the A720. I would like to test the PTP functions.
post_levent_to_ui("PressRecButton")If the lens comes out, it would be great.No problem, here is the dump:
set_capture_mode -> trunk versionThanks, corrected.
set_capture_mode_type -> necessary for PTP version
But we can unlock it using simple script (value is only for a710):Unbelievable... How did you find out?! (I cannot find it, even knowing what to look for...)
post_levent_to_ui(0x1144) // unnamed event, only for a710
0x1143 value also unlocks keyboard, but blocks power button.
Unbelievable... How did you find out?! (I cannot find it, even knowing what to look for...)Using script with loop, something like:
Today, I tested the newest CHDKCAM and ewavr's BIN, but the "Shooting mode"-Box are empty.Maybe delay for script execution was too small for your camera :(
Hint:
Answer "modes_script" form "main.cpp": > 0000000000000017631c30b7e <
Can you check this? Thanks!
In this moment show me the client an error-box without any message. After "OK" it works.Oops! Fixed (maybe).
Now I know: "start at first the camera and then the application"Just after connecting PC client executes some scripts in camera to read its capabilities (maximum zoom value and available shooting modes). Maybe we need some delay between these events.
p.s. In your camera script autostart is off?Yes, it is.
1. Is it possible to flick through taken images in 'playback' mode without physicaly touching the camera?
2. Is it possible to zoom into an image during 'playback' mode without touching the camera? For example, is there anyway of manipulating the 'zoom slider' so that a taken image can be looked at in more detail?
Other uBasic commands (such as n=1 to 10) appear to be misunderstood.Ewavr's builds (used to) only interpret Lua thru PTP, not uBasic. Therefore, you have to use pure Lua. Not sure about 'press "shoot_half"' though -- isn't Lua supposed to understand that? Otherwise try press("shoot_half") instead. Hope that helps.
have now sorted out scripts with Lua code. Fantastic!Great to hear that your astrophotography setup gets perfect shape, Nick!
Nick
BTW I've noticed that when I reopen CHDKcam, the scripts I have assigned F keys no longer work. That is, only the first 'line' of the script is saved, the rest is deleted. Do you know how to stop this? (Save me re-typing each time).That's a question to ewavr ;)
BTW I've noticed that when I reopen CHDKcam, the scripts I have assigned F keys no longer work. That is, only the first 'line' of the script is saved, the rest is deleted.
It seems not to work after aplying the patch. How can I be sure it works ?
Thanks, now it is OK.
ASSERT!! OperTrns.c Line 594
Occured Time 2009:12:24 19:05:20
TCB: 00212218
Task: tPTPSessio
SP: 0x002120C0
StackDump:
0x00000000
...
(at FFE4EB1C)I am thinking about making a PTP branch in the main SVN. It would make it easier to keep in sync with the trunk (I have no problem with merging my trunk checkins to a branch). Or just put it in the trunk and make it compile time optional ?
connected directly (on board USB, nVidia GeForce 8200 chipset) it behaves very instable, the connection mostly get lost after some seconds.I saw the same thing with SX10 and some computer (I don't remember chipset model). When camera is switched to USB 1.1 ("B") mode, connection is stable, but frame rate is 1..2 fps ;). On the same computer connection with A710 and IXUS700 is perfect.
Trying to shoot after switch to rec crashed the cam.RAW saving is off?
Note, I added add_ptp_handler to sig_ref_vxworks_1.txt, seems to be a good match.Gives good match (though <100%) and finds the function correctly here, too (Ixus950): NSTUB(add_ptp_handler, 0xFF9EC59C).
There remains the mode switching stuff, for which I'd hard-code the "logical events" PressRecButton/PressPBButton -- if I read Microfunguy correctly, this works on most (all) cameras that he supports.It seems that this works only on some VxWorks cameras (a540, a710, ixus700, a610). On DryOS (SX10) "PressRecButton" event has no effect if USB cable is connected.
Maybe the functions used on the other cameras can be found with the sig finder. I was hoping to find an eventproc that did the same thing, but I haven't yet.There remains the mode switching stuff, for which I'd hard-code the "logical events" PressRecButton/PressPBButton -- if I read Microfunguy correctly, this works on most (all) cameras that he supports.It seems that this works only on some VxWorks cameras (a540, a710, ixus700, a610). On DryOS (SX10) "PressRecButton" event has no effect if USB cable is connected.
It worked on the SX200 but I used the binaries from ewavr. I would like to have the worked sources (trunk, patch, etc) because I want to add new features and also to port to SX120. Can Anybody tells me where I can download it ?Hmmm - what are you missing ?
There remains the mode switching stuff, for which I'd hard-code the "logical events" PressRecButton/PressPBButton -- if I read Microfunguy correctly, this works on most (all) cameras that he supports.It seems that this works only on some VxWorks cameras (a540, a710, ixus700, a610). On DryOS (SX10) "PressRecButton" event has no effect if USB cable is connected.
Everything seems to work except switching from record mode to play back.If camera hangs, the problem maybe located in CHDK function:
void switch_mode(int mode)
{
if ( mode == 0 )
{
_Rec2PB();
_set_control_event(0x80000902); // 0x10A5 ConnectUSBCable
} else if ( mode == 1 )
{
_set_control_event(0x902); // 0x10A6 DisconnectUSBCable
_PB2Rec();
}
}
It seems that Rec2PB() address and magic number 0x80000902 are correct... Sorry, I can't find solution remotely, developer with SX1 wanted.
Please read my post a few pages back. Switchting mode is very tricky on G9. I'm currently reverse engineering the whole shooting process to set up an own mode switch. Until now I can run out the lens, activate the image system, set properties and start shooting. But somewhere inside the shooting process the camera comes into panic mode and switches off. I think I missed some preparating call and I did not find all the correct calls until now.
I have a question about adding code to the trunk. When you add code to the trunk, is there a policy that all of the cameras that support the new code have to be updated?No, features can be only enabled for certain cameras. Extra long exposures are and example.
Special application and some general observations. I have attached my SX1 via a USB cable to an ethernet to USB server. That way I can make my SX1 part of my local network. I have actually tried two different kinds of ethernet to USB servers, the results were different, but similar.
1) The frame rate and network utilization is different between the two servers
2) If I turn off the "live image" and the "overlay" the CPU usage of ChdkCam jumps way up. With one of the servers the network utilization stayed very high even without the live feed. With the other server the network utilization dropped to close to 0, but the CPU utilization was higher.
I'm guessing that this behavior is do to the delay in the ethernet to USB server. Obviously, I am not getting as fast of a connection as I do when the camera is connected directly to the USB port on my computer. Perhaps ChdkCam is polling the camera on a regular basis, but with a very short time out. When it doesn't get a response in time it keeps polling, chewing up CPU cycles and ethernet bandwidth. It would appear that ChdkCam needs a bit more optimization for this type of application to work well.
Ewavr, I think I can compile ChdkCam. Do you have any suggestions on what I might change in the code to make this work better.
Thank you.
..., I noticed that there is a file size limit to the files that can be downloaded with my original patch/download functionality. I got a maximum of 251312 while experimenting, which seems to relate to USB packet sizes of 256k.
Regarding downloads and file size limits, there are two options: either use ewavr's method of repeatedly getting parts of the file or, and this would be much nicer if possible, figuring out how the send_data function really works to enable repeated calls (similar to how uploading works).
I don't personally see any reason to add ubasic support. If you restrict yourself to doing the things that ubasic can do, an equivalent lua script isn't really more complicated.
A further (somewhat OT) thought I've had on scripting is to allow PTP to communicate with an already running script, rather then sending a full script to be executed each time. If you also added a way to send messages back, you'd have a very flexible interface with very little code required.
I am not sure what percentage of users will actually use this, clever though it is.
I am certainly not adding Lua support, that will terrify SDM users.And you have every right not to. However, adding Lua support doesn't mean people have to use it themselves or that they can't use uBASIC (assuming support for it is implemented at some stage).
How much does the filesize increase with this compile-option enabled ?This is what I get when building for the IXUS870IS:
| DISKBOOT.BIN | PS.FI2 | |
| full support | 259337 | 153424 |
| no CHDK PTP (undef in camera.h) | 258097 | 152528 |
| no CHDK PTP and no support in platform | 257865 | 152448 |
| clean (i.e. current CHDK trunk) | 257633 | 152320 |
Actually, my current code for the A620 does contain some of your basic ptp code.Although the warning isn't anything serious, the solution I described here (http://chdk.setepontos.com/index.php/topic,4338.msg41513.html#msg41513) is what I've been using everywhere since pretty much that moment.
As I recall, I get a compiler warning about implicit declaration of _ExitTask.
Has anyone here built CHDKCam recently. If so I would appreciate some advice on how to set up the build environment. How difficult is it? Is there a sort of plug and play build tool that will make it simple?I've only looked at the code briefly, but it seems to be build with (and requires) Borland C++ Builder (v6.0). If you have it (or can get it) then that's pretty much a "plug-and-play build tool".
I've just implemented the return-value feedback for script execution (see here (http://github.com/mweerden/CHDK/compare/master...ptp)). With this I think we have a pretty nice initial implementation to add to the trunk. Let me know what you think.great, i'm looking forward a merge with trunk, so every camera can easily benefit of this super addition!
I also figured out how to send events (http://www.mweerden.net/chdk_ptp2.html), but I don't really see an obvious and nice way to use it to get script output. Last time I seem to have conveniently forgotten the fact that events don't "carry" data (apart from three integer parameters). You can use them as notifications, but then you have to store output until it is retrieved by the client. Another option could be to use the parameters to store 4 characters and use as much events per line as necessary, but I'm not sure they'll arrive in order and thus far I've only been able to transmit one parameter per event.i think events could be useful for some chdk custom ptp capture (they could carry a capturefinished event and/or some object handle so one can easily retrieve/delete/whatever the just captured image(s) when they're ready) but i think this is not trivial to do..
i wanted to find ptp entry points for other cameras as well, do i have to follow the "standard" procedure for finding entry points or is there a faster version? :PSort of, by using the signature finder, (http://chdk.wikia.com/wiki/Signature_finder) which is a standard way to find most functions. This proved to work well on a few VxWorks cameras (flip back a few pages and look for reyalp's patch). You can try the same approach for DryOS, which probably is your target. For a start, take the reference A720 firmware, where add_ptp_handler address is known (see ewavr's patch back in the thread), use "gensig" to generate a signature, and try "finsig" with that signature on a different firmware. Of course, you will have to "manually" verify (by visual inspection and/or experiment) that the result is correct. Yes, human intellect still counts...
Note I am using linux.There's no publicly known linux client so far.
How are you getting the live feed from the camera?By repeatedly reading, through "custom" PTP requests, a chunk of camera memory that hosts the live view buffer (it is in YUV format). So far, there is no synchronization or such.
How can I get the return values from a lua script?Read the latest mweerden's posts up this thread and implement his ideas.
Is it possible to execute a script on the camera (merely for ease of use) or must I submit the entire script to be run from the command line?Nobody bothered so far, but it is pretty straightforward to implement.
When I try to issue the "shutdown-hard" or "reboot" commands, I get the following error: "unexpected return code 0x2001".` I took the build from here: http://ewavr.nm.ru/chdk/for_test/. (http://ewavr.nm.ru/chdk/for_test/.)This build is not fully compatible with mweerben's ptp patch: "shutdown-hard" or "reboot" commands are not supported.
when I run "lua shoot()" the camera doesn't do anything and I get no output on the terminal.Try "mode 1" command for switching camera to record mode.
Try "mode 1" command for switching camera to record mode.
I noticed that if I plugged in the USB cable after the CHDK had started on the camera that 4 out of 5 times the camera would switch to "disk" mode (for lack of a better term) and not accept commands and not come out of it until I opened the battery compartment.
char* liveb = NULL;
unsigned char* buf1= NULL;
int WIDTH = 720;
int HEIGHT = 480;
int BUFWIDTH = 720;
long int SIZE = (BUFWIDTH*HEIGHT*6)/4;
liveb = ptp_chdk_get_memory(ADDRESS, SIZE, ¶ms, ¶ms.deviceinfo);
if (liveb == NULL)
return NULL;
buf1=(unsigned char*)liveb;
unsigned char* nastybuf =(unsigned char*) malloc (sizeof(unsigned char*)*SIZE);
if (nastybuf == NULL)
return NULL;
memcpy(nastybuf, liveb, SIZE);
int* col = (int*)nastybuf;
int x = 0; int y = 0;
for (y=0; y<HEIGHT; y++) {
for (x=0; x<BUFWIDTH; x+=4, buf1+=6) {
if ( x < WIDTH) {
*col++=get_pixel_yuv(buf1[1], buf1[0], buf1[2]);
*col++=get_pixel_yuv(buf1[3], buf1[0], buf1[2]);
*col++=get_pixel_yuv(buf1[4], buf1[0], buf1[2]);
*col++=get_pixel_yuv(buf1[5], buf1[0], buf1[2]);
}
}
}
free(liveb);
return nastybuf;the current code returns a nice uchar* with raw rgb data. using the Qt toolkit i can magically convert this to whatever image format i want with two simple lines QImage img(nastybuf, WIDTH, HEIGHT, QImage::Format_RGB32);
img.save("converted_image.bmp");however a cool thing would have been returning "complete" image data instead of raw rgb without involving the use of a complete toolkit in orderd to be fast and more platform-independant as possible (i see ewavr uses Windows' GDI+ for converting the image data in his chdkde tool). I was thinking on linking some c library like libjpeg, what do you think? Is there a simpler solution in your opinion?I'm very new to this and don't understand what to do to get my a720 recognised by Win XP? I have latest chdk build and libusb for windows installed and CHDKCAM too. I cannot get my a720 to become available to the PC at all now, where at least it used to be seen by the Canon pc transfer app before.The PTP stuff is not integrated in the standard CHDK builds from the Autobuild server; there are compiled binaries (i.e. "ready-to-use") for some cameras...
I ran chdkcam in win 98 compatability mode (on XP) and it worked but there is a window asking to select a program to use for my camera. The problem is that the program doesnt appear on the list so I cannot force it to use chdkcam. When I dismiss the window then the program looses control of the camera and it disconnects.
<conn> download /dcim/100canon/test.jpg .
unexpected return code 0x2ff
download failed!I ran chdkcam in win 98 compatability mode (on XP) and it worked but there is a window asking to select a program to use for my camera. The problem is that the program doesnt appear on the list so I cannot force it to use chdkcam. When I dismiss the window then the program looses control of the camera and it disconnects.Although I've got no experience with this on Window, but I suspect Windows is interfering. There is some info on what to disable earlier in this thread.
Anyway, I want to use the PTP interface of chdk with my SX10, but my firmware is 103a. Is there anything I can do to downgrade to 102b, which ewavr's chdk-ptp is built upon?Downgrading seems like overkill to me. How about asking ewavr to include the 1.03a version? Should be almost the same as 1.02b.
Also, is there a list of pt-codes that describe which code does what? I use exclusively Linux, and downloaded and installed libptp2, and can query my SX10is, but it says all the properties are UNKNOWN.Do you mean the general PTP codes or the ones added for CHDK? The latter are described in the ptp.h header in the source and the former (partially) in the PTP specification.
How can I see/make sure that the loaded chdk has the PTP support?There's no way except connecting to with the modified ptpcam (with --chdk) or with CHDKCAM. Not sure what the "policy" is, but I don't see it happening (i.e. as permanent feature).
Does it show some special identifier in the Build Info menu?
If not, could it be added please?
Unfortunately, file upload/download doesn't work:Try A/DCIM/100CANON/test.jpg. If I recall correctly the first A should always be capitalised. Also, specify a filename as destination; I don't remember putting in anything that derives the target filename from the source filename.Code: [Select]<conn> download /dcim/100canon/test.jpg .
Any idea how I can debug file download?Either write to a log file or print messages to the screen.
Try A/DCIM/100CANON/test.jpg. If I recall correctly the first A should always be capitalised. Also, specify a filename as destination; I don't remember putting in anything that derives the target filename from the source filename.
Btw.: What is the meaning of A? Is it the first partionion?I'm not sure it really has a meaning (as the firmware probably always assumes just one partition).
But I can't download files exceeding a specific size, 115KB works, 248KB doesn't.That's a bug discussed earlier here. I've fixed it in my code, but it probably hasn't propagated to other places.
That's a bug discussed earlier here.I've found it here (http://chdk.setepontos.com/index.php/topic,4338.msg48083.html#msg48083). :)
I've fixed it in my code, but it probably hasn't propagated to other places.Can you share it (maybe pm)? Or can it be found somewhere already?
Can you share it (maybe pm)? Or can it be found somewhere already?Here (http://github.com/mweerden/CHDK/tree/ptp). Note that this is a version that is probably not fully compatible with the source you are using. However, the download part should be ok. You can always pm me if need some help with the details.
I try to code one myself right now. But I assume you have a better/nicer solution compared to what what I might
produce ad hoc.
How can I see/make sure that the loaded chdk has the PTP support?There's no way except connecting to with the modified ptpcam (with --chdk) or with CHDKCAM. Not sure what the "policy" is, but I don't see it happening (i.e. as permanent feature).
Does it show some special identifier in the Build Info menu?
If not, could it be added please?
Would init_chdk_ptp be visible in CHDK debug menu task list if PTP support is there?Not likely; it's only to wait for a while until certain things are initialised. As soon as you can connect to the camera it's gone.
B.t.w., I've taken some time to make a wiki page (http://chdk.wikia.com/wiki/PTP_Extension) for the PTP stuff.
I guess liveview really needs USB 2.0.Does chdk need to be fixed to use USB 2.0?
Does chdk need to be fixed to use USB 2.0?
Unfortunately, none of the CHDK developers (whoever they are) took-up your suggestion to discuss a 'basic' PTP feature.That's not entirely true, but I guess the point is that there is nobody with the authority and responsibility to make "real" decisions. That's just the nature of this project.
Do we know what aspects of PTP cause problems and which features are reported as working on all cameras tested ?I'm not aware of any actual PTP problems. For something like "my" minimal extension I'd say the only difficult issue is possibly the mode switching. I don't really have a good overview of the different methods in use and whether there's one that works for all cameras (or for all cameras one, for that matter).
I see that you propose leaving-out mode switching, I assume there is no problem working in record mode, even for file uploading ?Well, I propose moving mode switching from the PTP code to scripting. Again, I'm not aware of any problems, but that doesn't mean much.
We have discussed Lua support previously and in my opinion it attracts a 'certain type' of individual to CHDK.With my proposal both could be supported and easily made compile-time conditional. I can't really see this as an issue.
The general public would not even be happy with uBasic, but at least some would.
On your Wiki, many people would not understand the comments about disabling services.I just took that from this thread as I know nothing about it. Also, it's not mine, it's everybody's.
@mweerden: files seem to be loaded into RAM before written to disk.It's true that ptpcam stores the file in memory first. You should see the tool as a proof of concept; I just took whatever seemed easiest to make work. The original code only provided a function to get all data at once, so that's what I used.
This might cause problems for videos.
I guess the point is that there is nobody with the authority and responsibility to make "real" decisions. That's just the nature of this project.
I would love to get this in CHDK, just been really busy and haven't had time to digest the different variants or proposals. I keep thinking I see the light at the end of the tunnel, but it turns into a train every damn time.I guess the point is that there is nobody with the authority and responsibility to make "real" decisions. That's just the nature of this project.In that case, I vote for reyalp :)
Not really what I meant. ;)I guess the point is that there is nobody with the authority and responsibility to make "real" decisions. That's just the nature of this project.In that case, I vote for reyalp :)
If mweerden wants to post the definitive patch and have fe50 or myself commit it, that's fine with me.I'll look over it and make a patch, but I think it would be useful to have ewavr's opinion before adding anything. Especially w.r.t. the interaction with CHDKCAM - most likely the prime interface for most users - his view on the matter should have some weight.
I'd also support giving him commit access if he wants to continue developing this.As I don't really see myself as an active party in this, I don't think that will be necessary.
Downgrading seems like overkill to me. How about asking ewavr to include the 1.03a version? Should be almost the same as 1.02b.
Also, is there a list of pt-codes that describe which code does what? I use exclusively Linux, and downloaded and installed libptp2, and can query my SX10is, but it says all the properties are UNKNOWN.
Do you mean the general PTP codes or the ones added for CHDK? The latter are described in the ptp.h header in the source and the former (partially) in the PTP specification.
Here's a patch with my proposed changes.
- http://chdk.wikia.com/wiki/PTP_Extension (http://chdk.wikia.com/wiki/PTP_Extension) lists memory read/write and remote function calls as excluded from the patch but they appear to be included in the proposal patch ptp.c anyway.Indeed. I corrected the wiki.
- I added a570 ptp startup to platform/generic/main.c and don't know how to do it any other way. The proposal patch adds code for ixus870_sd880 in each sub's boot.c because apparently some asm magic is required. I can only assume my addition interferes with it. How to deal with this?There's no magic involved. ;) I just added it where SpyTask is created, but your methods seems to be fine as well (as long as it's platform independent, I guess). It seems a few different approaches are in use.
- It would be easier to review the patch if tools/ptpcam stuff would be in a separate diff.I'm not sure I see why. In the patch itself all the non-ptpcam stuff comes first and if applied, it creates a new directory for ptpcam. But if you want I can split the file.
- Script command reboot() should be in a separate diff because it's not related to PTP.Perhaps. I tend to see it as primarily useful in the context of (debugging via) PTP.
- Maybe switch_mode_usb() lua command should be renamed to something like set_record_during_ptp or maybe set_record() could be given another option to signal that this alternative functionality is desired.W.r.t. consistency in naming you're absolutely right. Though set_record is a horrible name in my opinion. ;) I did use usb instead of ptp because the additional code seems to be USB specific and not so much directly related to PTP. Perhaps it's also useful in non-PTP communications some day.
- What is PTP_CHDK_TempData?It's a command to temporarily store data or receive such data. It's useful in cases like downloading where you have to specify a filename but cannot transmit it in the same transaction as the actual download (due to the limited PTP arguments and the fact that each transaction can only send data in one direction).
I think remote function calls should be disabled by default like they are in Lua, e.g. using buildconf.inc (OPT_PTP_CALL_NATIVE). Maybe even poke?- http://chdk.wikia.com/wiki/PTP_Extension (http://chdk.wikia.com/wiki/PTP_Extension) lists memory read/write and remote function calls as excluded from the patch but they appear to be included in the proposal patch ptp.c anyway.Indeed. I corrected the wiki.
There's no magic involved. ;) I just added it where SpyTask is created, but your methods seems to be fine as well (as long as it's platform independent, I guess). It seems a few different approaches are in use.I'm not an expert here I'm afraid but if the code in platform/generic works on many cameras and only some or none require platform or sub specific code, this definitely should be done in the generic code whenever possible, right?
I didn't know there wasn't any chdk stuff among the ptp files, which made reading the patch slow...(I use a text editor for this, no dedicated tools). When committing this stuff to svn, the changeset diff view in assembla web interface could suffer from the same problem.Quote- It would be easier to review the patch if tools/ptpcam stuff would be in a separate diff.I'm not sure I see why. In the patch itself all the non-ptpcam stuff comes first and if applied, it creates a new directory for ptpcam. But if you want I can split the file.
Quote- Script command reboot() should be in a separate diff because it's not related to PTP.Perhaps. I tend to see it as primarily useful in the context of (debugging via) PTP.
Agreed...Quote- Maybe switch_mode_usb() lua command should be renamed to something like set_record_during_ptp or maybe set_record() could be given another option to signal that this alternative functionality is desired.W.r.t. consistency in naming you're absolutely right. Though set_record is a horrible name in my opinion. ;) I did use usb instead of ptp because the additional code seems to be USB specific and not so much directly related to PTP. Perhaps it's also useful in non-PTP communications some day.
I think remote function calls should be disabled by default like they are in Lua, e.g. using buildconf.inc (OPT_PTP_CALL_NATIVE). Maybe even poke?This is something I'm not opposed to, but I do wonder why.
I'm not an expert here I'm afraid but if the code in platform/generic works on many cameras and only some or none require platform or sub specific code, this definitely should be done in the generic code whenever possible, right?Ideally, yes. At the moment it seems the opposite is (mostly) the case.
I didn't know there wasn't any chdk stuff among the ptp[cam] files, which made reading the patch slow...(I use a text editor for this, no dedicated tools). When committing this stuff to svn, the changeset diff view in assembla web interface could suffer from the same problem.Ok, I can see that this might not be clear. I guess I just assumed that it was clear that it's a completely separate tool (code wise). One could indicate it in the commit message, but I guess committing them separately would be clearer.
Related to this, ptpcam sources should probably somehow state from which version of the original ptpcam it was forked. Committing the original in one patch and then your CHDK'ed ptpcam changes in another changeset would leave a cleaner track I suppose?If you wish to be able to distinguish the code I've written from the original, yes. Otherwise a line in README or the commit message should suffice. Personally I wouldn't (and didn't) bother because I'd call the link with the original ptpcam "accidental".
This is something I'm not opposed to, but I do wonder why.Because having it enabled by default would make it possible for a psychopath to e.g. author a malicious program that clears the ROM of your camera once connected to a windows computer and distribute it on the internet under the name "l33t appz0r to double the mpiX3lz on ure powash00t lawl.exe". I.e. sort of the reason why it's disabled for Lua by default, but squared since you can actually review Lua scripts you run on your camera but you can't do that for binary malware that runs against your will.
Ideally, yes. At the moment it seems the opposite is (mostly) the case.Okay, I suppose cams like a570 that can live with the generic code should then have something like #define PTP_GENERIC_STARTUP 1 in camera.h?
Because having it enabled by default would make it possible for a psychopath to e.g. author a malicious program that clears the ROM of your camera once connected to a windows computer and distribute it on the internet under the name "l33t appz0r to double the mpiX3lz on ure powash00t lawl.exe". I.e. sort of the reason why it's disabled for Lua by default, but squared since you can actually review Lua scripts you run on your camera but you can't do that for binary malware that runs against your will.I don't think those are very good examples. I'll keep it on security for the paranoid and the off chance that cameras with CHDK become so widely used that people will want to display ads on them or something. ;)
Isn't this already taken care of by CAM_CHDK_PTP? (As far as I know the other cameras don't call task_start_hook.)QuoteIdeally, yes. At the moment it seems the opposite is (mostly) the case.Okay, I suppose cams like a570 that can live with the generic code should then have something like #define PTP_GENERIC_STARTUP 1 in camera.h?
Btw, I added a command line interface to your CHDK-ptpcam.Nice. This has been on my todo(-butnotlikelytoeverbedone)-list for some time.
My Ubuntu uses PTP for normal camera USB access, which interferes with ptpcam. Basically the first ptpcam connection attempt nearly always fails, but retrying solves this. So I changed ptpcam to retry before giving up.If it works, great. I think it's also useful when rebooting; I've noticed that my just-wait-a-few-seconds approach didn't always work so well.
I also changed the 'luar' command a bit.My intention was that you'd be able to type luar 2*21 and get 42 back without having to add return. If I understand the Lua semantics correctly, and I believe I tried it as well, without the return you don't get 42 back. Also, having more complex commands (or actually commands that take more time) can have undesired effects due to the fact that the PTP handler waits for completion.
The modified files are CHDK/tools/ptpcam/ptpcam.c and ptp.c. You can find them and a simple example usage shell script in http://drop.io/a570_chdk_ptp (http://drop.io/a570_chdk_ptp) file ptpcam-climod.zip.In ptp.c, the malloc/sprintf is now pretty useless; I'd remove that (together with the later free). As for ptpcam.c:
Isn't this already taken care of by CAM_CHDK_PTP? (As far as I know the other cameras don't call task_start_hook.)
If it works, great. I think it's also useful when rebooting; I've noticed that my just-wait-a-few-seconds approach didn't always work so well.
My intention was that you'd be able to type luar 2*21 and get 42 back without having to add return. If I understand the Lua semantics correctly, and I believe I tried it as well, without the return you don't get 42 back. Also, having more complex commands (or actually commands that take more time) can have undesired effects due to the fact that the PTP handler waits for completion.
I do understand the use of your version, but I'd probably either use (my) luar (function () body end)() (not tested) or make another command for it.
In ptp.c, the malloc/sprintf is now pretty useless; I'd remove that (together with the later free).
- Apparently you have some automatic style changer or something. Diffing your version with mine gives a lot more than one wishes. (Deja vu! J/k ;))
- It seems open_camera tries a (MAXCONNRETRIES+1)th time or, in case of success before, always calls ptp_opensession another time. Perhaps moving the call in the condition and the code after (and just before) the loop inside (into a "big" if) is a solution.
- How about just a char *buf = chdkarg; in chdk()?
The reboot command is something I won't be able to test as there is no such thing for the vxworks cams.
- Something to be aware of for the CLI is that the reboot commands try to reconnect after issuing the reboot. It's probably still the right behaviour as it gives you the result of the reboot (without having to look at the camera).
The reboot command is something I won't be able to test as there is no such thing for the vxworks cams.
I am trying to write a value to the" PARAM_FILE_COUNTER" by using " set_parameter_data" but the camera crashes down ,
any idea why ?
Actually...is there a reason for the lua command not to give a return value?The reason is that at the moment the task handling the PTP command will wait until the script is done to send back the return value and I'm unsure of the consequences of waiting for a long time. Perhaps it's completely harmless. An alternative is to make it asynchronous, having the camera send an event on completion or having a command to poll for a result.
I only get one change you made in ptp.c:In ptp.c, the malloc/sprintf is now pretty useless; I'd remove that (together with the later free).
I'm not 100% certain which one you mean...
The reason is that at the moment the task handling the PTP command will wait until the script is done to send back the return value and I'm unsure of the consequences of waiting for a long time. Perhaps it's completely harmless. An alternative is to make it asynchronous, having the camera send an event on completion or having a command to poll for a result.
Please excuse me if this question has been asked.
I was wondering what bytes store the image on the LCD for the A590 IS. PTPCAM has the ability to read bytes which is quite useful. I want to stream whatever is on the LCD display live. I just don't know what bytes store the LCD display. What should I read to figure this out? Or, is it documented somewhere?
In theory, there is a Canon PTP operation code (0x901d) to send the 320x240 JPG viewfinder image.
Has anyone been able to read the values of Canon custom operation codes (9xxx) ?
Type ptpcam -o to see the codes and ptpcam --show-property=9002 for example.
Are you saying that the DRYOS-based cameras will work with PTP in Record mode without any mode switching from playback ?
My VxWorks A620 only works if connected in Playback mode and then switched to Record by a 'mode 1' command with ptpcam.
void switch_mode(int mode)
{
if ( mode == 0 )
{
PostLogicalEventForNotPowerType(0x1003);
} else if ( mode == 1 )
{
PostLogicalEventForNotPowerType(0x1001);
}
}
Is it worth adding the signature for set_control_event ?If it's needed by many cameras, and matches reasonably well.
Are you saying that the DRYOS-based cameras will work with PTP in Record mode without any mode switching from playback ?No. I'm saying that (some ? most ? all ?) dryos cams will need something more than PressRecButton (0x1001)/PressPBButton(0x1003), possibly involving set_control_event and/or pb2rec etc. so they can switch to record mode in PTP. Without this, shooting will not be available.
My VxWorks A620 only works if connected in Playback mode and then switched to Record by a 'mode 1' command with ptpcam.
Hej,Unfortunately, the current version is not compatible with chdkcam. You have to use ptpcam patched with mweerdens code, and as mentioned before there's no support for chdkcams live view.
Well it's great news, except I couldn't make it work on my A540. :/
I compiled it with the option enabled and got the tool from ewrar website, I also needed some libusb but not sure if I got it right, anyway nothing happens when I start CHDKcam. :(
I feel kinda left-handed with all that, please someone could sum up again the steps to make it work, thanks.
Hej,My build of ptpcam is here http://drop.io/reyalp_chdk/asset/ptpcam-mingw-reyalp-test-zip (http://drop.io/reyalp_chdk/asset/ptpcam-mingw-reyalp-test-zip)
Okej guys thanks for the answer, there's what I missed : PTPcam. Is there a usable version downloadable somewhere ? I managed to compile CHDK with the help of whim's tool, but I don't think I'll go further than that. :/
How far are we from being able to use CHDKcam ?1) CHDKcam is written with Borland c++ builder, so people without that can't easily update it. I don't know if there is a free tool that can build this code (Borland released some of their command line compilers for free, but it's been a long time since I looked at this)
CHDKcam is written with Borland c++ builder, so people without that can't easily update it. I don't know if there is a free tool that can build this code (Borland released some of their command line compilers for free, but it's been a long time since I looked at this)https://downloads.embarcadero.com/free/c_builder (https://downloads.embarcadero.com/free/c_builder)
My build of ptpcam is here http://drop.io/reyalp_chdk/asset/ptpcam-mingw-reyalp-test-zip (http://drop.io/reyalp_chdk/asset/ptpcam-mingw-reyalp-test-zip)I tried to build my ptpcam mod with mingw (32-bit vista) once and finally got it to compile but wasn't able to connect to a camera. I'm pretty sure I didn't install enough/correct device drivers/whatever, so I have no clue what part isn't working so just gave up...I have next to no patience when it comes to things on windows, has anyone written a beginner's guide to getting this thing to work there? :'(
I have next to no patience when it comes to things on windowsQuite understandable. Although, IMHO, it is better to do exercise patience in the face of such a calamity.
It appears to me that if you plug the camera into different USB ports, you have to set up the filter once for each one.
I also had trouble if I didn't set the action when the camera is connected to "none". This again has to be set for each port.
ptp.param1=((int(*)(int,int,int,int,int,int,int,int,int,int))buf[0])(buf[1],buf[2],buf[3],buf[4],buf[5],buf[6],buf[7],buf[8],buf[9],buf[10])
This will work with the functions that nicely return with "BX LR" or similar, but will likely crash at the often encountered MOV PC,LR, failing to return to thumb. Also, vararg functions cannot be called this way.Does PTP really need its own native calls interface separate from Lua?Good question. There are currently a couple of differences
Hej,Hello,
Well it's great news, except I couldn't make it work on my A540. :/
I compiled it with the option enabled and got the tool from ewrar website, I also needed some libusb but not sure if I got it right, anyway nothing happens when I start CHDKcam. :(
I feel kinda left-handed with all that, please someone could sum up again the steps to make it work, thanks.
NHSTUB(get_ptp_handler, 0xFFA4EC7C)
//NHSTUB(add_ptp_handler, 0xFFA4EABC) // sigfinder address ok
NHSTUB(remove_ptp_handler, 0xFFA4EBB0)
NHSTUB(PB2Rec, 0xFF898B40) // search String "AC:PB2Rec"
NHSTUB(Rec2PB, 0xFF897590) // search String "AC:Rec2PB"
NHSTUB(set_control_event, 0xFF8955C4) // via eventproc_export_IsControlEventActive (last call), levent_table contains control event id's (Logical Event Table)
//NHSTUB(reboot_fw_update, 0xffa9b228) // sigfinder address ok
Is the patched ptpcam source available somewhere (didn't find anything besides original ptpcam source) ?You can find chdk-ptpcam source in mweerden's proposal CHDK patch, it goes in trunk/tools/ptpcam:
A590-patch: add only A590 PTP-functions (with mode-command)
In current version I don't know is lua-script finished or not
I detect a problem on use eg. large for-loops (runtime over 5 sec.) without any sleep-command inside the loop. The timeout error come back.
I have a solution for that. It is part of my next patch in a series to disable scripting in CHDK. Will be posting it soon.Thanks, that sounds good. I'm waiting for.
lua_pushnumber(L, switch_mode_usb(mode)); //success to stack
return 1; //count of return value on stackFor windows, the easiest route is this one (I'm not 100% sure if this matches proposal patch version functionally):I'm pretty sure it doesn't match. I can't quickly build a new version at the moment, but let me know if it's needed. I'm guessing that others have more up-to-date versions.
http://www.mweerden.net/download/chdk-ptp/libptp2-chdk-win32.zip (http://www.mweerden.net/download/chdk-ptp/libptp2-chdk-win32.zip)
If I want to implement new PTP-query in the camera that returns only a single number or two, is my understanding of the code correct that I only need to do the following?It seems so, yes.
I think, I found the answer for "switch_mode_usb()"-result. It is a tricky tri-state result (mode-val on stack for switching ok and nil for any error at switching). Is that correct?Seems so.
//equal levent_set_record();
<conn> lua post_levent_to_ui("PressRecButton");post_levent_to_ui("UnpressRecButton")
//for example make a shoot
<conn> lua shoot()
//equal levent_set_play();
<conn> lua post_levent_to_ui("PressPBButton");post_levent_to_ui("UnpressPBButton")The best is, all cams use only one routine. We don't need to find entrypoints for set_control_event, PB2Rec and Rec2PB on DryOS-cams. I think the two methods are a little bit different and the event-method is a better way (feel stable on tests).I don't understand why native call is required, you can use without levent native call.
The downside is OPT_LUA_CALL_NATIVE is required for OPT_PTP, but for VxWorks-cams it is already required.
One reason to use ubasic is cameras with very little memory. Once your scripting optional stuff is done, people will have the option of ubasic only, and might still want ptp.But this argumentation is only valid if OPT_LUA_CALL_NATIVE is not made a requirement for OPT_PTP.
The way things are now, the "supported" bit will be (1<< (executescript number)).What is "executescript number"?
There is no reason CALL_NATIVE should be required for OPT_PTP. At most you'd need to disable PTP_CHDK_CallFunction (although I'm not sure that's really needed)One reason to use ubasic is cameras with very little memory. Once your scripting optional stuff is done, people will have the option of ubasic only, and might still want ptp.But this argumentation is only valid if OPT_LUA_CALL_NATIVE is not made a requirement for OPT_PTP.
PTP_CHDK_SL_LUAThe way things are now, the "supported" bit will be (1<< (executescript number)).What is "executescript number"?
Is there a wiki or a text explain how ptpcam work ?I have not had much luck with non-chdk commands in ptpcam. To upload/download files, use
I want transfer the chdk files to camera so i need not put SD-Card in Card reader always.
I search in google and see the option --get-all-files for testusage of ptpcam
seem my camera work with ptp.
when i type this in, LCD display get dark, and the LED blink for about 1 sec.
then come the output.
I can redo this command and work always same.
C:\Users\pc>E:\chdk\ptpcam.exe --get-all-files
Camera: Canon IXUS 1000HS
C:\Users\pc>
but some new files are not in c:\users\pc written
I dont like using the shell special windows shell, and i always forget how can in windows change the drive, maybe there is also a GUI for newest ptpcam, that allow the transfer of camera binary zip files to Camera easy ?
<conn> upload test.txt a/test.txt
I don't understand why native call is required, you can use without levent native call.That's right. For mode-switching we don't need OPT_LUA_CALL_NATIVE. Thanks for your hint.
It looks like the code you are using is identical to lua set_record, which doesn't work on (most ?) dryos cameras when USB is connected. I know I tried a bunch of things on D10 and couldn't get it to switch using levent.Yes, it was my idea. It's a pity. It would be so easy to switch mode on DryOS cameras. Now, I will find the entrypoints for DryOS reference BINs.
<conn> upload test.txt a/test.txt
Write the camera driveletter uppercase e.g. upload test.txt A/test.txt
I try that but not work.My guess either you have not correctly implemented and enabled CHDK PTP, or your libusb driver is not set up correctly. There is some discussion of the latter in this thread.
I also see, when remove USB cable error is same.so seem ptpcam do just nothing in --chdk
<conn> upload test.txt A/TEST.TXT
unexpected return code 0x2ff
upload failed!
C:\Users\pc>e:\chdk\ptpcam.exe --get-all-filesCHDK only uses ptpcam as a convenient place to put some code. As I said earlier, I wasn't able to get many of the non-chdk ptpcam functions to work. It appears to me that ptpcam development was abandoned a long time ago.
Camera: Canon IXUS 1000HS
C:\Users\pc>
I read some posts, but find no source of ptpcam
and what need change in chdkcam to get it working ?Two things.
there is for Ixus 1000 trunk 958 trunk 969 and trunk 1004 workingCamera protocol version should be 0.2 if using current trunk code, but using 0.2 ptpcam with 0.1 chdk is fine, only the new script-support and script-status commands will fail.
I can revert to an old trunk,
Do you get not such a message on ptpcam --chdk ?
"""""
error: camera has unsupported camera version 0.1; some functionality may be miss
ing or cause unintented consequences
""""""
ptpcam --chdk
upload ptpmsg.lua A/ptpmsg.lua
lua loadfile('A/ptpmsg.lua')()
putm ls A/
getm
exec return 1+1
getm
@reyalpIt's a C++ builder app, so it needs the corresponding c++ builder libs. Definitely not portable. I don't know whether the free Borland command line tools (linked earlier in this thread) are sufficient to build it, and I'm not really interested in spending the time to find out.
>- you need to build it, which requires borland c builder
hope this is not too complicate and use special borland libs, i can try when i have time to compile with wxdevcpp.it use then wxwidgets(work on win /Linux/Mac OS/ Unix too).
error come and i can now upload files.This seems normal to me. If you unplug the camera, then you need to do reset command to reconnect.
only i see that ptpcam do no full init after a conn command.
for example when i do this lines
C:\Users\pc>e:chdk\ptpcam.exe --chdk
<conn> upload ps.fi2 A/ps.fi2
<conn> upload ps.fi2 A/ps.fi2
now i remove the USB cable on camera and then i put it in again
then i get that output
<conn> upload ps.fi2 A/ps.fi2
unexpected return code 0x2ff
upload failed!
<conn>
ptpcam --chdkupd ......zipThe fugeys cli stuff that I added in the latest ptpcam also lets you do
I see only little Problem in Ixus 1000 that when the USB cable is plug in, the firmware keys do not work.So when i switch camera on in play mode, i can not press menu to do firmware update.Yes, cannon keys are disabled. Alt mode should still work.
I need remove the USB cable
is this on other camera same ?
<conn> lua write_usb_msg(tostring(1 + 1))
<conn> getm
message 1 size=1
2
<conn>
exampleCode: [Select]ptpcam --chdk
upload ptpmsg.lua A/ptpmsg.lua
lua loadfile('A/ptpmsg.lua')()
putm ls A/
getm
exec return 1+1
getm
You need to build a patched PTPcam too. Patch is against the source in chdkde
exampleCode: [Select]ptpcam --chdk
upload ptpmsg.lua A/ptpmsg.lua
lua loadfile('A/ptpmsg.lua')()
putm ls A/
getm
exec return 1+1
getm
Not sure what I'm doing wrong here; but 'putm' and 'getm' give 'unknown command' error message.
Using ptpcam from 'Ptpcam-chdkde-r515.zip' and I've applied the CHDK patches from 'ptpmsg-work-2.zip' and rebuilt (camera is SX30).
I've shamelessly stolen most of chdkde changeset 530 (http://tools.assembla.com/chdkde/changeset/530), trunk changeset 1018 (http://tools.assembla.com/chdk/changeset/1018)Such a thing! But you must it do another one, because the G12_100c is also ready.
I've shamelessly stolen ...This is not shameless, that's great. So we have more opportunities to locate any errors.
I see they are also working on a PTP GUI ...The GUI design is a small attempt to help people who have problems with a console. So we get more feedback (maybe). Wim will probably laugh at this Autoit attempt. ;)
The "switch_mode_usb"-routine for all cams are in "/generic/lib.c", see changeset 465 (http://my-trac.assembla.com/chdkde/changeset/465) and 470 (http://my-trac.assembla.com/chdkde/changeset/470).Thanks. Equivalent changes are in trunk 1022-1023. I did it a little differently so I wouldn't have to touch every lib.c
unless you hardcode some actions in the ptp code, the possibilities without scripts are very tiny (upload, download, mode change?)We could expose the "action stack" commands and PostLogicalEvent with very little overhead.
No, no, no, bad conclusion: if we are going to end up with more memory from this exmem thingy, then use it for compiled PTP (and better zebra for Ixus 95). CHDK is for photographers, scripting is for programmersI disagree completely with this, but it's off topic to this thread. Feel free to start a new one in general discussion about CHDK development directions.
But to stay on the topic, I do agree to keep PTP-functionality minimal. However, I would add just two more commands: 1) shooting a picture and 2) transferring a single frame from the live buffer. Note that I'm NOT proposing webcam-like streaming or anything that would require real-time transfer rates. Merely a request to send over a single frame from the current live buffer. I'd propose these in addition to what is already there to be available without scripting, rest to be scripted. Reason is that then the PTP feature set would be enough and adaquate for basic computer controlled camera without the overhead of scripts.
If PTP must be kept somehow useful without scripting, then instead of a shooting command I'd go with access to half+full shutter press/release (or even better, to the entire keyboard). But my vote would go against that.As I said, it would be quite easy (almost trivial) to add "action stack" and/or levent support. I'm pretty tempted to do this, just because it would be a tiny amount of code, and would allow basic camera control natively.
AFAIK, CHDK PTP doesn't currently support UBASIC, but that's not the only thing that's not quite there yet -- Lua is just so much more powerful that it hasn't made any sense to use anything but it so far.Agreed.
My understanding is that most users who do not want scripting at all don't want it because it complicates camera usage, mainly because of the alt+shutter shortcut which is sort of a trap for beginners and those that don't know CHDK is running on the camera someone gave them.
As for live view, ewavr already implemented it in real time (more or less, depending on USB speed). If my memory serves, the main obstacle for adding that to the trunk CHDK PTP is that it needs some work to make it more generic across cameras with different sorts of live view buffers, resolutions and colors -- something that probably doesn't become much easier even if it's not in real time.It has always been my intention to add this back. This is one of the main reasons I've started a new gui tool.
This adds the necessary functions for mode changing on most dryos cameras. I haven't turned it on in camera.h, but this should give us what we need to support PTP out of the box on almost all cameras.
Probably. The entry points for the various functions may not have been identified correctly on your camera, but most should be OK.This adds the necessary functions for mode changing on most dryos cameras. I haven't turned it on in camera.h, but this should give us what we need to support PTP out of the box on almost all cameras.
@reyalp So at this point do I simply need to use the latest trunk release, add #define CAM_CHDK_PTP in the appropriate spot in camera.h, enable the OPT_PTP box in the CHDK-Shell compile options, rebuild and start testing ?
If people can report which cameras have been tested successfull (including mode switching through PTP) that would be helpful
If people can report which cameras have been tested successfull (including mode switching through PTP) that would be helpful.No luck for the SD940 with just enabling CAM_CHDK_PTP. Camera is detected by PC but crashes as you try to open folders.
EDIT : Hmmmm ... CreateTask_init_chdk_ptp() exists in some copies of boot.c. Time to see what that's all about.It should be removed from any boot.c it exists in, started from spytask now.
It should be removed from any boot.c it exists in, started from spytask now.Thanks. One less thing to worry about. Its commented out in the g12 and sx30 of the current trunk.
Hi allWhat ptpcam binary are you using ? What a495 build are you using ?
I'm trying to use PTPCAM whit a A495 and 'm getting the following error
c:\Ptpcam-chdk-1005>ptpcam --chdk
unexpected return code 0x2005
error: cannot get camera CHDK PTP version; either it has an unsupported version
or no CHDK PTP support at all
<conn>
Does anyone know what can be that?
The command move out the lens and Camera switch off with lens out.no image see, seem crash.later commands with ptpcam give same error message as no camera is connectMore likely some entry points (pb2rec, rec2pb, set_control_event) are wrong.
I use trunk 1023.maybe this is too old ?
More likely some entry points (pb2rec, rec2pb, set_control_event) are wrong.
This will break compatibility with previous protocol, but might be a better use of time than debugging this.
Agreed. I'm working on this now (instead of my new ptp tool :-[ )This will break compatibility with previous protocol, but might be a better use of time than debugging this.
I don't see a reason to maintain compatibility -- ptp has been announced as experimental and subject to change for the better, and the particular flavour now in trunk probably doesn't even have many users yet (at least not the kind that gets confused by change) given the state and age of the PC side tools.
@reyalpYou are right, missing ptp_script.h. It's only prototypes for some functions which are in ptp.c. Attached.
Thank you for your tireless work.
I think the last patch missing ptp_script.h / ptp_script.c. The compiler generates an error message.
If I understand this patch correctly, it is the right way. We need a good solution, with which we can send e.g. mod maps or directories from the camera to the computer. In addition, there should be only one command for script functions, lua and not another luar.Agreed. New client I'm working on is mostly lua, so you can send lua table from camera as string, make back into table and use very easily.
Then we have all the options to build a GUI.Lua client is nice. Real example:
button_shoot=iup.button{
title="shoot",
action=function(self)
chdk.execlua('shoot()')
end,
}
So what is luar ? And how is it different than lua ?They are commands from the ptpcam tool. One just runs a script, the other also waits for a return value. It was discussed in this thread a while ago. The reasons why they both exist (in ptpcam, mind you) were also discussed.
But it help not on the switch to record mode crash.I seem to remember it being mentioned here before that some cameras work with the "normal" mode switch (i.e. without doing PB2Rec or something) in Lua.
I dont know, whats diffrent, but it work, when i start a script in play mode, Camera switch to record mode, as soon a shoot command come and shoot a image.
but there seem no pb2rec use for this.
From what I've seen/read I really like reyalp's message-queue patch and would suggest it above his getr approach (if that's still on the table after rudi's fix). It doesn't really seem more complex and probably provides all the flexibility you'll "ever" need.Yes, i thought getr was going to be simpler, but it didn't really turn out that way. My problem with the message stuff was it was too generic: For most cases, you probably just want to run a script and return a value, and it doesn't really distinguish those form user defined messages. Having a status available for each script invocation is also important.
I think the IX1000 can work too with normal mode switch, i test with USB cable in, it work ok.So set_record(1) works ? This is very surprising, it doesn't work on any other recent dryos cameras
Yes, i thought getr was going to be simpler, but it didn't really turn out that way. My problem with the message stuff was it was too generic: For most cases, you probably just want to run a script and return a value, and it doesn't really distinguish those form user defined messages. Having a status available for each script invocation is also important.Why all this hate towards luar? ;)
Definitely going to remove the current luar method one way or the other.
>So set_record(1) works ?This is the lua function that switches between record and play normally, when PTP isn't involved. This also works in PTP for most vxworks cameras.
I dont understand what ypu mean.do you mean in C source or as lua command use set_record(1) ?
I have USB cable plug in camera and press when camera is in play mode and alt mode the shoot button full.this script switch then to record mode and shoot the images without problems.please read below tooIf I understand correctly, this would also be very unusual. Normally when the camera is connected by USB, it doesn't respond to any canon buttons. CHDK alt buttons are expected to work.
I'm pretty sure this is what mweerden was referring to.Me too.
so maybe can call what this script do too with C functions ?Try levent_set_record() (and levent_set_play() for set_record(0)).
I add
set_record(1);
in C source, but give error
I try now with ptpcam(with the lua fix) this lineAs I told you before, this method only works on old cameras, it would be very surprising if it worked on yours.
C:\Users\pc>e:\chdk\ptpcam --chdk="lua set_record(1)"
levent_set_record();Pointless, this is the same thing is the lua call would do. If the lua version doesn't work, then this also will not work.
_set_control_event(0x902); // 0x10A6 DisconnectUSBCableRandom mixing and matching code is unlikely to help. It's possible the value for set control event is wrong.
//_PB2Rec();
levent_set_record();
maybe Camera have diffrent values for diconnect USB cable ?
But what does levent_ mean ?"logical event" from the camera functions PostLogicialEvent
is this the code lua execute.Why don't you look and find out for yourself ?
What C code do the basic script execute ?
It seems diffrent because the basic script switch when call shoot from play mode to record mode and shoot correct when USB cable is plug in.LUA do this not.It's not different, shoot uses exactly the same code. Shoot does not switch any other camera to record mode with USB connected that I know of.
I find now wy shoot can work on my Camera.its because key mask2 for pysw_status[2] is wrong.If you mask the USB power bit (for example, by enabling USB remote), then your cameras isn't in PTP mode, so you can switch and shoot without any special tricks, it's just like unplugging the cable.
Do somebody know if there are a version of ewavr's application that works with A495 or A480? When I try to use that crashes camera, currently I'm using the CHDK 1053 trunkNo there is not. As I've explained before, getting any of this working requires hacking.
ptpCamGui just needs a recent ptpcam.exe, and (on the cam) a plain vanilla CHDK-DE trunk to run.Hmmm... trying to get PTP working with the current trunk version of CHDK. I'm not too interested in exploring the German versions of CHDK - I'm just working on a beta for the SD940 in the current trunk.
Does ptpCamGui need CHDK PTP to support commands not in the current (not-DE) trunk ? And if so, then can the CHDK_GUI be modified to not require the DE version of CHDK ?Good questions, but wrong person to address them to :D
Does ptpCamGui need CHDK PTP to support commands not in the current (not-DE) trunk ? And if so, then can the CHDK_GUI be modified to not require the DE version of CHDK ?
I've tested the current trunk (rev1053) and the GUI work fine
A CHDK-DE build trunk560 or newer
is required both in CHDK-Shell and
on your camera
I can issue commands via ptpcam and send LUA scripts that execute. However, once a USB cable is plugged in, the camera buttons stop working and if the camera was in shooting mode it switches to playback mode.This is completely normal.
Where do I start looking ?Use switch_mode_usb. If it doesn't work, implement it correctly for your camera.
Do I need to enable remote modeAbsolutely note.
- I thought that might interfere with PTP ?Correct, enabling the remote will completely disable all USB protocol functionality. "USB" remote is not a USB protocol connection, it's just detecting whether there is power on the USB port. To make this useful, it hides the signal from the canon formware (which in turn hides it from CHDK PTP).
does this mean, i must activate remote on in chdk menu, to get ptpcam correct working ?
@reyalpNo, it means completely the opposite. How can I make this more clear ? CHDK "USB REMOTE" PREVENTS THE CAMERA FROM SEEING A USB CONNECTION AT ALL. If you look at the code, this should be quite obvious. If you test it, this should be obvious. If you read the wiki pages on how to build and use the remote, it should be obvious.
does this mean, i must activate remote on in chdk menu, to get ptpcam correct working ?
Also, you really don't need to post a romlog for every crash in your broken port. It's up to *you* to fix it.
Since i've discovered UART for SD4000 i've start documenting and playing with Event Procedure (http://chdk.wikia.com/wiki/Event_Procedure)...This works with PTP connected ? Very interesting. If it works on other cameras, this might be a good alternative to the PB2Rec/REC2PB/set_control_event stuff.
On SD4000 this works for switching Playback / Record mode:
1. register required functions with UI.Create
2. switch mode:
UIFS_Capture - Switch to Record Mode or start Capture if we are already in Record Mode
ModeDialToMovie - Switch to Movie Record Mode
UIFS_SetDialPlay - Switch to Playback Mode
3. switch to different Still Image Capture modes:
ModeDialToAuto
ModeDialToPortrait
ModeDialToKidsAndPets
Use switch_mode_usb. If it doesn't work, implement it correctly for your camera.
On SD4000 this works for switching Playback / Record mode:
@pixeldoc2000 : Any chance you can share a code snippet using this to implement switch_mode_usb() ?I only did some quick test using UART console on SD4000. Probably the easiest way would hack a little Canon Basic script. If script does work on other cameras we can implement an alternate switch_mode_usb() for particular cameras.
You should be able to do this using eventprocs from lua ? You can even replace the existing switch_mode_usb function if you want...@pixeldoc2000 : Any chance you can share a code snippet using this to implement switch_mode_usb() ?I only did some quick test using UART console on SD4000. Probably the easiest way would hack a little Canon Basic script. If script does work on other cameras we can implement an alternate switch_mode_usb() for particular cameras.
I'll try to create a Canon Basic test script.
ROM:FF895504 set_control_event ; CODE XREF: sub_FF89570C
ROM:FF895504 ; sub_FF89576C:loc_FF8957C0
ROM:FF895504 AND R1, R0, #0xFF00 ; Sigfinder OK
ROM:FF895508 AND R12, R0, #0x40000000
ROM:FF89550C CMN R0, #1
ROM:FF895510 MOV R12, R12,LSR#30
ROM:FF895514 MOV R1, R1,LSR#8
ROM:FF895518 AND R2, R0, #0xFF
ROM:FF89551C MOV R3, R0,LSR#31
ROM:FF895520 BXEQ LR
ROM:FF895524 CMP R12, #0
ROM:FF895528 LDR R0, =0x3D460
ROM:FF89552C BNE loc_FF89554C
ROM:FF895530 MOV R12, #1
ROM:FF895534 CMP R3, #0
ROM:FF895538 LDR R3, [R0,R1,LSL#2]
ROM:FF89553C MOV R2, R12,LSL R2
ROM:FF895540 MVNEQ R2, R2
ROM:FF895544 ANDEQ R2, R3, R2
ROM:FF895548 ORRNE R2, R3, R2
ROM:FF89554C
ROM:FF89554C loc_FF89554C ; CODE XREF: set_control_event_sigfinder
ROM:FF89554C STR R2, [R0,R1,LSL#2]
ROM:FF895550 BX LR
ROM:FF895550 ; End of function set_control_event
I got a "line 934 _GUICtrlEdit_BeginUpdate function not found" error...Use a current version of the CHDK Shell (3.04).
I do understand the use of your version, but I'd probably either use (my) luar (function () body end)() (not tested) or make another command forI tested this a bit outside the camera (Ubuntu's Lua) but couldn't cook up a syntax Lua would accept (for defining and running a function inside the return command).
I got a "line 934 _GUICtrlEdit_BeginUpdate function not found" errorTry this:
@waterwingz
After a long night of tinkering around with switch_mode_usb() i figured out my set_control_event() address was wrong. Signature Finder (stubs_entry.S) address is correct.
Since reverting to sigfinder address "ptpcam --chdk" --> mode command does work.
However, using ptpcam.exe --chhd with the mode 1 command puts the camera into shooting mode but the mode 0 command does not put it back into playback mode. Is this normal ?I don't know, really, but this is how I switch between play and rec with ptp:
ptpcam --chdk="lua set_record(true)"
ptpcam --chdk="lua set_record(false)"
ROM:FF898B40 AC_PB2Rec ; CODE XREF: AC_PB2PC:loc_FF897B44
ROM:FF898B40 ; AC_EnryPB+D0 ...
ROM:FF898B40 STMFD SP!, {R4,LR} ; PB2Rec
ROM:FF898B44 LDR R0, =0x10A5
ROM:FF898B48 BL eventproc_export_IsControlEventActive
ROM:FF898B4C CMP R0, #0
ROM:FF898B50 LDMNEFD SP!, {R4,PC}
ROM:FF898B54 MOV R0, #0x60
ROM:FF898B58 ADR R1, aAcPb2rec ; "AC:PB2Rec"
ROM:FF898B5C BL LogPrintf ; LOCATION: CameraLog.c:237
ROM:FF898B60 MOV R0, #0xD
ROM:FF898B64 BL _sub_FF896F60__CameraConState.c__215 ; LOCATION: CameraConState.c:215
ROM:FF898B68 BL DispSwCon_MuteOnPhysicalScreen
ROM:FF898B6C MOV R0, #1
ROM:FF898B70 BL AC_ExitPB
ROM:FF898B74 LDMFD SP!, {R4,LR}
ROM:FF898B78 MOV R0, #0
ROM:FF898B7C B sub_FF83C1D8
ROM:FF898B7C ; End of function AC_PB2RecROM:FF897590 AC_Rec2PB ; CODE XREF: EnrySRec:loc_FF8978EC
ROM:FF897590 STMFD SP!, {R4,LR} ; Rec2PB
ROM:FF897594 ADR R1, aAcRec2pb ; "AC:Rec2PB"
ROM:FF897598 MOV R0, #0x60
ROM:FF89759C BL LogPrintf ; LOCATION: CameraLog.c:237
ROM:FF8975A0 LDR R1, =0x3580
ROM:FF8975A4 MOV R0, #0
ROM:FF8975A8 STR R0, [R1,#0x80]
ROM:FF8975AC BL sub_FF89BBFC
ROM:FF8975B0 MOV R0, #1
ROM:FF8975B4 BL sub_FF83C1D8
ROM:FF8975B8 BL sub_FF97A664
ROM:FF8975BC CMP R0, #0
ROM:FF8975C0 BEQ loc_FF8975D0
ROM:FF8975C4 BL sub_FF89073C
ROM:FF8975C8 CMP R0, #0
ROM:FF8975CC BLNE DispSwCon_MuteOnPhysicalScreen
ROM:FF8975D0
ROM:FF8975D0 loc_FF8975D0 ; CODE XREF: AC_Rec2PB+30
ROM:FF8975D0 BL ShutdownRecMode
ROM:FF8975D4 LDMFD SP!, {R4,LR}
ROM:FF8975D8 MOV R0, #0x10
ROM:FF8975DC B _sub_FF896F60__CameraConState.c__215 ; LOCATION: CameraConState.c:215
ROM:FF8975DC ; End of function AC_Rec2PB
I don't know, really, but this is how I switch between play and rec with ptp:So does mode command work for you too or only lua command?
ptpcam --chdk="lua set_record(true)"
ptpcam --chdk="lua set_record(false)"
@waterwingz
Did you check PB2Rec and Rec2PB address too?
Else maybe overwrite switch_mode_usb() with different control event id may help?
Still no luck with lua set_record(true) though.This is expected on dryos cameras. That's why switch_mode_usb exists.
So does mode command work for you too or only lua command?
waterwingz: the camera shouldn't be expected to retract the lens because it doesn't do that when you switch to play during normal use either...until a configurable delay.
void my_kbd_read_keys() {
......
if(keyb_stopusb == 400){keyb_stopusb = 0;}
if (keyb_stopusb == 1){
sprintf(osd_buf, "want switch");
draw_txt_string(28, 14,osd_buf, conf.osd_color);
physw_status[2] = physw_status[2] & ~(SD_READONLY_FLAG | USB_MASK);
}
if (keyb_stopusb == 50){ _set_control_event(0x902);_PB2Rec();
sprintf(osd_buf, "call _PB2Rec");
draw_txt_string(28, 14,osd_buf, conf.osd_color);}
if ((keyb_stopusb >= 1) && (keyb_stopusb <= 399)){physw_status[2] = physw_status[2] & ~(SD_READONLY_FLAG | USB_MASK);
}
if(keyb_stopusb)keyb_stopusb++;
I noticed what I think is a typing error around line 1260Yes, that's true. I've change in ptpcamGUI rev.#80. Thanks achillies!
' if fd~=nli and #fd>0 then' & @LF & _
I think should be ' if fd~=nil and #fd>0 then' & @LF & _
format: key 1st dimension \t value or values 2nd dimension sparated by \t and \n
values for 1st dimension: BOOL, INT, STRING (NIL is not enumerated)
values for 2nd dimension: NIL, BOOL, INT, STRING<conn> luar get_buildinfo()
platformid 12662
platform a590
version CHDK
platsub 101b
build_number 0.9.9-1077
os dryos
build_date Mar 4 2011
build_time 19:08:04
<conn> luar {1,2,3,4}
1 1
2 2
3 3
4 4
<conn> luar {X=44,G=2}
G 2
X 44
<conn> luar {"Monday","Tuesday","Wednesday","Thursday","Friday"}
1 Monday
2 Tuesday
3 Wednesday
4 Thursday
5 Friday
<conn> luar {{3,8,55},G=2}
1 3 8 55
G 2
<conn> luar {{3,8,55}, D={"S","M","D"},G=2,"value",678}
1 3 8 55
2 value
3 678
G 2
D S M D
<conn> luar {U={3,8,55}, D={"S","M","D"},G=2}
U 3 8 55
D S M D
G 2
<conn> luar {}
<conn> luar {{},{}}
<conn> luar {A={},{}}
<conn> luar {A={},B={}}
<conn> luar {A={1},B={33}}
A 1
B 33
<conn> luar {A={1,2},B={33}}
A 1 2
B 33
<conn> luar {A={1,2},B={33,{},44}}
A 1 2
B 33 44
<conn> luar {A={1,2,"true",false},B={33,{},44}}
A 1 2 true false
B 33 44
<conn> luar {A={1,2,true,"false"},B={33,{},44}}
A 1 2 true false
B 33 44
<conn> luar {3,4,nil,5,6,true,7}
1 3
2 4
4 5
5 6
6 true
7 7
<conn> luar {{3,4,nil,5,6,true,7}}
1 3 4 nil 5 6 true 7
<conn> luar require("capmode")
mode_to_name AUTO P TV AV M PORTRAIT NIGHT_SC
ENE LANDSCAPE VIDEO_STD VIDEO_SPEED VIDEO_COMPACT VIDEO_MY
_COLORS VIDEO_COLOR_ACCENT VIDEO_COLOR_SWAP STITCH MY_COLORS
SCN_UNDERWATER SCN_NIGHT_SNAPSHOT LONG_SHUTTER SCN_LANDSCAPE COLOR_SW
AP SCN_SNOW SCN_BEACH SCN_FIREWORK SCN_COLOR_ACCENT
SCN_COLOR_SWAP VIDEO_HIRES SCN_AQUARIUM COLOR_ACCENT SCN_NIGHT_SCENE
SCN_ISO_3200 SCN_SPORT SCN_KIDS_PETS INDOOR KIDS_PETS NIGHT_SN
APSHOT DIGITAL_MACRO SCN_FOLIAGE VIDEO_TIME_LAPSE SCN_INDOOR
SCN_PORTRAIT SUPER_MACRO VIDEO_PORTRAIT VIDEO_NIGHT VIDEO_INDOOR
VIDEO_FOLIAGE VIDEO_SNOW VIDEO_BEACH VIDEO_AQUARIUM VIDEO_SUPER_MACR
O VIDEO_STITCH VIDEO_MANUAL SPORTS QUICK SCN_SUNSET SCN_CREA
TIVE_EFFECT EASY SCN_DIGITAL_MACRO SCN_STITCH SCN_LONG_SHUTTER
LOWLIGHT SCN_NOSTALGIC SCN_SMART_SHUTTER SCN_LOWLIGHT
SCN_SUPER_VIVID SCN_POSTER_EFFECT SCN_FISHEYE SCN_MINIATURE SCN_HDRan other theme: "table results with ptpcam.exe"Good. This is important.
I provide a solution for return results from simple lua-tables with ptpcam. The most tables have one or two dimensions. My code convert tables to a formatted string for retrieve.
function t_to_s(t)
local v2s=function(v)
local t=type(v)
if t=='string' then
return v
end
if t=='number' or t=='boolean' or t=='nil' then
return tostring(v)
end
return ""
end
local r=""
for k,v in pairs(t) do
local s,vs=""
if type(v)=='table' then
for i=1,table.maxn(v) do
s=s..'\t'..v2s(v[i])
end
else
vs=v2s(v)
if #vs then
s=s..'\t'..vs
end
end
vs=v2s(k)
if #vs>0 and #s>0 then
r=r..vs..s..'\n'
end
end
return r
end
test (! is pc side lua in my client)___> !print(t_to_s({1,2,3,4}))
1 1
2 2
3 3
4 4
___> !print(t_to_s({X=44,G=2}))
G 2
X 44
___> !print(t_to_s({"Monday","Tuesday","Wednesday","Thursday","Friday"}))
1 Monday
2 Tuesday
3 Wednesday
4 Thursday
5 Friday
___> !print(t_to_s({{3,8,55},G=2}))
1 3 8 55
G 2
___> !print(t_to_s({{3,8,55}, D={"S","M","D"},G=2,"value",678}))
1 3 8 55
2 value
3 678
G 2
D S M D
___> !print(t_to_s({U={3,8,55}, D={"S","M","D"},G=2}))
U 3 8 55
D S M D
G 2
___> !print(t_to_s({}))
___> !print(t_to_s({{},{}}))
___> !print(t_to_s({A={},{}}))
___> !print(t_to_s({A={},B={}}))
___> !print(t_to_s({A={1},B={33}}))
A 1
B 33
___> !print(t_to_s({A={1,2},B={33}}))
A 1 2
B 33
___> !print(t_to_s({A={1,2},B={33,{},44}}))
A 1 2
B 33 44
___> !print(t_to_s({A={1,2,"true",false},B={33,{},44}}))
A 1 2 true false
B 33 44
___> !print(t_to_s({A={1,2,true,"false"},B={33,{},44}}))
A 1 2 true false
B 33 44
___> !print(t_to_s({3,4,nil,5,6,true,7}))
1 3
2 4
4 5
5 6
6 true
7 7
___> !print(t_to_s({{3,4,nil,5,6,true,7}}))
1 3 4 nil 5 6 true 7
<conn> lua return get_mode()
<conn> getm
false
false
513 (201)
<conn> lua foo()
<conn> getm
runtime error: :1: attempt to call global 'foo' (a nil value)
<conn> lua 'blah
syntax error: :1: unfinished string near '<eof>'
execution failed!
<conn> luar get_mode
unsupported data type: function
<conn> lua return usb_msg_table_to_string(os.stat('A/ptpmsg.lua'))
<conn> getm
dev 2
attrib 32
ctime 1299672574
atime 1299628800
blksize 512
blocks 6
mtime 1299585998
is_file true
mode 33279
is_dir false
size 2683
<conn> lua return usb_msg_table_to_string(os.stat('A/ptpmsgB.lua'))
<conn> getm
runtime error: :1: bad argument #1 to 'pairs' (table expected, got nil)a small change for all types -> attachment<conn> lua return usb_msg_table_to_string(os.stat('A/ptpmsgB.lua'))
<conn> getm
nil
A/ptpmsgB.lua: error
0 (0)<conn> lua return '1'
<conn> lua return '1\n2'
<conn> lua return '1\n2\n3'
<conn> lua return '1\n2\n3\n4'
<conn> getm
message from unexpected script id 22
1
message from unexpected script id 23
1
2
message from unexpected script id 24
1
2
3
1
2
3
4<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua return usb_msg_table_to_string(get_buildinfo())
<conn> getm
runtime error: :1: attempt to call global 'usb_msg_table_to_string' (a nil value
)
<conn> lua return usb_msg_table_to_string(get_buildinfo())
<conn> getm
platformid 12662
platform a590
version CHDK
platsub 101b
build_number 0.9.9-1083
os dryos
build_date Mar 9 2011
build_time 12:56:47
<conn> shutdown
< > r
<conn> lua return usb_msg_table_to_string(get_buildinfo())
<conn> getm
runtime error: :1: attempt to call global 'usb_msg_table_to_string' (a nil value
)
<conn> lua return usb_msg_table_to_string(get_buildinfo())
<conn> getm
platformid 12662
platform a590
version CHDK
platsub 101b
build_number 0.9.9-1083
os dryos
build_date Mar 9 2011
build_time 12:56:47a solution with a dummy-command (type 'lua'+space)<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua return 1
syntax error: :1: malformed number near '1 '
execution failed!
<conn> getm
<conn> lua return 1
syntax error: :1: malformed number near '1 '
execution failed!
<conn> getm
<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua return (1)
<conn> getm
1 (1)
<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua
<conn> lua return 1
<conn> getm
1 (1)some comments:In my code, this function is called automatically, and only if the argument is a table. See luascript.c lua_create_usb_msg
TEST 'function usb_msg_table_to_string(arg)':
The function fails is arg not a table.
TEST script id and results:This is one of the features that's mostly intended for the new client, I only put enough in ptpcam to test.
Very good idea, but how much results are from current id and how much from script-id 24?
For example we can use a text 'message' to separate old id from current id?Code: [Select]<conn> lua return '1'
<conn> lua return '1\n2'
<conn> lua return '1\n2\n3'
<conn> lua return '1\n2\n3\n4'
<conn> getm
message from unexpected script id 22
1
...
<conn> lua return '1'
script:22
<conn> lua return '1\n2'
script:23
<conn> lua write_usb_msg('hi')
script:24
<conn> lua return 2
script:25
<conn> getm
22 return string:"1"
23 return string:"1
2"
24 message string:"hi"
25 return number:1
Of course you should format it whatever way is easiest for you.TEST 'lua return'-commands after CHDK-start/rebootStrange, I don't see this on d10 in my build.
It looks like the luar-problem from CHDK-rev.1032 to 1033. My fast solution at that time 'return(...)' in ptpcam.exe. But I know that is not the real cause. Is it back again? Lua initialisation fails?Code: [Select]<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua return usb_msg_table_to_string(get_buildinfo())
<conn> getm
runtime error: :1: attempt to call global 'usb_msg_table_to_string' (a nil value)
To witch time is ptp-handler destroyed. I'm looking for a solution to switch the powersavemode for ptp-duration to 'always'.If you mean stop the camera from going to sleep when PTP is connected, I want this too :)
In my code, this function is called automatically, and only if the argument is a table. See luascript.c lua_create_usb_msgYes, I see. That's my fault. On first time use I get this error message. On this time I don't know, that the reason is on a other part and not the table.
Users shouldn't need to call it.
<conn> reboot
Could not find any device matching given bus/dev numbers, retrying in 1 s...
<conn> lua return get_buildinfo()
<conn> getm
attempt to call a nil value
<conn> lua return get_buildinfo()
<conn> getm
platformid 12662
platform a590
version CHDK
platsub 101b
build_number 0.9.9-1083
os dryos
build_date Mar 9 2011
build_time 12:56:47I tried to not change the output to much so I wouldn't break ptpcamgui. If you want to change the output, that's fine with me.The ptpcamGUI command routines need to rewrite for version 1.0 in any cases. It would be better to make all necessary changes for simply parse the messages.
We had here (http://forum.chdk-treff.de/viewtopic.php?f=3&t=2220) a test for this problem, but not a final solution. I know, you written "an older gcc3 is a way", but all build-server use gcc4.QuoteTEST 'lua return'-commands after CHDK-start/rebootStrange, I don't see this on d10 in my build.
We had here (http://forum.chdk-treff.de/viewtopic.php?f=3&t=2220) a test for this problem, but not a final solution. I know, you written "an older gcc3 is a way", but all build-server use gcc4.Oh, I thought the cause of this problem was found and fully fixed. I think I understand a little better now, fix stopped syntax error from crashing, but syntax error happens when it shouldn't.
<conn> lua return 1
script:1
<conn> lua return 'hi'
script:2
<conn> lua write_usb_msg('a message'); return true
script:3
<conn> lua return 'one',2,{a='a',b='b'},nil,false
script:4
<conn> getm
1:ret:1 (1)
2:ret:'hi'
3:msg:'a message'
3:ret:true
4:ret:'one'
4:ret:2 (2)
4:ret:'a a
b b
'
4:ret:nil
4:ret:false
<conn> lua 'a bad script
script:5
5:syntax error: :1: unfinished string near '<eof>'
execution failed!
<conn> lua bad()
script:6
<conn> getm
6:runtime error: :1: attempt to call global 'bad' (a nil value)
<conn>
<conn> lua return 1
script:1
<conn> getm
1:ret:1 (1)
gcc4<conn> lua return 1
script:1
1:syntax error: :1: malformed number near '1 '
execution failed!
<conn>
If I put a space after the 1, it works. (note error message above shows space, but there isn't one in my command!)<conn> lua n=1; return n
script:2
2:syntax error: :1: '<eof>' expected
execution failed!
<conn> lua n=1; return n
script:3
<conn>
Again, works with space after n, works on gcc3We have currently enough other options to delete files with CHDK.
return strangeness:Sorry reyalp, I have not enough time in this moment, so only a short comment.
Sorry reyalp, I have not enough time in this moment, so only a short comment.Thanks, this is very a good clue. I also don't have much time during the week, but this gives me something work on.
I tested many options for two month ago (rev.1032 to 1033) to find the reason for return-fail. An easy way for gcc4 was to put $(OPT_OBJS) in core/Makefile to begin of line.
rudi
I changed some lines and now that doesn't happen. And Again.. the stuff for deleting after download is in here too (because I use it all the time). Based on build 82.
If someone like me makes changes, how would you all prefer to see them?The best way would be to generate a patch/diff file. But this is not necessary. We can also compare the current version with other versions and merge, e.g. with KDiff3 (http://kdiff3.sourceforge.net/).
Attached are the Timelapse/Delay Changes.Sorry, there are no changes for time lapse in the attached file. The changes relate to the already built-in delete function.
A little more on this: Just moving $(OPT_OBJS) to not be the last item is sufficient. Trying the order of things within $(OPT_OBJS) seems like the next step, to narrow it down to the smallest change.Sorry reyalp, I have not enough time in this moment, so only a short comment.Thanks, this is very a good clue. I also don't have much time during the week, but this gives me something work on.
I tested many options for two month ago (rev.1032 to 1033) to find the reason for return-fail. An easy way for gcc4 was to put $(OPT_OBJS) in core/Makefile to begin of line.
rudi
unsigned char _ctype[] = {
...
int isdigit(int c) {
return _ctype[c]&_D;
}
note that c is an integer. If the value falls outside of the _ctype array, a random value in memory will be used. Solved:
The dryos ctype functions have a bugCode: [Select]unsigned char _ctype[] = {note that c is an integer. If the value falls outside of the _ctype array, a random value in memory will be used.
...
int isdigit(int c) {
return _ctype[c]&_D;
}
It is legal to pass EOF (generally int -1) to ctype functions, and lua does this.
FYI - this also fixes the problem where scripts crash with an 'unexpected symbol' error if the last line of the script does not have a 'newline' character at the end (on versions of CHDK built under GCC4 / Windows).Yes, turns out this wasn't really a PTP problem. Fixed in changeset 1092 (http://tools.assembla.com/chdk/changeset/1092)
FYI - this also fixes the problem where scripts crash with an 'unexpected symbol' error if the last line of the script does not have a 'newline' character at the end (on versions of CHDK built under GCC4 / Windows).Yes, turns out this wasn't really a PTP problem. Fixed in changeset 1092 (http://tools.assembla.com/chdk/changeset/1092)
You do very nice work. After updating all components (ptpcam.exe = r601, ptpcam.dll = 85, gui = 85), everything seems to work as expected.The changes of reyalp do not affect the current trunk version. These are preparations for the future ptp-interface.
These attachments are changes to the latest ptpcamgui (ver 85). The TDchange85.txt file contains the changes I make to my timelapse-timedelay, and the HLPDNLDChange85.txt file contains a minor change in the DCIM progress bar as well as changes to the Main Menu Help.I have integrated the minor change for the DCIM progress bar in revision 86 with some small cosmetic changes.
I've checked in the ptp message code in the trunk changeset 1101 (http://tools.assembla.com/chdk/changeset/1101)
This means that ptpcamgui and old ptpcam version will no longer work correctly. Non script functions should be ok.
- The most important features, upload and download, works quite well. You can download all picture files. The GUI can automatically download the current CHDK or CHDK-DE from the autobuild servers, extracts the files and sends it to the camera
msl
Whether this option is still existing, or am I missing something as we are constantly repeated message "No internet connection"?
Also, when I try to use CHDK-DE builds (sx130is compiled) with PTPCamGUI I can not take pictures. it seems to go into a loop with ">> luar get_mode() << false (Length: 5)" repeating. When I use the international builds (eg. 1089 patched with quids sx130is patch) everything works fine. What might be the cause? The CHDK-DE build seems to work well with this camera if I don't try to use PTPCamGUI (fresh version.. none of my new changes).
ptpCamGui:
The command luar post_levent_to_ui(4484) to unlock the keys in play mode is now a start option for all DryOS cameras. That means, for all DryOS camera the keys are directly after GUI start unlocked.
I hope the value 4484 is really valid for all DryOS cameras.
msl
I did notice that after using 'Download' with 'Delete files' option checked the camera was rebooted after deleting the files; but it did not do the luar post_levent_to_ui(4484) command so the keys were locked again.
I finally got it to recognize the camera, but I now get a continuous loop that I have let run for over 5 minutes before aborting.You'll have to make sure the directory A/CHDK/LUALIB is already created. The tool should probably do a "mkdir -p" before trying to upload. (I encountered this my self one or two days ago but didn't get around to posting it yet.)
actual live image dimensions (= bitmap dimensions?)There is no simple, general relation between bitmap and live dimensions. The CHDK code is currently very ugly in the way it deals with live view dimensions. The best thing to do would be to clean this up before trying to build a PTP interface for it, but this would be a big job.
palette (either as a whole or just CAM_BITMAP_PALETTE)The actual camera palette would be best, but requires find the relevant function for all cameras. Also, it changes at unpredictable times, so it wouldn't be clear when you'd need to update it. Doing it every frame would be wasteful, but as above maybe the camera could notice and only send it when requested or it changes.
I would also like to note that my experience with installing and figuring out how to draw an image with IUP has been quite frustrating. In the end I just piped the data to an external program. It definitely has resulted in my not being a fan of it. Just wanted to get that of my chest.IUP documentation isn't great. My understanding is that it doesn't really support drawing directly to a window. To do this you should either use CD http://www.tecgraf.puc-rio.br/cd/ (http://www.tecgraf.puc-rio.br/cd/) or the GL canvas, or roll your own function in C that draws directly the the hwnd or equivalent.
In include/platform.h some vid_* functions return long, some int. I don't think there is a reason, but I'd like to be sureNo point, long and int are identical. I would really like to get rid of all the longs from everything except possibly functions that are defined as 'long' in some external standard.
Proper serialisation in Lua looks nice, but also complicated/expensive. And it would indeed bump up the PTP version. Perhaps an option is to leave implicit serialisation as it is now and just provide a proper serialise function for Lua programmers to use if they want to communicate more complex things (as is the case in most languages I believe).My current thinking is to add an option to lua exec, which uses serialize for everything (except maybe strings, because escaping them does add quite a bit of overhead for not much value.) The serialize function really isn't very complicated (currently 100 lines in chdku), and the overhead for simple values should be negligible compared to ptp overhead. It doesn't handle cyclic tables or code, but there's not much need for those. Handling cyclic tables wouldn't be that hard if you needed to, and you can just override the serialize function with a smarter one.
The CHDK code is currently very ugly in the way it deals with live view dimensions. The best thing to do would be to clean this up before trying to build a PTP interface for it, but this would be a big job.And that's why it's not what we should do. ;)
Reading the post you referred to I feel the need to clarify that with "actual live image" I mean the "display area" or the image as it should be shown to the user. Perhaps presentation image/dimensions is a clearer term.Quoteactual live image dimensions (= bitmap dimensions?)There is no simple, general relation between bitmap and live dimensions.
I wonder if you'd be better off with a new ptp function that's something like "get video frames" which takes flags specifying which items you want (bitmap, viewport, bitmap palette if we can get it) and returns a structured result. The camera side could notice when aspect and possibly palette changed, and automatically send updated information. edit: batching them all together should be more efficient.Yeah, this is pretty much the "big send_details_and_buffers()" function I mentioned. Except I would not make it part of CHDK PTP on account of separation of concerns. (Although that means you'll have to either get the function pointer first or send the data as script result.)
Some discussion of viewport layout is in http://chdk.setepontos.com/index.php?topic=5415.0 (http://chdk.setepontos.com/index.php?topic=5415.0) and http://chdk.wikia.com/wiki/Frame_buffers (http://chdk.wikia.com/wiki/Frame_buffers)Thanks for that. I searched using the function names and didn't come up with anything like this.
The actual camera palette would be best, but requires find the relevant function for all cameras. Also, it changes at unpredictable times, so it wouldn't be clear when you'd need to update it.It changes? (Wouldn't expect that at all looking at the code.)
My current thinking is to add an option to lua exec, which uses serialize for everything (except maybe strings, because escaping them does add quite a bit of overhead for not much value.)That's even better.
It changes? (Wouldn't expect that at all looking at the code.)The CHDK palette doesn't change. The canon one does (switching between play and rec for example, or opening the menu.) The CHDK palette should be selected such that it is readable in either canon palette, but the exact colors vary, and it doesn't cover anywhere close to the entire canon palette.
unsigned int *vid_get_palette()
{
static unsigned pal[16];
unsigned int*syspal= (unsigned int*)0x634E0; // GetPaletteFromPhysicalScreen
int i;
for (i=0;i<16; i++) { // big-endian to little-endian
pal[i] = (syspal[i]>>24) |
((syspal[i]<<8) & 0x00FF0000) |
((syspal[i]>>8) & 0x0000FF00) |
(syspal[i]<<24);
}
return pal;
}
As for the remark about the mismatch of dimensions w.r.t. the S90 and G11: I currently have a DISKBOOT.BIN where viewport_width is 360 and a PS.FI2 where it is 720. No noticeable difference with edge overlay. Very robust code I guess...
... so "360x240" in lib.c is actually 720 in Y. 360x240 gives square pixels.Just to make sure I absolutely understand correctly: the value vid_get_viewport_width() returns is not the actual width of the image in the buffer (or the number of Y's, if you want)?
Hmm... It seems that, just as in life, the more you know, the more it sucks.Right, on the classic cameras where the viewport is "360x240" there are actually 720 y values. Histogram, zebra, MD etc, just ignore half of them. This made things easy on these cameras, because the bitmap was actually 360x240.... so "360x240" in lib.c is actually 720 in Y. 360x240 gives square pixels.Just to make sure I absolutely understand correctly: the value vid_get_viewport_width() returns is not the actual width of the image in the buffer (or the number of Y's, if you want)?
One thing I keep wondering about is what happens with these changing viewport dimensions. The bitmap dimensions seem to be constant, right?I'm nut sure what the bitmap does on cameras with widescreen modes... I don't own one ;)
I think it is key to first figure out what information we precisely need, without looking at what is available (in CHDK).Agreed. Looking at ewars CHDKCAM and corresponding CHDK patches may also be useful. He addressed a lot of these issues (by adding new functions) but the cameras permutations have also become more complicated.
So that seems to suggest that the presentation(/display area) dimensions are constant as well. If so, and if the unused pixels in the viewport buffer are set to black by the camera (which I'm hoping),Worth experimenting, but I wouldn't count on it. My guess is the canon video hardware is doing the overlay stuff much later.
Looking at ewars CHDKCAM and corresponding CHDK patches may also be useful. He addressed a lot of these issues (by adding new functions) ...Yeah, but until our discussion I had no idea what these "_for_ptp()" functions were about. Looking at it now it has become a lot clearer.
I have test Ix 1000 HS with 1154 build and chdkptp.You need to double click to expand the tree. This automatically re-freshes. When I said the file browser was very alpha, I wasn't joking ;)
I guess when i click on files tab and i call refresh, the directory should show.But camera show only in log short some "finished ***" messages and nothing happen
reboot button work, but it do not start chdk when sd card is not write protect.What exactly did you expect to happen ?
shutdown button do not shutdown.there is only "finished ***" on cam log show.when i remove USB cable, i am not able to switch off camera.i need remove battery.Sounds like shutdown script function is broken in your port.
same happen when i do after a connect and upload file or press play.camera is not dead, show finish on every action, but it can not switch off with the buttons.when remove USB cable canon keys work all,but i am not able to switch to record mode, even if no USB cable is in.Sounds like: http://chdk.setepontos.com/index.php?topic=4338.msg64324#msg64324 (http://chdk.setepontos.com/index.php?topic=4338.msg64324#msg64324)
do a disconnect before remove of USB cable too not help to switch camera off.
the switch to record mode crash this camera as before.lens move out and after that, camera crash.seem this camera is not able to switch to record modeSounds like your ports implementation of mode switching is broken.
1)Thread where there is the development of G11 USB/PTP aside form "G11 porting"?I don't think there is one. All I can find is that ERR99 gave it a shot (http://chdk.setepontos.com/index.php?topic=4647.msg50425#msg50425), but had problems with his setup.
2) Entry point in G11 (Start dumping -> loading USB/PTP scripts into CHDKIt seems that the required functions are already found* for the G11. Defining CAM_CHDK_PTP in platform/g11/platform_camera.h could be all that is needed to get CHDK PTP to work on the G11. At least for the basic functionality. Try that and see how it goes.
It seems that the required functions are already found* for the G11. Defining CAM_CHDK_PTP in platform/g11/platform_camera.h could be all that is needed to get CHDK PTP to work on the G11. At least for the basic functionality. Try that and see how it goes.
USB/PTP
1) Remote Capture
-capture image and download
-capture-video
2) Remote Zoom
Note that CAM_CHDK_PTP is already defined by default for all cams (in /include/camera.h) since r1116Ah, right. Thanks. I guess I still have to get used to the new situation. ;)
At GitHub (https://github.com/mweerden/CHDKPTPRemote/tree/remote-preview) you can now find my patch for remote live preview as well as a CHDK PTP .NET library (+test app)....
....I've made a separate branch for the remote live preview stuff. The master branch (https://github.com/mweerden/CHDKPTPRemote/) should be compatible with the current CHDK trunk (i.e. CHDK PTP v2.0).
gphoto2/libgphoto2 are developing nicely and now include some chdk functions, I believe.Hmm.. interesting. They seem to have added something, but it is based on ewavr's patch. So I'm not sure it works with the current trunk.
It appears your code is made for Windoze. How do I get it going under linux? I downloaded the latest G9 chdk build but I guess from your post that I need to compile a special G9 build to enable remote live preview. Thank you and well done!Yeah, you'll need to apply the chdk.patch to the CHDK trunk and build your own binaries. You might need to (re)implement some of the new functions to return the correct sizes for your camera. If you want to get the overlay bitmap you'll certainly have to implement the vid_get_bitmap_active_palette() function.
USB/PTP
1) Remote Capture
-capture image and download
-capture-video
2) Remote Zoom
The G11 should work with the ptp interface.
You can try this (http://chdk.setepontos.com/index.php?topic=6285.msg64905#msg64905) or this (http://chdk.setepontos.com/index.php?topic=6231.0) ptp interface project. Do not forget to install the alternative USB driver.
msl
USB/PTP
1) Remote Capture
-capture image and download
-capture-video
2) Remote Zoom
The G11 should work with the ptp interface.
You can try this (http://chdk.setepontos.com/index.php?topic=6285.msg64905#msg64905) or this (http://chdk.setepontos.com/index.php?topic=6231.0) ptp interface project. Do not forget to install the alternative USB driver.
msl
Using set_record(1) did not help to enable the buttons. But luar post_levent_to_ui(4484) does, it is just that I didn't think that it should. But still cannot get the other functions (such as timelapse and change mode) to do anything except loop forever on the luar get_mode() command.Does set_record(1) switch it into record mode ?
Using set_record(1) did not help to enable the buttons. But luar post_levent_to_ui(4484) does, it is just that I didn't think that it should. But still cannot get the other functions (such as timelapse and change mode) to do anything except loop forever on the luar get_mode() command.Does set_record(1) switch it into record mode ?
If it does, what does
=return get_mode()
do in chdkptp after calling set_record(1)
In vxworks, switch_mode_usb and set_record do the same thing (switch mode with levent) but maybe this camera needs the dryos style switch_mode since it's right on the transition...
Unfortunately, I do not know how to tell if it has switched to record mode.The lens will extend and the display will switch to live view.
> = return get_mode()Looks like it's not switching.
9:return:false
9:return:false
9:return:513
Never does.Unfortunately, I do not know how to tell if it has switched to record mode.The lens will extend and the display will switch to live view.
OK, so mode switch does not work on this camera. Thanks for tracking that down. I would guess it need PB2Rec and REC2Pb like dryos cameras.Never does.Unfortunately, I do not know how to tell if it has switched to record mode.The lens will extend and the display will switch to live view.
OK, so mode switch does not work on this camera. Thanks for tracking that down. I would guess it need PB2Rec and REC2Pb like dryos cameras.
@nstatic
It's up to you to understand the use off diff/patch. This is not a CHDK problem or a problem with mweerdens patch.
You can look at the .rej files to see what failed. You can probably find some tutorials on working with patches on the internet somewhere. You may also find some gui tools that let you interactively resolve the problems, but understanding the code will still be required to do this successfully. If the original patch was generated with svn, tortoisesvn apply patch is very good, assuming the patch came from the same svn tree.
Note that you should almost certainly not answer yes to the "reversed" question.
The patch should apply cleanly if you start with whatever revision it was made against, but assuming that was from the CHDK tree rather than the CHDKDE tree, you won't have the sx120 so that won't solve your problem.
As you may have realized by now, this is not something you should expect to get working without some hacking...
......
i am not against putting in some ( or considerable ) effort to make it work :)
i will do as you say and see what failed
where do i go from here - how do i try getting a preview. i have your .net test project compiled but it doesnt give me a preview (it works excellently otherwise !), and also the get overlay and get image buttons are disabled. ill look in to the c# code next but a little help on the way would be welcomeSounds to me like you are using the master branch instead of the remote-preview one. In the first the buttons are indeed disabled (as that one is intended for unpatched CHDK) and in the other they are always enabled. The latter also needs the version to be 0x10002 (unless you change it as well in CHDKPTPUtil.cs).
thanks
oh by the way while patching i numbered the ptp to version 2.1 instead of 0x10002.0 - hope that is not the problem.
where do i go from here - how do i try getting a preview. i have your .net test project compiled but it doesnt give me a preview (it works excellently otherwise !), and also the get overlay and get image buttons are disabled. ill look in to the c# code next but a little help on the way would be welcomeSounds to me like you are using the master branch instead of the remote-preview one. In the first the buttons are indeed disabled (as that one is intended for unpatched CHDK) and in the other they are always enabled. The latter also needs the version to be 0x10002 (unless you change it as well in CHDKPTPUtil.cs).
thanks
oh by the way while patching i numbered the ptp to version 2.1 instead of 0x10002.0 - hope that is not the problem.
however 'Get Overlay' does nothing. should i be getting a live moving preview ??My test program only gets one image when you press the button. If you want something more dynamic you'll have to write the code to repeatedly get the image. Or just press the button rapidly. ;)
Quotehowever 'Get Overlay' does nothing. should i be getting a live moving preview ??My test program only gets one image when you press the button. If you want something more dynamic you'll have to write the code to repeatedly get the image. Or just press the button rapidly. ;)
As for the overlay, it does get it, but per default it uses an "empty" palette (i.e. every colour is fully transparent). To change that you'll have to patch the CHDK code to send the correct palette for your camera. For the ixus870 you can see how it is done in the patch. This does mean figuring out the right address from the firmware, though. Or you can hack the code to use some made up palette.
ok then ill probably put it on a timer - do you think it will be able to give me around 7-10 fps (rapidly pressing the button gives me a feeling it may be possible)When I started working on this I tried it out by just dumping the data to files. This got me about 100 images in 5 to 6 seconds, if I recall correctly. The code on the camera is a bit different now, but I don't think it should matter much.
ok then ill probably put it on a timer - do you think it will be able to give me around 7-10 fps (rapidly pressing the button gives me a feeling it may be possible)When I started working on this I tried it out by just dumping the data to files. This got me about 100 images in 5 to 6 seconds, if I recall correctly. The code on the camera is a bit different now, but I don't think it should matter much.
Also, the C# code I wrote to convert the image to an RGB .NET Image might not be the fastest method to get it on the screen. So I would guess 10 fps should definitely be possible (assuming your camera isn't much slower than mine for some reason).
just putting a timer and calling the getimagebutton_click() gave me about ( 4 fps ). im just doing a little dirty testing for the moment.That's very low. Perhaps you compiled the program for debugging? If found that setting it to "release" and running it outside of VS made quite a difference.
would using overlaybutton_Click() be much different and much faster ? (ill be paching for the pallete for SX 120 next)The overlay is smaller (by a factor 3 in my camera). Processing it should also easier/faster. There might be room for improvement by getting and/or processing the palette less often, but this might have a minimal effect.
i have indeed compiled for debugging - will try and compile it for release and see. although the gut feeling says the bottleneck is on the camera side. i am almost cretainly wrong as you have recorderded close to 17fps. whatever may be the case, that will be my next step ...just putting a timer and calling the getimagebutton_click() gave me about ( 4 fps ). im just doing a little dirty testing for the moment.That's very low. Perhaps you compiled the program for debugging? If found that setting it to "release" and running it outside of VS made quite a difference.
once i convert it to VB and can start thinking i may be able to optimize it a little and use the overlay image if necessaryQuotewould using overlaybutton_Click() be much different and much faster ? (ill be paching for the pallete for SX 120 next)The overlay is smaller (by a factor 3 in my camera). Processing it should also easier/faster. There might be room for improvement by getting and/or processing the palette less often, but this might have a minimal effect.
i have indeed compiled for debugging - will try and compile it for release and see. although the gut feeling says the bottleneck is on the camera side.You can try disabling the image processing (and drawing) code. That is, make it just receive the image data from the camera but no do anything with it. This won't necessarily give you the maximum output of the camera, but it might be a lot closer.
i have indeed compiled for debugging - will try and compile it for release and see. although the gut feeling says the bottleneck is on the camera side.You can try disabling the image processing (and drawing) code. That is, make it just receive the image data from the camera but no do anything with it. This won't necessarily give you the maximum output of the camera, but it might be a lot closer.
i always verify that the USB and the Cable is V 2.0 compliant. this was very much the issue while using the camera (older versions SX 110, SX 100, S5, S3 etc) with the CANON SDK which was very picky about V2.0.
Perhaps it could also be you are not using a USB 2.0 port?
just a thought - can a loop be incorporated in to the CHDK itself which will post the image at regular intervals which the receiving program collects - maybe on another thread ? so instead of requesting one instance of the image the command from the computer turns on the feed which the camera posts and another command which can turn it off - maybe suspending the loop while other commands are received if necessaryIt's possible, I'm pretty sure. Not so sure about why, though. It seems to me that it is much easier and much more flexible to pull from the camera when needed.
i would still like to try it once this porting thing s complete. well the problem was the difference in how bit shifting works in c# and vb i think.just a thought - can a loop be incorporated in to the CHDK itself which will post the image at regular intervals which the receiving program collects - maybe on another thread ? so instead of requesting one instance of the image the command from the computer turns on the feed which the camera posts and another command which can turn it off - maybe suspending the loop while other commands are received if necessaryIt's possible, I'm pretty sure. Not so sure about why, though. It seems to me that it is much easier and much more flexible to pull from the camera when needed.
@mweerdenDepends on what your camera provides and how much work you are willing to do.
is it possible by any chance to get a bigger preview image. i am getting a 360x240. any possibility, it can be around something like 640x480 or so :)
how do i go about knowing what my camera can provide. its a SX120 and in future i am planning to be working on a SX 130 ( it may be possible to get the SX 200 series ). 720x480 is still a 4 fold improvement and i can live with it for the moment :) if it is possible on the SX 120 or the 200 series.@mweerdenDepends on what your camera provides and how much work you are willing to do.
is it possible by any chance to get a bigger preview image. i am getting a 360x240. any possibility, it can be around something like 640x480 or so :)
For my ixus870 the actual size of the live image is 720x240 (so it is stretched) and I know the G12 has one that is 720x480. In my test app I just always scale the image (and overlay) to 360x240.
I'm pretty sure this format is fixed (i.e. per camera). The only way I see to get more is to somehow access the real raw data provided by the sensor. I have no idea about the possibilities of this. I do know that it means transferring more data and thus likely a (much) lower fps. Only solution to that would be compression before transfer, but that would mean figuring out how to use the camera's dedicated hardware for this or somehow letting the camera just take pictures/movies like normal and directly stream it to your PC. As far as I know the knowledge for these kinds of things is currently not available.
Depends on what your camera provides and how much work you are willing to do.there no problem with me putting in the effort. only problem is my limited exposure to this kind of source. still can you guide me what may be needed to do so that i can evaluate if it is in realm of my current capabilities ;).
... and I know the G12 has one that is 720x480.
Reasonably sure, but I admit I haven't checked it myself. My remote-preview code has been tested by someone with a G12. The code uses its own functions for widths and heights that default to 720x240 for the viewport. This gave only the upper half of the viewport. Changing the height to 480 gave everything. So unless I'm missing something crucial, the viewport should be 720x480 on the G12.... and I know the G12 has one that is 720x480.
Are you sure?
The G12 has the same resolution live view buffer as all the other recent cameras - 720x240.
Are you sure?
The G12 has the same resolution live view buffer as all the other recent cameras - 720x240.
Phil.
FF371D70 loc_FF371D70 ; CODE XREF: sub_FF371C18+13Cj
FF371D70 CMP R2, R0
FF371D74 BHI loc_FF371D58
FF371D78 STR R5, [R8,#0xC]
FF371D7C MOV R2, R6
FF371D80 MOV R1, R5
FF371D84 ADR R0, aMaxyLdMinyLd ; "MaxY %ld MinY %ld\r"
FF371D88 STR R6, [R8,#0x10]
FF371D8C BL sub_FF15334C
FF371D90 LDR R0, [R8,#8]
FF371D94 STR R0, [R10]
FF371D98 LDR R0, [R8,#4]
FF371D9C LDR R1, [R8]
FF371DA0 MOV R2, R0,LSL#2
FF371DA4 LDR R0, [R9]
FF371DA8 BL sub_FF060874
FF371DAC LDR R2, =0xA8C00 //960x720
FF371DB0 LDR R1, =0x405D7980
FF371DB4 LDR R0, [R9,#4]
FF371DB8 BL sub_FF060874
FF371DBC MOV R0, R4
FF371DC0 BL j_free
FF371DC4 LDR R0, [R8]
FF371DC8 LDMFD SP!, {R4-R12,PC}
how do i go about knowing what my camera can provide.Looking it up online, in the CHDK code or in the firmware. And by experimentation.
there no problem with me putting in the effort. only problem is my limited exposure to this kind of source. still can you guide me what may be needed to do so that i can evaluate if it is in realm of my current capabilities ;).One thing you probably need is functions to send commands to the host instead of the other way around. This means disassembling the firmware and trying to find such functions (or functions that allow you to build it yourself). Unless you have quite some experience reverse engineering like this, I don't expect it is something that you can pick up easily.
pls correct me if i am wrong or if the whole idea is ridiculousI wouldn't go as far as calling it ridiculous, but I really don't see the point. It's much easier to regularly pull from the PC and I can't see any advantage in pushing from the camera.
how do i go about knowing what my camera can provide.Looking it up online, in the CHDK code or in the firmware. And by experimentation.
Regarding the resolution of the viewport buffer, if you didn't change the patch related to this and you're getting the full image, then it's 720x240. And, as said before, this is almost certainly fixed.Quotethere no problem with me putting in the effort. only problem is my limited exposure to this kind of source. still can you guide me what may be needed to do so that i can evaluate if it is in realm of my current capabilities ;).One thing you probably need is functions to send commands to the host instead of the other way around. This means disassembling the firmware and trying to find such functions (or functions that allow you to build it yourself). Unless you have quite some experience reverse engineering like this, I don't expect it is something that you can pick up easily.
As for timing, I guess you could start a separate task and just call msleep. It probably isn't very accurate or reliable though. I'm sure there are some other timing facilities elsewhere; I expect the firmware to need it as well.pls correct me if i am wrong or if the whole idea is ridiculousI wouldn't go as far as calling it ridiculous, but I really don't see the point. It's much easier to regularly pull from the PC and I can't see any advantage in pushing from the camera.
it is only that i was using the previous version - SX 110 and previous - with the canon sdk and it provided a fluent preview.Which also suggest the camera shouldn't be the bottleneck (although the firmware might have used compression; which might be interesting to look into). In any case, reversing the roles should have no (significant) effect on the speed.
so i would try getting the overlay instead of the bitmap which should be around 3x smaller (like you mentioned). any pointers as to how i should locate the location for the palette in my camera SX120 ?The overlay is the bitmap and note that it does not contain the live image. I found the palette for my camera in the firmware by comparing it with where ewavr found it for other cameras, if I remember correctly. So you'll need two firmware disassemblies, one where you know where it is and the other of your own camera.
I would bet that the canon firmware jpeg'd the live view before sending it, using something similar to whatever does the mjpeg for video.it is only that i was using the previous version - SX 110 and previous - with the canon sdk and it provided a fluent preview.Which also suggest the camera shouldn't be the bottleneck (although the firmware might have used compression; which might be interesting to look into)
* ptp_canon_getviewfinderimage:
* This operation can be used to read the image which is currently
* in the camera's viewfinder. The image size is 320x240, format is JPEG.
* Of course, prior to calling this operation, one must turn the viewfinder
* on with the CANON_ViewfinderOn command.
* Invoking this operation many times, one can get live video from the camera!
It's not clear what generation of Canon camera this refers to.Canon most certainly compresses (used to compress - for thepowershots) the live feed as there were heavy compression artifacts in the image. and both your and mweerden's explanation makes me realize that it is indeed true that camera with a USB 2.0 should not be a bottleneck.I would bet that the canon firmware jpeg'd the live view before sending it, using something similar to whatever does the mjpeg for video.it is only that i was using the previous version - SX 110 and previous - with the canon sdk and it provided a fluent preview.Which also suggest the camera shouldn't be the bottleneck (although the firmware might have used compression; which might be interesting to look into)
In fact, there are references to this in canon (non-chdk) code in ptpcam (ptp.c)Code: [Select]* ptp_canon_getviewfinderimage:It's not clear what generation of Canon camera this refers to.
* This operation can be used to read the image which is currently
* in the camera's viewfinder. The image size is 320x240, format is JPEG.
* Of course, prior to calling this operation, one must turn the viewfinder
* on with the CANON_ViewfinderOn command.
* Invoking this operation many times, one can get live video from the camera!
This also shows that the Canon SDK probably used polling just mweerdens code. This isn't a surprise, AFAIK USB up to 2.0 doesn't really have device initiated communications, at the lowest level everything is initiated by the host, even though the software stack might expose it as an event. I could be wrong here, my understanding of USB is fairly limited...
The overlay is the bitmap and note that it does not contain the live image. I found the palette for my camera in the firmware by comparing it with where ewavr found it for other cameras, if I remember correctly. So you'll need two firmware disassemblies, one where you know where it is and the other of your own camera.i dont think i totally understand mweerden - "does not contain the live image" part. what does it contain. i was under the impression that get_bitmap and bet_overlay were .....
i dont think i totally understand mweerden - "does not contain the live image" part.You already know what the live image (a.k.a. viewport) is: the image produced by the camera sensor (or at least a derivative of it).
I have written a ColorTab-Viewer (http://chdk.bplaced.net/extra/ColorTab.zip).
can you upload this somewhere else ?
Did anyone ever manage to compile or adapt EWAVR's CHDKCAM code ?Does this (http://chdk.setepontos.com/index.php?topic=4338.msg66042#msg66042) qualify?
- rudi has now implemented a very nice multi camera feature. You can use more than one camera with the GUI. Every Camera get a unique identification number.
Is that discussed in the German forum ?
Yes, a little bit.
hi phil,
any progress on your changes to the live preview or adding it to the trunk. ?
not that everything is not working as it should, just as it is, just hoping to get a little more juice out of it. at the moment im getting around 7fps. i know it is the code at my client side which is to be blamed but was hoping to optimize it only after the thing got incorporated in the trunk so that i do not have to worry about patching every time the trunk changes, :)
Had a busy week last week, then went whale watching over the weekend - actually took some photos for a change :)
Quote
Had a busy week last week, then went whale watching over the weekend - actually took some photos for a change :)
hope u had a good time :)
your old binary posted on the 130 forum does not recognize the cam - probably due to the PTP vers 2.1-You can also change the CHDK code (see core/ptp.h) to match the version, but that won't help here as the code is incompatible, I'm sure.
by the way is there a better way to patch - i am using the DE version with CHDK Shell and the patch always failsThis depends on the problem. If CHDK-DE doesn't differ too much in the changed files, patching shouldn't be much of a problem. There might be an issue with line endings, though, which seem to be able to confuse the patch tool.
ive since stopped trying to apply the patch and just make the changes in an editor
hi phil,
managed to patch the DE version that i had and it compiled ok after a bit of tweaking
however could not compile your .net app as it is in a newer version (vs 2008) i could try adding the files manually and tweaking the few incompatible function but if you could just post the binary for the moment i could verify that the thing works ok
your old binary posted on the 130 forum does not recognize the cam - probably due to the PTP vers 2.1-
by the way....
excellent pics !!
I see philmoz added live view support in trunk changeset 1316 (http://tools.assembla.com/chdk/changeset/1316/trunk)
This is useful, commonly requested functionality. It has sat on the back burner for far to long, and that's mostly my fault. I appreciate the effort both mweerden and philmoz have put into it.
That said, the implementation leaves me shaking my head.
Why is there PTP code in luascript.c (so lua can return a pointer to a static I guess but ...) ?
Why are we overloading PTP_CHDK_CallFunction with new magic behavior ?
Why are we passing a single hard coded address of a function from Lua (in a struct stuffed into a binary string), to the PTP client just so the PTP client can just pass it back for CHDK to call it ? How is this preferable than just making another CHDK PTP opcode that calls the function ? We have 2^32 available...
Why are we involving Lua at all when we are just passing back a hard-coded binary blob ? Sure, some of these values might be useful for other things, but this is not a very usable format. If that blob ever changes, it will be an incompatible protocol revision.
Why are there magic numbers all over the place ?
Guess this is what I get for not looking at it sooner... :(
I understand the desire to get something working, but we've already reached the point where CHDK is nearly unmaintainable due to people piling on half baked hacks. We really need a development branch or something where we can refine larger features like this without worrying too much about compatibility in the intermediate revisions.
Why is there PTP code in luascript.c (so lua can return a pointer to a static I guess but ...) ?As said, I don't see it as PTP code. However, there is little consequence (if any) if you wish to move it to a different location.
Why are we overloading PTP_CHDK_CallFunction with new magic behavior ?Why is it magic? I believe it's properly defined and on purpose in a way that is non-specific to the remote live view functionality.
Why are we passing a single hard coded address of a function from Lua (in a struct stuffed into a binary string), to the PTP client just so the PTP client can just pass it back for CHDK to call it ? How is this preferable than just making another CHDK PTP opcode that calls the function ? We have 2^32 available...The core question, as discussed above.
Why are we involving Lua at all when we are just passing back a hard-coded binary blob ? Sure, some of these values might be useful for other things, but this is not a very usable format. If that blob ever changes, it will be an incompatible protocol revision.Apart from what I've said so far, because Lua is (at this moment) the only way we can call a named function.
Why are there magic numbers all over the place ?I believe "all over the place" isn't entirely fair. In any case, it's not so much criticism on the new features as something that I'm sure can be easily taken care of if asked.
I understand the desire to get something working, but we've already reached the point where CHDK is nearly unmaintainable due to people piling on half baked hacks. We really need a development branch or something where we can refine larger features like this without worrying too much about compatibility in the intermediate revisions.And, as happens often, we come to the bigger discussion. I believe I've spoken my mind about it before, so I'll try to restrain myself. Fact is that things are the way they are and they don't seem likely to change significantly any time soon. "Don't forget what the H stands for." comes to mind.
I can't comment on the PTP_CHDK_CallFunction, that was there when I started looking at it (I assumed this was something already worked out).Fair enough. This feels like the wrong way to do it to me. I can sort of see the argument that you might want other functions that produce a similar stream of data, but passing them around seems suspect to me. Since these functions would have to be defined in CHDK code anyway, it's not as if clients are going to make these up on the fly.
Putting the live view code in Lua makes sense to me (mweerden convinced me of this).I'm not convinced of this. It doesn't make sense to handle the live view data (framebuffer/palette etc) in Lua, so passing this one chunk a different way seems needless complexity. You already need a non-lua part on both ends to handle the data, and you are already sending *some* of the information about framebuffer layout with that. If you could sanely do the whole thing in Lua, I'd be all for that.
Almost everything else done in regards to controlling the camera via PTP is done via Lua code - it made sense to do this the same way rather than put the live view as a special PTP function.
I agree changes to the data structures could cause problems if not done carefully; but, short of using XML, I don't know how else you could do this.If you are going to use Lua, put it in a Lua table. That gives you a nice key/value representation, without the overhead of XML.
I tried sending back the viewport/bitmap/palette buffer addresses and then using the PTP 'get memory' call to retrieve the data - but it was a lot slower for some reason. Plus there's more chance the buffer address will change between the time the client gets it and the time it uses it to get the data.This all makes sense. I agree with packing them together like this. In fact, I would like to take this one step further, which I'll get to in another post.
None of this is set in stone, there are no real clients using it - the goal was to get something that was being used and tested to a point where active development on clients could proceed without having to constantly patch the source.The problem is the trunk is the build used by end users, so the "not-set-in-stone" bit has the potential to come back and bite you. But I agree, we need something to work from without constantly shuffling patches. My gut reaction aside, it needed to be done :)
This could have been in a different branch; but then you have to deal with the metadata mess in SVN every time you want to merge.I've removed the spurious mergeinfos (generally you aren't supposed to mess with it but these were introduced by mistakes in the first place...)
I'd actually consider this to be a reasonably small change, it's localised and (apart from the PTP change) doesn't affect anything else. The amount of extra code is also pretty small.Agree this doesn't directly impact anything else. My reaction is to the (IMHO) ugliness of the code.
Not sure what magic numbers you are referring toThe numbers for the different outputs of handle_video_transfer etc. Understand this is work in progress. Not a big deal, but they should appear as constants in ptp.h along with comments. I'm trying to keep that as the canonical definition of our protocol.
It seems your main point is about adding functionality to Lua vs. adding it to CHDK PTP. At least, most of the big things you question seem to be a consequence of this choice.Agreed.
I think the rationale has been given in the past, but it comes down to me having preferred a minimal CHDK PTP. And a large argument for that is that adding more to it makes incompatible protocol revisions more likely. The new functionality is not a part of CHDK PTP, just something that is used through it.That seems like a marginal distinction. It is effectively part of the CHDK PTP protocol. To actually get the live view data, any client has to accept a data phase from PTP_CHDK_CallFunction, and then understand the results. Realistically, this is one of the main features people will want PTP for, so it makes sense to have the method of doing that be a first class part of the protocol.
To me that is (almost) like saying HTTP is part of TCP.QuoteI think the rationale has been given in the past, but it comes down to me having preferred a minimal CHDK PTP. And a large argument for that is that adding more to it makes incompatible protocol revisions more likely. The new functionality is not a part of CHDK PTP, just something that is used through it.That seems like a marginal distinction. It is effectively part of the CHDK PTP protocol.
I can understand the concern over overloading the PTP_CHDK_CallFunction function parameters / return so let me propose a slight variation.Perhaps I'm missing your point, but I believe your variation still needs the extended PTP_CHDK_CallFunction. I also don't really see what the concern regarding this extension is.
- Add a major / minor version number to the live view data structure.Adding a version is probably a good idea. Instead of adding it to structure you could also make it part of the function name. This allows multiple versions to coexist. Or perhaps a combination where backwards-compatible versions are in the structure, and other revisions use a new function. Not yet sure how much sense this makes, though. ;)
I can understand the concern over overloading the PTP_CHDK_CallFunction function parameters / return so let me propose a slight variation.Perhaps I'm missing your point, but I believe your variation still needs the extended PTP_CHDK_CallFunction.
Quote- Add a major / minor version number to the live view data structure.Adding a version is probably a good idea. Instead of adding it to structure you could also make it part of the function name. This allows multiple versions to coexist. Or perhaps a combination where backwards-compatible versions are in the structure, and other revisions use a new function. Not yet sure how much sense this makes, though. ;)
Not sure how manageable that would be. For example if an updated client connects to an older version of CHDK then it first has to check if the function name exists for the latest version. If it doesn't then it has to check the the next oldest one, and so on until it finds a function (or give up and fail). Would make it hard to to support older versions of CHDK in the client code.That's quite inevitable (but optional) when supporting multiple versions, I think. So the question is whether or not such support is desirable at all or whether it is all too hypothetical anyway.
Similar problem going the other way, if an old client connects to a newer version of CHDK the function won't exist (unless you keep all the function names in CHDK which could get messy) - so how would the client tell if it's a newer version or just not there at all.Is such a distinction relevant? It's not like knowing there is a newer version somehow allows it to do more (except perhaps tell the user).
Not sure how manageable that would be. For example if an updated client connects to an older version of CHDK then it first has to check if the function name exists for the latest version. If it doesn't then it has to check the the next oldest one, and so on until it finds a function (or give up and fail). Would make it hard to to support older versions of CHDK in the client code.That's quite inevitable (but optional) when supporting multiple versions, I think. So the question is whether or not such support is desirable at all or whether it is all too hypothetical anyway.
QuoteSimilar problem going the other way, if an old client connects to a newer version of CHDK the function won't exist (unless you keep all the function names in CHDK which could get messy) - so how would the client tell if it's a newer version or just not there at all.Is such a distinction relevant? It's not like knowing there is a newer version somehow allows it to do more (except perhaps tell the user).
But making it harder to support older versions won't help encourage people to do it :)Not sure I see how it's harder than other obvious options.
Giving the end user accurate information is important - a generic 'somethings wrong' message is not good. Being able to tell the user that 'you don't have the live view extensions in CHDK', or 'you have and older/newer version you need to update your CHDK/client program' is preferable.In principle I agree. However, I think "no or incompatible support" is about as accurate and useful as you can get. The extra distinction doesn't seem 100% reliable to me and still means that the user has to go out and find compatible versions of CHDK and "client".
It's also about style.The function does change. Only looking at its signature or something is rather arbitrary. :P
To me, changing the function name implies that something has changed about the function itself (new signature, different return value etc).
Apart from the change to PTP_CHDK_CallFunction I'm not sure I really see much difference between the two approaches. One puts the code in Lua although this could be refined (see below), the other puts it in PTP - does the distinction really make a difference?The distinction is the Lua part is completely artificial. There is no use for it in Lua, it is, as mweerden said, just a way of calling a function by name, so you can pretend that the protocol hasn't changed. This is not helpful, because anything that actually *uses* it will be affected by changes just the same as a protocol change. It just makes the whole thing more opaque and complicated.
To me that is (almost) like saying HTTP is part of TCP.Right, if you want to define another protocol for live view, that runs on top of CHDK PTP, but IMO that's not warranted.
that's also why I wouldn't be a fan of your monolithic PTP_CHDK_GetStreamingData. It only takes one of the flags to change in meaning to destroy backwards compatibility.Then don't change the flags unless you really need to, deprecate them, add new ones, and only drop them when you are forced to make a protocol change for other reasons. If you use up all 128 bits of params to PTP_CHDK_GetStreamingData, you can always add PTP_CHDK_GetStreamingDataEx if you really want to ;)
To me that is (almost) like saying HTTP is part of TCP.Right, if you want to define another protocol for live view, that runs on top of CHDK PTP, but IMO that's not warranted.
I think it makes sense to define the live view data transfer as a seperate protocol on top of PTP.Exactly. And because of this tools that do not user remove live view should never feel any consequences of changes to it.
It's a complex transfer including metadata, isolating it away from the core PTP keeps things cleaner.
PTP becomes a means to initiate the live view transfer (and the conduit over which the data travels), other than that it shouldn't care what's going on.
The model is extensible to other data transfers (no idea what this might be) without changing the PTP protocol again.
Centralizing other things that a client would also want to poll also has real value. If getting the palette and frame buffer separately hurts frame rate, then polling script status and live view will do the same.Now this is an actual argument for centralization. However, as I understand it the PC dealing with the image data is the main bottleneck at the moment. As mentioned a while back, I got a significantly higher fps by just dumping the image data in a file. I remember Phil having a similar result. So I wouldn't want to jump too quickly to this option.
Centralizing other things that a client would also want to poll also has real value. If getting the palette and frame buffer separately hurts frame rate, then polling script status and live view will do the same.Now this is an actual argument for centralization. However, as I understand it the PC dealing with the image data is the main bottleneck at the moment. As mentioned a while back, I got a significantly higher fps by just dumping the image data in a file. I remember Phil having a similar result. So I wouldn't want to jump too quickly to this option.
What I would like to know is if there is another way I could delete the file without causing this reset (maybe via os.remove)?os.remove from lua should be fine. Note that some parts of the camera OS may not notice that it has been deleted, so for example if you are browsing images in playback mode or over a regular ptp file browser, you might see the names of images that aren't really there. This is the same as if you delete from the CHDK file browser.
the USB connection resets (or at least disconnects/reconnects) when you erase an image (ptp_deleteobject).
Compiled version attached.
One note, there is a checkbox to 'autoupdate' the live view (also calculates the frame rate).
If you have this checked it will crash if you try and send other commands to the camera (it needs to stop the autoupdate before sending the command; but I haven't coded this yet).
Phil.
Back on page 10 there was a patch to make this work on an A540. Those links are broken now. Is there anyway to get this on my A540's?
got message "CHDK isn't started on cameraMake sure CHDK is loaded in play mode before you start ptpCamGui.
Make sure CHDK is loaded in play mode before you start ptpCamGui.CHDK is loaded automatically after power on. The yellow label with CHDK version is appeared on camera screen. But camera starts in play mode with retracted lens.
>> version << error: cannot get camera CHDK PTP version; either it has an unsupported version or no CHDK PTP support at allThis message means "CHDK ptp-task not run".
ptpcam: 2.0 (Length: 122) [unexpected return code 0x2005
unexpected return code 0x2005 (Length: 60)]
>> version << ptpcam: 2.0
camera: 2.0 (Length: 24)Another question I have is if this is possible to run continuously? Kind of like setting up a camera to take a certain amount of images over a given time AND transfer the data without me having to interfere inbetween? I am sorry again for the noobie questions... and thank you for any replies.You could buy or make an A/C adapter to replace the camera battery. Then use an Eye-Fi (http://www.eye.fi/) card to tranfer your pictures over wifi
Or does reading occur directly from the SD card?As far as I know, yes. The writing of the .jpg can't be prevented yet.
has anyone tested the speed of this process?
So, in theory it would be ideal to upload in batches rather than in single uploads. I suppose that if we were to take a bigger batch, the time should decrease even more, correct?I'd expect you will hit diminishing returns quite quickly.
Another question, could the receiving end of the image be nothing more than a minipc, where the whole loop wouldn't require a human to operate it? Of course, initial programming would be done, but is this feasible? For example, program the whole procedure to take 20 images, upload to the minipc, repeat, without any human interference inbetween. (1000 images total we would say)Yes, with the caveat that CHDK as a whole is a hack, and the PTP stuff even more so. You should not assume that it will be 100% reliable and stable. It might work fine, or you might run into mysterious crashes, hangs or other difficulties. If it turns out to be unstable in your situation, you should not assume that someone here will be able to magically fix it for you. If further hacking is required to make it work for your project, that will be up to you.
So, in theory it would be ideal to upload in batches rather than in single uploads. I suppose that if we were to take a bigger batch, the time should decrease even more, correct?I'd expect you will hit diminishing returns quite quickly.QuoteAnother question, could the receiving end of the image be nothing more than a minipc, where the whole loop wouldn't require a human to operate it? Of course, initial programming would be done, but is this feasible? For example, program the whole procedure to take 20 images, upload to the minipc, repeat, without any human interference inbetween. (1000 images total we would say)Yes, with the caveat that CHDK as a whole is a hack, and the PTP stuff even more so. You should not assume that it will be 100% reliable and stable. It might work fine, or you might run into mysterious crashes, hangs or other difficulties. If it turns out to be unstable in your situation, you should not assume that someone here will be able to magically fix it for you. If further hacking is required to make it work for your project, that will be up to you.
Or does reading occur directly from the SD card?As far as I know, yes. The writing of the .jpg can't be prevented yet.
Some older PowerShots do have official remote capture capability, but Canon decided to remove this from all recent models. Some info: http://www.gphoto.org/doc/remote/ (http://www.gphoto.org/doc/remote/)
is at all possible to take a picture and directly transfer it to the PC, without saving it to the card
is at all possible to take a picture and directly transfer it to the PC, without saving it to the card
Yes, with DNG.
I do it now.
For jpg and movies I simply delete the images from the card after download to the PC.
David
** init() start ...
>> version << ptpcam: 2.0
camera: 2.1 (Length: 24)
>> script-support << script-support:0x1 lua=yes (Length: 26)
>> luar not(os.stat("A/CHDK/LUALIB/lptpgui.lua")==nil) << script:15
15:ret:true (Length: 22)
>> luar require("lptpgui").version << script:16
16:ret:113 (71) (Length: 26)
>> luar get_buildinfo() << script:17
17:ret:'platformid 12816
platform sx30
version CHDK
platsub 100p
build_number 0.9.9-1487
os dryos
build_date Dec 15 2011
build_time 10:13:08
' (Length: 161)
is DryOS=True
CHDK-DE=False
>> luar get_config_value(67) << script:18
18:ret:1 (1) (Length: 23)
current powersave mode: 1
>> luar get_raw() << script:19
19:ret:0 (0) (Length: 23)
>> help << q quit quit program
h help list commands
r reset reconnect to camera
version get CHDK PTP version (ptpcam and camera)
shutdown shutdown camera (soft)
reboot reboot camera
reboot <filename> reboot camera using specified firmware update
reboot-fi2 reboot camera using default firmware update
m memory <address> get byte at address
m memory <address>-<address> get bytes at given range
m memory <address> <num> get num bytes at given address
set <address> <long> set long value at address
c call <address> <arg1> ... call function at address with given arguments
upload <local> <remote> upload local file to camera
download <remote> <local> download file from camera
mode <val> set mode (0=playback,1=record)
lua <code> execute lua code
luar <code> execute "return <code>" and retreive result
script-support show supported script interfaces
script-status show script execution and message status
getm get messages / return values from script
putm <message> send <message> to running script (Length: 1359)
* enable keys for GUI control: >> lua post_levent_to_ui(4484) << script:20 (Length: 9)
>> script-status << script-status:0x0 run=no msg=no (Length: 31)
** init() successful
Btw,It is in CHDK.
is there any reason why we not activate the PTP support for all cameras in camera.h?
It is in CHDK.
You're right. Sorry, I haven't seen the entry. So we can delete all entries in the platform_camera.h files for a better overview.Yes :)
Hello lads !This question doesn't really make sense to me.
Knowing we've just hit trunk1654, may I ask if the ptp handlers & usb descriptors are implemented generically, or does each platform (already ported) needs its own implementation of the CHDK ptp extension to enable live preview with mweerden's .Net app ?
Properly supporting live view is on the list for the next release ...List ? There's a list ?
http://chdk.wikia.com/wiki/Release_Strategy#Roadmap (http://chdk.wikia.com/wiki/Release_Strategy#Roadmap)Properly supporting live view is on the list for the next release ...List ? There's a list ?
http://chdk.wikia.com/wiki/Release_Strategy#Roadmap (http://chdk.wikia.com/wiki/Release_Strategy#Roadmap)This would be less embarrassing if my nick was not there in the revision history ...
The live view stuff is completely disabled in in the CHDK release branch, to avoid having an under development protocol "released". There is some of this code in the trunk, I don't know if it is compatible with whatever version of CHDKPTPRemote you are trying to use.
Okay, so if I understand it right, the device's code matching the expectations of CHDKPTPRemote is in the developper edition of CHDK,I'm not sure, there was some back and forth. In any case, existing builds will probably not be compatible with the final live view protocol.
and should appear in the trunk once the two of them are merged, as it has recently been meant in this topic : http://chdk.setepontos.com/index.php?topic=7601.0 (http://chdk.setepontos.com/index.php?topic=7601.0)That thread is discussing merging CHDKDE and the CHDK trunk. It looks like that stuff never made it into CHDK, but I'm not sure. Unless someone continues development of CHDKPTPRemote, then it will probably not be compatible with any future version of CHDK. If you want, you can try building the trunk build with chdkshell and see if it works.
So far, I've used mweerden's C# code to do a step-by-step debug of the LibUsbDotNet calls : his "ptpUtils" class filters the devices by enumerating its ptp-class interface endpoints, and flags the device as "ptp-able" if and only if one of its usb endpoints presents a 512-bytes capacity.This sounds to me like it's unrelated to CHDK protocol issues. It may be that mweerden just assumed all cameras would support that. CHDK doesn't see or change that kind of stuff on the camera, it just responds to certain a specific PTP commands. Setting up the endpoints would all be done by the Canon firmware. I'd suggest just changing his code to skip that check...
Maybe if the "live view" becomes a trunk's asset, I'll look into implementing a unified DLL (.Net and Mono) for everyone to control them easily via C# or VB code. I need it for automation vision anyway :P, so me making it "generic" would be a good idea. I'd love to feed a GPU's pixel shader with a kick-arse bitmap freshly grabbed from a Canon camera.If you can come up with shaders that efficiently transform the live view YUV format and paletted bitmap into something suitable to display, that could be very useful.
This sounds to me like it's unrelated to CHDK protocol issues. It may be that mweerden just assumed all cameras would support that. CHDK doesn't see or change that kind of stuff on the camera, it just responds to certain a specific PTP commands. Setting up the endpoints would all be done by the Canon firmware. I'd suggest just changing his code to skip that check...
I've used an USB sniffing app to record some USB activity
Digital camera are way much more adequate for vision purposes than crappy 768p webcams
I'd love to feed a GPU's pixel shader with a kick-arse bitmap freshly grabbed from a Canon camera.
No. :) 360x240 is the resolution CHDK works with (at least this was the case a few months ago).Digital camera are way much more adequate for vision purposes than crappy 768p webcams
Well, LiveView is 320x240.
QuoteDigital camera are way much more adequate for vision purposes than crappy 768p webcams
Well, LiveView is 320x240.
QuoteI'd love to feed a GPU's pixel shader with a kick-arse bitmap freshly grabbed from a Canon camera.
You cannot grab a bitmap, just the low-resolution YUV viewfinder image or a raw image (at far slower framerate) that needs converting to bitmap.
The raw image (especially at wide angle) has very large lens distortions, not exactly suitable for machine vision unless individually calibrated.
I convert the viewfinder YUV image to bitmap for capture and display.
The real resolution you usually get is 720x240.that has been the case from the earliest days, GrAnd even posted an image showing it was expanded 2x in width.
process a large amount of raw data (YUV)
Can you give more details of that, I need it to solve a problem ?
Luminance resolution is 720. U/V is 160!The real resolution you usually get is 720x240.that has been the case from the earliest days, GrAnd even posted an image showing it was expanded 2x in width.
It was suggested this might be to give better results when interpolated to 360x240.
It is not really 'resolution'.
Raw data, YUV, YCrBr, CMYN, whatever.... color matrix lad, co.lor. maaa.trix . :DRaw data = bayer 10 or 12 bit RGB. Different animal. But you can't get that at any reasonable speed. Downloading a 18mb raw file from my D10
con 1> !cli.showtime=true
time 0.0000
con 1> d -nolua DCIM/100CANON/CRW_0714.CRW
time 4.4093
Taking the SD card out of the equation would help a bit, but not enough, I think...
So, what matrix will you use to convert raw data to a colour space, see line 219 here http://trac.assembla.com/chdk/browser/trunk/core/raw.c. (http://trac.assembla.com/chdk/browser/trunk/core/raw.c.)
It is raw sensor data.
Raw data = Bayer 10 or 12 bit RGB. Different animal. But you can't get that at any reasonable speed. Downloading a 18mb raw file from my D10Code: [Select]con 1> !cli.showtime=trueTaking the SD card out of the equation would help a bit, but not enough, I think...
time 0.0000
con 1> d -nolua DCIM/100CANON/CRW_0714.CRW
time 4.4093
Fact is, I'm not planning to retrieve RAW data, it was Microfunguy who brought it in the conversation. :-X As long as I manage to get picture data, whatever the encoding is (being intertwined bayer or not) without relying on an extraneous jpg compression and file exchange, I'm good.The only other data you can get easily is the live view, which as discussed is quite low resolution. Of course if you want to take an actual exposure, you can get a nice high res jpeg after a small delay ;)
Should we grab the frame buffer once the camera has processed its sensor data, if it ever exist sometime in its RAM (the jpeg encoder may process the Bayer data directly, I dunno, that your expertise domain. :PI'm pretty sure this is the case, I don't think there is a de-bayered version of the image outside of jpeg, with the probable exception of the half res high ISO/fast modes and video.
I get a "Trac Error , No node /trunk/core/raw.c. at revision 1665".Try without the period on the end:
I'm not planning to retrieve RAW data, it was Microfunguy who brought it in the conversation. :-X
I don't think there is a de-bayered version of the image outside of jpegOn (at least) DIGIC II, when reduced image size is requested, the first phase of the picture creation process is a resampled YUV (UYVY) image.
, with the probable exception of the half res high ISO/fast modesAs I wrote in another thread a few days ago, modern CCDs (I read some Sony CCD product briefs) have a special readout mode which combines pixels (therefore the Bayer matrix arrangement probably gets lost), and is fast.
I rather think you are, unless you prefer the distortion-corrected,nicely colour-rendered JPG's.
Only drawback is white-balance has already been applied.
Before you get too carried away with this can I ask for technical details of the proposed application ?
Based on what existing PTP clients can do (including my own) I cannot see that you need anything special, bearing in mind the constraints mentioned.
machine tool inspections (what is the actual tap or drill bit dimensions
This can only be achieved with high-resolution devices.
direct snapshot downloading without messing with the flash card
Well, if you are serious about such applications you need optics that are telecentric in object space, normally available in 'C' mount for machine vision cameras.
Hello lads !This question doesn't really make sense to me.
Knowing we've just hit trunk1654, may I ask if the ptp handlers & usb descriptors are implemented generically, or does each platform (already ported) needs its own implementation of the CHDK ptp extension to enable live preview with mweerden's .Net app ?
CHDK PTP protocol has certain commands. These are supported generically in CHDK code. A build for any camera should be compatible with with any CHDK PTP client implementing the matching protocol version. You don't need to worry about handlers and descriptors... although I suppose it's possible that the a430 is old enough that it's weird in some way. If you can upload/download with chdkptp or ptpcamgui, then CHDK PTP is implemented and working properly. Neither one will give you live view however.
The live view stuff is completely disabled in in the CHDK release branch, to avoid having an under development protocol "released". There is some of this code in the trunk, I don't know if it is compatible with whatever version of CHDKPTPRemote you are trying to use.
Properly supporting live view is on the list for the next release, but is not finished. The protocol will likely change from whats in the trunk. I keep meaning to do some work on this Real Soon Now. Maybe this weekend...
Yes, that is OK.
I was doing it yesterday with a stop-motion test.
at the moment i cannot switch to SDM
too dependent on other features of CHDK
can you tell me if direct capture to PC without the card is possible with CHDK ??
i am using mweerden's live preview code along with regular remote control, my application is actually simple the camera is inaccessible in a container(i open up the camera and solder the power connector, so applying power to the camera turns it on) and i need to control it via the pc - however i need a preview, i might use a A/V cable but live preview works ok.at the moment i cannot switch to SDM
It is not available in the 'official' 1.85 version.
Spring is almost here in the Northern hemisphere so it is about time that I got this released.Quotetoo dependent on other features of CHDK
As a matter of interest, which features ?
i do not know how much of a lag downloading and converting from DNG will introduce. but i dont think it will save me much time. at the moment capturing a picture and saving takes around 6-8 seconds with the preview constantly on (pauses during download). in your opinion is it worth the effort - i mean will direct capture to pc and its subsequent conversion to jpg (with the distortion correction) halve the process time, can i expect to download process to complete in aroud 3-4 seconds. that may justify my changing the application interface.Quotecan you tell me if direct capture to PC without the card is possible with CHDK ??
I do not know specifically about CHDK and it depends what you mean by 'without the card'.
With SDM, the best you can do is save a tiny jpg (which is later deleted) and download the raw data and instantly convert to DNG on the PC.
i am contemplating doing this offline. capturing a video
can i expect to download process to complete in aroud 3-4 seconds. that may justify my changing the application interface.
It is the only way you will get 5 fps.
as a matter of interest, I used 640x480 movie frames from a SINGLE rotating camera to create the top stereo panorama here : http://www.stereomaker.net/panorama/panoramae.htm (http://www.stereomaker.net/panorama/panoramae.htm)
Here are some technical details http://stereo.jpn.org/eng/stphmkr/help/file_27.htm (http://stereo.jpn.org/eng/stphmkr/help/file_27.htm)
...
Either you need 5 fps or you do not.
The only in-between is to run in burst-shooting mode at whatever resolution you wish.
So, you can have 5 fps, 0.3 fps or something in-between.
The accumulated images can be batch-downloaded at intervals via PTP.
Bear in mind that my A620 uses MJPEG encoding for the movies, every frame is a JPG.
With the more recent MOV files, most frames are interpolated.
I would use VirtualDub to change the framerate to 5 fps, save as an image sequence and then inspect them to determine the quality loss.
i dont intend changing the frame rate in virtualdub
i think it is the way to go. also as i am using SX 130 i would be getting 1280 x 720 (or thereabouts i think) and i have done some test runs the video quality is excellent for my requirement.
sorry about the delay ....
here are the snaps from the clip
Many thanks for that.i hope it is sufficient for your purposes. if you require i could get the thing outdoors and get you a clip. lazy is that i am, i just took the clip sitting at my table :). yes that mess is actually my office table.
Yes, they are not bad at all, especially when you consider the fairly low light level.
I set framerate in VirtualDub to 3 which therefore selected 26 frames.for movie i use VirtualDub. for extracting the frames from the .mov files i am going to write a utility in vb6. (hoping it would not be more effort than what i am anticipating). if you are intrested i will upload it once it is done.
It would not detect that value for export as image sequence so i saved as uncompressed AVI, reloaded and saved as image sequence.
What software are you using ?
i do not think movie mode allows any setting other than White balance, Movie Quality and resolution etc. i am not sure about CHDK. dont have it at hand, will revert back.
Can you lock the exposure in movie mode ?
This camera, and others, would capture 180 frames in six seconds .. sufficient for most 360 degree stereo panoramas.
Just need a 10 rpm rotator.
Something like a cordless screwdriver could rotate the camera.
Wa wa wa wa wa wait... :oit is not a PTP capture, just a regular capture from the camera. but PTP is perfectly capable of capturing in movie mode and downloading the clip.
What's the nature of this clip again ? Is it a PTP capture, or a movie-mode shoot, downloaded as a file from the device ? ???
i am going to write a utility in vb6. (hoping it would not be more effort than what i am anticipating). if you are intrested i will upload it once it is done.
in case AC is a problem outdoors
yes a plugin in VDub would be one way of doing it, but i need the end user to feel that the whole damn thing is a proper solution without patchwork. i could probably call VirtualDub in the background but still i would like to have control and know when exactly the process is completed. also i always think of a solution in VB so probably i can do it faster that way.i am going to write a utility in vb6. (hoping it would not be more effort than what i am anticipating). if you are intrested i will upload it once it is done.
Yes, thanks.
If you were really clever you could write a VDub plugin but ther last time I looked at the code it was very complicated.
thanks will checkQuotein case AC is a problem outdoors
I will probably use DC motor but you have reminded me that I have a 12V DC/240C AC converter that I have not used.
You can lock the exposure on your SX130is in movie mode, see page 103 of the manual.
David
i always think of a solution in VB so probably i can do it faster that way.
QuickTime or Quicktime alternative (i think that is what it is called) is still required, but it wont be requiring VirtualDub. i already have the utility for extracting frames from avi's ready and working. mov should not be very different. however i will not be available in the next week so will be able to see how it turns up only after that.i always think of a solution in VB so probably i can do it faster that way.
Are you saying a VB application can extract the frames without requiring the VDub MOV plugin and without having QuickTime installed on your PC ?
contrary to belief, you can do quite a lot in vb. eg. my HID dongle (software lock) is totally accessed by vb6(totally no dlls etc), the hardware i use -stepper motor controller, etc is controlled by vb6 via usb apart from the database access etc which my regular softwares use. it is relatively faster to code in vb6, generally(compared to c or c++ for eg) which leaves more time for better features. and for a small (tiny) enterprise like mine (around 10 people) it is the way to go.Slightly off topic now, but Microsoft no longer supports vb6. Obviousy, its still usable but you never know when the next OS version or other package will no longer be compatible.
sorry to keep off topic - you are perfectly right waterwingz - but remember in most of these kind of utilities we are using the api anyways and if the next os breaks compatibility, unless it breaks the vb runtime specifically, i dont see y it would not break the same utility written in C or C++ too. anyways i am taking my risk - no doubt my new projects are in .net. but i hardly see the logic in writing a small utility in .net and then expecting for someone to download and install the framework (you would not believe how great of a percentage of my clients still use XP)contrary to belief, you can do quite a lot in vb. eg. my HID dongle (software lock) is totally accessed by vb6(totally no dlls etc), the hardware i use -stepper motor controller, etc is controlled by vb6 via usb apart from the database access etc which my regular softwares use. it is relatively faster to code in vb6, generally(compared to c or c++ for eg) which leaves more time for better features. and for a small (tiny) enterprise like mine (around 10 people) it is the way to go.Slightly off topic now, but Microsoft no longer supports vb6. Obviousy, its still usable but you never know when the next OS version or other package will no longer be compatible.
I'm trying to get PTPcamGui working with my A560. Downloaded the latest build, and PTPCamGUI hangs on get_mode() when trying to switch to capture mode. There was a post about half a year ago, but couldn't really find a followup. Assuming this is the same issue, but wondering where to start to tackle this...
i hardly see the logic in writing a small utility in .net and then expecting for someone to download and install the framework
(you would not believe how great of a percentage of my clients still use XP)
hello webguy,- Using latest PTPcamgui, version 2.0.113
can you verify the ptpcamgui version is latest
try using the CHDK DE version and see if it solves the problem
try PTPcam command line and verify that it actually fails on get_mode()
check if other commands work
>> luar get_mode() << script:10
10:ret:false (Length: 23)
>> luar get_mode() << script:11
11:ret:false (Length: 23)
>> luar get_mode() << script:12
12:ret:false (Length: 23)
>> luar get_mode() << script:13
13:ret:false (Length: 23)
>> luar get_mode() << script:14
14:ret:false (Length: 23)
>> luar get_mode() << script:15
15:ret:false (Length: 23)
>> luar get_mode() << script:16
16:ret:false (Length: 23)
>> luar get_mode() << script:17
17:ret:false (Length: 23)Other commands on the command line seem to work, even when using the ptpcam command tool.- Other commands seem to work, but get mode() or set_record() don't seem to work.All my tests showed, that we should not use set_record() on lua code over ptp. The better way is ptp command "mode".
unless it breaks the vb runtime specifically, i don't see y it would not break the same utility written in C or C++ too.
i hardly see the logic in writing a small utility in .net and then expecting for someone to download and install the framework (you would not believe how great of a percentage of my clients still use XP)
I really hate this lazy reliance on the .Net framework.
It makes it easier for the developer but inconvenient for many users.
I prefer to do development on an old PC because it is connected to a large-screen TV and is easier to view than my modern system.
There are a number of applications I will not run because of the bloat the .Net framework would add.
Quote
(you would not believe how great of a percentage of my clients still use XP)
I believe it, I do.
I see no reason to change, it does not cause me any problems, just the opposite.
I've finally got live view working enough in chdkptp that I can work on the protocol.
The CHDK side of this is in a branch http://trac.assembla.com/chdk/browser/branches/reyalp-ptp-live (http://trac.assembla.com/chdk/browser/branches/reyalp-ptp-live) but will probably move it back to the trunk once I've got a few things sorted out.
Maybe this is only for developers.Exactly. CD_SUPPORT and LIVEVIEW_SUPPORT are makefile variables in the chdkptp source, not files anywhere.
Does this mean your code only concern the A540 platform ? ;)Most of it is generic, but there will be platform specific issues, and some code will need to be added to most CHDK ports for full functionality.
The trunk would need to implement something bigger than the tiny 64 bytes bulk endpoints I see on the A430 :haha). But anything different from bulk would make us step out of the PTP domain, right ?As I explained before, this is a low level detail outside of the scope of anything CHDK interacts with or modifies. Whatever reason mweerdens code checked it is not directly related to this functionality. If file download works, then we can transfer framebuffer data too, there's no difference as far as the low level USB stuff goes.
I can't wait to see a new set of Chdk PTP subcommand to request a live PTP feed
As a matter of interest, what application do you have that requires the live feed ?
I have no specific application to use live feed in (I don't even own a Canon device that would handle it).Does CHDK file download work on your camera ?
I'm just thinking that adding this functionality to the existing CHDK PTP extension subcodes would be the best way to have uniform live-feed implementation over every platformThis has always been the plan. However, getting the appropriate data for each camera necessarily requires some new code in the ports. There's no way around that, Canon has used different frame buffer layouts etc on different cameras.
supporting USB 2.0, rather than distinct implementations and protocols for each platform.USB 2.0 is irrelevant. As I've repeatedly tried to tell you, CHDK PTP code know doesn't or care about anything at that level.
I have no specific application to use live feed in (I don't even own a Canon device that would handle it).Does CHDK file download work on your camera ?
If you actually can't run live view, then I certainly don't see how you'd successfully develop a client library.
QuoteI'm just thinking that adding this functionality to the existing CHDK PTP extension subcodes would be the best way to have uniform live-feed implementation over every platformThis has always been the plan. However, getting the appropriate data for each camera necessarily requires some new code in the ports. There's no way around that, Canon has used different frame buffer layouts etc on different cameras.
Quotesupporting USB 2.0, rather than distinct implementations and protocols for each platform.USB 2.0 is irrelevant. As I've repeatedly tried to tell you, CHDK PTP code know doesn't or care about anything at that level.
I think the CHDK Ptp extension needs a few new subcodes
I didn't had/take the time to dwell into the CHDK code deeply enough to do it myself, but if I did
I think the CHDK Ptp extension needs a few new subcodes
I didn't had/take the time to dwell into the CHDK code deeply enough to do it myself, but if I did
aaahhhh .... yes ... of course you would.
actually, you caught him on a good day ::)Are youIs he always this denigrating and condescending to others ? :blink:
I'm aware of that. As a matter of fact, I think the CHDK Ptp extension needs a few new subcodes, something like :If you want to contribute to this in a meaningful way, you'll need to familiarize yourself with what is actually being done. FWIW, one of the complications is the that the frame buffer characteristics change at runtime depending what the camera is doing. That's why many of them are included in each frame (and I'm leaning to just sending them all every frame)
- GetFrameBufferDesc which would return a frame buffer structure and layout description structure as a ptp data payload.
- BeginLiveView (or whichever name you give it), which start the VF stream
- AbortLiveView (note : same as above), which stops the feed.
Shame is, I didn't had/take the time to dwell into the CHDK code deeply enough to do it myselfThis limits your ability to meaningfully participate in the discussion.
, but if I did, things like Script execution status or memory exchange controls would be implemented as device's ptp EVENTS, not polled commands. As far as I know, ptpCam doesn't even USE the interrupt endpoint, although can MUST have it implemented on every device, meeting protocol standards.If you reverse engineer the camera stack enough to do this, we certainly might be interested. At the moment, we don't have this capability, and polling is good enough.
So, the endpoints MaxPacketSize(s) are platform-dependant ? Nothing the CHDK can fiddle with ?Nothing that CHDK currently fiddles with. We use the cameras PTP stack, we don't talk to the low level USB stuff. It might be doable, but there's no obvious need... if your camera is USB 1.1, changing the packet size isn't going to help your transfer rate that much.
asmodyne,
Do not you think it is time to dwell into the CHDK code deeply enough, as you said.
And then came back and be so kind to tell us what you did.
10x
QuoteShame is, I didn't had/take the time to dwell into the CHDK code deeply enough to do it myselfThis limits your ability to meaningfully participate in the discussion.
QuoteSo, the endpoints MaxPacketSize(s) are platform-dependant ? Nothing the CHDK can fiddle with ?Nothing that CHDK currently fiddles with. We use the cameras PTP stack, we don't talk to the low level USB stuff. It might be doable, but there's no obvious need... if your camera is USB 1.1, changing the packet size isn't going to help your transfer rate that much.
where should I start reading if I want to understand you live-view code btw ? The A540 sub in the repository link you gave earlier, is that it ?)The chdk side is currently in http://trac.assembla.com/chdk/browser/branches/reyalp-ptp-live (http://trac.assembla.com/chdk/browser/branches/reyalp-ptp-live)
I see. So it solely relies on the mechanism mweerden describes: registering extraneous/extern handlers to ptp commands, right ?Exactly. You need to appreciate that CHDK is hack. It's based on reverse engineering and manipulating the internals of a completely undocumented system. So often the approach taken is not the one you would think is "best" in normal design terms or based on reading specifications. Furthermore, as I mentioned before there is a very large number of platforms, so even if a "better" way is discovered, it may not be pursued if implementing requires a lot of work on each camera.
It is not A540 specific, although obviously I'm working with the cameras I have before worrying too much about others. The basic live view should work on most cameras. Getting things like the UI overlay and exactly the right display and aspect rations etc will require additional work on most cameras.
Okay then. What made me think your work on live view was relying on A540 was the "reyalp-ptp-live" and "live view ...smtg :-[" in the "comments/description" column of the repository browser. I (stupidly) assumed you were using platform-dependant hardcoded offsets were to fetch the frame buffer.Of course I am, the offsets *are* platform dependent, and since they are found by reverse engineering, they are generally hardcoded somewhere, or pulled from variables in the camera which are themselves platform dependent. See platform/<camera>/lib.c, platform/camera_platform.h and platform/<camera>/sub/<version>/lib.c
As I said before, I'll try to put a binary build of chdkptp soon.This is done, see here http://chdk.setepontos.com/index.php?topic=6231.45 (http://chdk.setepontos.com/index.php?topic=6231.45)
However, the basic frame buffer addresses and characteristics are already known for all complete ports, since they are required for histogram, zebra, md etc. There are some additional characteristics and special cases required for live view which that are *not* already in the code, and these will have to be added for each camera to enable full functionality. There is no way around this.
// PTP display stuff
int vid_get_palette_type() { return 1; }
int vid_get_palette_size() { return 16*4; }
void *vid_get_bitmap_active_palette() {
return (void *)0x634E0; // GetPaletteFromPhysicalScreen
}
void *vid_get_bitmap_active_buffer()
{
return (void*)(*(int*)0x5ED0); // FFD23420 DisplayPhysicalScreenWithYUVPalette
}
Uuuh...since I'm not sure... the A540's base ROM address is 0xFFC00000, right ?Yes.
The main diff I noticed (apart from the platforms offsets, of course), is at the very end of your file :FWIW, a better way to see what I've changed is http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live (http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live) (or check out the source with your favorite svn client and look at the logs and diffs)
I don't actually remember :-[ Someone (probably me, possibly ewarv) found them a long time ago to support ewavrs chdkcam liveview code. That history can probably be found somewhere near the start of this thread...
To do so, I have to locate any usage references about "GetPaletteFromPhysicalScreen" and "DisplayPhysicalScreenWithYUVPalette"... but DisplayPhysicalScreenWithYUVPalette doesn't even appear in the A540 dump ! I've run it through IDA: nothing ! Where did you find it reyalp ? :-X
FWIW, a better way to see what I've changed is http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live (http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live)...
DisplayPhysicalScreenWithYUVPalette appears to be something identified by the A series flirt sigs. See http://chdk.wikia.com/wiki/Loading_dump_to_IDA (http://chdk.wikia.com/wiki/Loading_dump_to_IDA)
GetPaletteFromPhysicalScreen is an eventproc, so should be easier to find.
Very useful indeed. I'm a noob with assembla. TIL something new about it, thanks reyalp ! :xmasIf you are going to be working with the source, I strongly suggest getting tortoisesvn (assuming you are on windows) and checking out the code with it.
Excellent. I now have the offsets I missed in "./sub/lib.c". 8)FWIW, you don't need any of these bitmap/palette functions for the most basic live view function.
If you are going to be working with the source, I strongly suggest getting tortoisesvn (assuming you are on windows) and checking out the code with it.Command line svn commands under Linux turn out to be not too bad. Still, the integration of TortoiseSVN into Windows that allows the file and directory icons in change to reflect their svn status is a nice feature I miss a lot. Yes, you get the same info with svn st chdk on the command line but its not the same somehow.
Still, the integration of TortoiseSVN into Windows that allows the file and directory icons in change to reflect their svn status is a nice featureYou may have a look to RapidSVN (http://rapidsvn.tigris.org/ (http://rapidsvn.tigris.org/) ) for a Linux GUI.
bm_max_width:360
bm_max_height:240
bm_buffer_width:360
lcd_ascpect_ratio:0
update_vidinfo: changed
vp_xoffset:nil->0
vp_yoffset:nil->0
vp_width:nil->720
vp_height:nil->240
vp_buffer_start:nil->0
vp_buffer_size:nil->0
bm_buffer_start:nil->44
bm_buffer_size:nil->86400
palette_type:nil->1
palette_buffer_start:nil->86444
palette_buffer_size:nil->64
palette:
0x00, 0x00, 0x00, 0x00, 0xff, 0xe0, 0x00, 0x00, 0xff, 0x3c, 0xde, 0x4a, 0xff, 0xb9, 0x00, 0x00,
0x7f, 0x00, 0x00, 0x00, 0xff, 0x6e, 0xa1, 0xa2, 0xff, 0xb2, 0x8c, 0x1f, 0xff, 0x5f, 0x00, 0x00,
0xff, 0x9a, 0xa9, 0x50, 0xff, 0x8a, 0x50, 0xb0, 0xff, 0x4b, 0x3d, 0xd4, 0xff, 0x23, 0x1b, 0xe2,
0x7f, 0x00, 0x7b, 0xe2, 0xff, 0x30, 0x00, 0x00, 0xff, 0x69, 0x00, 0x00, 0xff, 0x00, 0x00, 0x00,
recording to chdk_30f8_20120228_132726.lvdump
But as soon as we ask for the viewfinder feed, the beast crashes.The camera, or the client ?
I suppose all those variable followed by a "nil->xxx" means the value was missing and replaced by default constants, right ?Not quite. It only prints this when the information changes (since it would be bad to spam every frame). The x->y just tells you the old and new value. On the first frame, the "old" values are all nil. The code for this can be seen in lua/gui_live.lua in the chdkptp package.
But as soon as we ask for the viewfinder feed, the beast crashes.The camera, or the client ?
I'm not clear if the information in the console comes from before crash (in which case, it tells us nothing) or relates to the data that caused the crash.
The camera may generate a romlog when it crashes, see http://chdk.wikia.com/wiki/Debugging#Camera_crash_logs_.28romlog.29 (http://chdk.wikia.com/wiki/Debugging#Camera_crash_logs_.28romlog.29)
watching the return value of vid_get_viewport_active_buffer may shed light on the situation (see misc debug vals in chdk core/gui.c ).
Note that you can enable "viewfinder" in playback mode, it will show you whatever picture is on the screen. This may fail (return null address, crash) if a video is selected.
| transferred | max fps | T/P avg kB/s | Xfer last msec | Xfer last kB/s |
| viewfinder | 3 (barely) | ~755 | ~310 | ~810 |
| ui overlay | 4 (5 is unstable) | ~337 | ~159 | ~530 |
| both | 2 | ~670 | ~415 | ~835 |
The camera hasn't crashed since I put the additional functions into lib.c (may be a coincidence).
void *vid_get_bitmap_active_buffer()
{
return (void*)(*(int*)0x5EAC); // sub_ffd0cae8 DisplayPhysicalScreenWithYUVPalette
}
void *vid_get_bitmap_active_palette() {
return (void *)0x714a8; // found also in sub_ffd0cae8
}
int vid_get_palette_type() { return 1; }
int vid_get_palette_size() { return 16*4; }
// PTP display stuff
int vid_get_palette_type() { return 1; }
int vid_get_palette_size() { return 16*4; }
void *vid_get_bitmap_active_palette() {
return (void *)0x714A8; // GetPaletteFromPhysicalScreen
}
void *vid_get_bitmap_active_buffer()
{
return (void*)(*(int*)0x5EAC); // loc_FF90CB50 DisplayPhysicalScreenWithYUVPalette for a430 with ROMSTART = 0xFF800000
}Index: platform/a430/sub/100b/lib.c
===================================================================
--- platform/a430/sub/100b/lib.c (revision 1701)
+++ platform/a430/sub/100b/lib.c (working copy)
@@ -46,7 +46,8 @@
void *vid_get_viewport_fb_d()
{
- return (void*)(*(int*)0x71AB0); // @ffd0f29c
+ int x=(*(int*)0x71AB0); // @ffd0f29c
+ return (void*) (x ? x : vid_get_viewport_fb()) ;
}
long vid_get_viewport_height()
@@ -59,3 +60,15 @@
{
return (char*)0x7C588;
}
+
+void *vid_get_bitmap_active_buffer()
+{
+ return (void*)(*(int*)0x5EAC); // sub_ffd0cae8 DisplayPhysicalScreenWithYUVPalette
+}
+
+void *vid_get_bitmap_active_palette() {
+ return (void *)0x714a8; // found also in sub_ffd0cae8
+}
+
+int vid_get_palette_type() { return 1; }
+int vid_get_palette_size() { return 16*4; }
For me, it appears that when I select a target fps that cannot be reached, the chdkptp ui controls seem to freeze (start to lag, text no longer updates).Yes, this is known. chdkptp is currently dumb, it just runs a timer to read usb and update the live view. This seems to behave particularly badly on linux when the code takes >= the timer interval.
Camera buttons on the gui have no effect until record mode is activated (but once that happens, they also work in play mode). After reconnecting to a running camera with a previously activated record mode the gui cam buttons also work.This is normal. On newer (dryos ?) cameras you can use the magic post_levent_to_ui(4484) to unlock. For old cams, I believe switching to rec is the only known method.
long vid_get_viewport_height()
@@ -59,3 +60,15 @@
{
return (char*)0x7C588;
}
long vid_get_viewport_height()
@@ -59,3 +60,15 @@
{
return (long)0x7C588;
}
@asmodyne
That was a unified diff (http://en.wikipedia.org/wiki/Diff). It has since been committed. Get a fresh svn copy, then see that lib.c :)
Next thing, I'll bring myself to undersand the mechanics of the view stream structure (if I want to decode it).The structures defining the live view data are in CHDK core/live_view.h. However... this is going to change because it doesn't adequately describe all the characteristics a client needs to display correctly in every case, so don't get too attached to it. This only describes the layout of the information, not the actual encoding of the framebuffers.
It's in your ChdkPtp/lua sources, izznit ?
I'll try to implement a proper way to pass the data container's payload directly to a pixel shader (to get an easy-to-use bitmap output).For this, you should mostly be worried about the frame buffer formats, described here http://chdk.wikia.com/wiki/Frame_buffers (http://chdk.wikia.com/wiki/Frame_buffers)
Will I use DirectX or OpenGL, I still cannot say. I'm a tadbit inclined to use SharpGL though (i.e openGL).If you use shaders that can be sent directly to opengl, it should be quite easy to use this in CHDKPTP, which has native support for GL canvases.
Another thing, developers are disappointed by low frame-rates.I think 10fps is perfectly adequate for real time control of the camera.
This is all great coding fun, but realistically why does the framerate matter, what will this feature be used for ?
doing the rendering in opengl or similar gives you a lot of flexibility, and performance for free. Opengl is also very portable.
If it was only for resizing or color shifting or clamping purposes, one wouldn't even think of a pixel shader for such low resolutions.
Not sure If I have to create a 1024x256 image if I want OpenGL to do the resizing.
That is a bit messy, slower than a simple copy of a continuous buffer area.
Also, not sure how common the graphics-card support is for the non-power-of-two functions (cannot remember which one offhand).By power-of-two, you're speaking about the graphic cards resolutions, right ? :)
Another thing, developers are disappointed by low frame-rates.Higher framerate ? To do what ? Some kind of remote video recording ? One just have to buy a bigger SD card, or think about implementing a compatibility extension for SDHC up to 16GB then, cuz' the camera would do it just fine, in higher resolutions than the crappy viewport's 240 lines.
However, I do not have shader support on my old PC that I use, not sure how many people do.
However, doing the rendering in opengl or similar gives you a lot of flexibility, and performance for free. Opengl is also very portable.Gosh, it's so portable I remember making in run on Pocket PCs ! ( <-- re-edit spoiler : IT WAS A TWEAK, don't get your hopes too high ! :P )
By power-of-two, you're speaking about the graphic cards resolutions, right ? :)Originally, opengl only supported power of 2 textures. There have been extensions for a long time that support other dimensions, I would guess they are widely supported.
Higher framerate ? To do what ? Some kind of remote video recording ? One just have to buy a bigger SD card, or think about implementing a compatibility extension for SDHC up to 16GB then, cuz' the camera would do it just fine, in higher resolutions than the crappy viewport's 240 lines.Not all the cameras are limited to 240 lines. a540 (and presumably most similar vintage digic II) does 528 lines in 640x480 video mode, and some newer cameras like the g12 variety of 480 line modes. So decent resolution video is sometimes available directly over USB.
Anyway, its obviousl the old A-series USB peripheral won't handle higher bandwidths. Even with the padding bytes removed from the buffer, I don't think one can achieve more than 5fps anyway...with high hopes.Only on the very old USB 1.1 cams. The vast majority of CHDK supported cameras are USB 2.0
At the moment, I resize from 720x240 to 320x240 in 'C' code.I currently do it even cruder (like the CHDK code that reads the viewport): Just skip 2 Y values of every U Y V Y Y Y when doing the initial conversion. If you are going to a 360x240 display anyway, I don't think you are losing much.
It is crude nearest neighbour.
So, the question is .. can OpenGL do the resizing (not just cropping but squashing horizontally) and if so does the appropriate function that renders to a quad require a power-of-two source image ?I'm not sure using OpenGL to do the resizing would be profitable if you absolutely WANT to write a method to do the interpolation.
How would we know if was been hardware accelerated anyway ?
I currently do it even cruder (like the CHDK code that reads the viewport): Just skip 2 Y values of every U Y V Y Y Y when doing the initial conversion. If you are going to a 360x240 display anyway, I don't think you are losing much.
I'm pretty sure that passing a 320x240 P-buffer to and OpenGL's 640x480 viewport would make your card's OpenGL driver implementation do the interpolation automatically
It's all INTEGERS magic ! No floating arithmetic here ! (^w^)
The viewport image is 720x240 or 360x240 (3:2 aspect-ratio not the correct 4:3).
Yes, the thought of software floating point is what has put me off writing interpolation code.
I guess everything could be scaled-up by a 1000x or so.Waddayamean ? :blink:
Viewport's dimension are irrelevant in this case, since you're working in windowed mode.You aren't following. In CHDK, we call the camera live view (view finder image or display of selected jpeg/avi in playback mode) the "viewport". This has specific fixed dimensions, aspect ratio etc, and is completely unrelated to the opengl concept of a viewport.
You can even use a 551 wide per 777 heigh viewport, FWIW, openGL won't care as long as you'r not in fullscreen mode.
have a look at those hqNx filters
Sorry, I realize I wasn't explaining myself correctly. :-XViewport's dimension are irrelevant in this case, since you're working in windowed mode.You aren't following. In CHDK, we call the camera live view (view finder image or display of selected jpeg/avi in playback mode) the "viewport". This has specific fixed dimensions, aspect ratio etc, and is completely unrelated to the opengl concept of a viewport.
You can even use a 551 wide per 777 heigh viewport, FWIW, openGL won't care as long as you'r not in fullscreen mode.
"The filter was not designed for photographs, but for images with clear sharp edges, like line graphics or cartoon sprites.Aaaawww... Too bad then. It was a good alternative to nearest-neighbor scaling, without having to use floating-point arithmetic though.
It was also designed to be fast enough to process 256x256 images in real-time."
"The filter was not designed for photographs, but for images with clear sharp edges, like line graphics or cartoon sprites.
It was also designed to be fast enough to process 256x256 images in real-time."
However, to get nice video you'd also have to somehow synchronize the frame grabs with the camera refresh, or you get tearing.You may want to look at sub_ffc95b68 in A540. This function is used by MovieRecordTask to install and remove (frame notification and information) hooks into the liveview code.
I had good hope this algorithm would give good result when resizing small frames in real time...Honestly, I don't think this is anything that we have to worry about for any vaguely recent PC.
Do you have any new about the way you transfer frames buffer descriptors in your modified trunk, or are those still included in each frame ?I have not changed that yet. You can see all changes in that branch here: http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live (http://trac.assembla.com/chdk/log/branches/reyalp-ptp-live)
Okay. Perfect then, it will give me time to catch up with your work.Note that for CHDK, you don't need most of this. We don't use events (don't know how to do them on the camera end), and we don't rely on any non-CHDK commands outside of those needed to set up a session. Having the standard PTP could be useful of course, just saying they aren't required.
So far, I've implemented every PTP command from the PIMA15740-2000 white papers.
The only things remaining are the events on the interrupt pipe I'll have to encapsulate somehow (I cannot leave tens of those firing raw PTP events to users, it have to be coherent to the "Device" object I implemented).
As soon as this is done, I'll dig into your C code in order to understand the structure of the raw frame's header, and how you use it in you decoding function. Everything is in your sources, so I wouldn't have any problem finding it.The header is currently defined in live_view.h in my branch of the chdk source. The decoding stuff is in liveimg.c in chdkptp. Various header fields are also used in several places in the lua code, since it is responsible for figuring out the final dimensions to render.
The only things remaining are the events on the interrupt pipe I'll have to encapsulate somehow...Note that for CHDK, you don't need most of this. We don't use events (don't know how to do them on the camera end), and we don't rely on any non-CHDK commands outside of those needed to set up a session. Having the standard PTP could be useful of course, just saying they aren't required.
Oh, I've just stumbled on this article about Bayer mosaics and pixel shaders.
It's very relevant to what live view requires (and what we're looking for). Here :
http://openendedgroup.com/field/wiki/GigECamerasAndField (http://openendedgroup.com/field/wiki/GigECamerasAndField)
Even if you could transfer the the raw sensor data you would be lucky to get 1 frame every 5 seconds over USB.Eeeyup, but as I said somewhere in this thread, using shaders here isn't about framerate, it's for quick image data preparation, for futher use : One would appreciate from the client library to provide quick convolution filters on large stills. And GPU shaders are the faster way to achieve it nowadays.
Phil.
One would appreciate from the client library to provide quick convolution filters on large stills. And GPU shaders are the faster way to achieve it nowadays.
That's why I plan on using GLSL for this: Shaders-processed filters for big stills.
Most information is transferred every frame, because it can (on some cameras at least) change at any time. Future versions will most likely put more (possibly everything) in the per frame information.
I know this will sound dumb
One would appreciate from the client library to provide quick convolution filters on large stills. And GPU shaders are the faster way to achieve it nowadays.
That's why I plan on using GLSL for this: Shaders-processed filters for big stills.
So, your PC is connected (probably indoors) to a camera and via PTP and your code you have LiveView.
What then ?
I am just curious, is this an abstract programming excercise or does it have some practical use ?
Camera can be in Record or Playback which will affect the palette.
In Playback, depending on the display option, you may only see a small image top left, you will not see shooting info or histograms.
On newer cameras, the bitmap overlay can have various formats and the displayed image can be in various resolutions.
In movie mode, on my A620 at least, the displayed image is just a distorted part of the full image.
Even if you could transfer the the raw sensor data you would be lucky to get 1 frame every 5 seconds over USB.Eeeyup, but as I said somewhere in this thread, using shaders here isn't about framerate, it's for quick image data preparation, for futher use : One would appreciate from the client library to provide quick convolution filters on large stills. And GPU shaders are the faster way to achieve it nowadays.
Phil.
That's why I plan on using GLSL for this: Shaders-processed filters for big stills. Using it to decode Bayer mosaic frames or translate YUV to RGB bitmaps is just gravy. But I love gravy. :P
By the way, since OpenGL is already usable to present the live view frames in CHDKPTP, having some minds to dig into the GLSL option isn't such a bad idea.
I know this will sound dumb (well, I am !), but I'm wondering... what's actually changing ? :blink:The palette changes. Note that this applies to the bitmap (ui overlay) which is separate from the viewfinder live view.
I know pretty well what the words "live view" or "liveview" (whatever) refers to,This is not at all apparent from you posts. If you understood our usage of live view, you would know bayer is completely irrelevant to it.
since I've been studying reyalp's DE trunk for about a month now."reyalps DE trunk" ? I honestly don't understand what you are referring to. There are two svns relevant to my live view work:
This dll is made to manipulate BITMAP BUFFERS (whatever the color space is, it's up the programmer to know what he have to decode).In CHDK "bitmap" refers specifically to the paletted UI overlay portion of the camera display.
Okay, so, the good news are : device as old as the A430 HAVE (yay !) all the events implemented to declare the status changes than can/may alter the frame properties I'll bet my right hand on the fact that Canon had those implemented on every devices they release, at least since the A430.You appear to be deeply confused. I would strongly suggest making an effort to understand how the existing system works.
Thus :
if play/rec mode changes -> DeviceInfoChanged or DevicePropChanged will certainly rise.Since many of the cameras do not support remote shooting in the canon firmware, there is no reason to expect them to send events when shooting related properties change, and even if they do, I would bet they don't send the information we'd need to correctly interpret the viewport data. We don't need to know the camera changed from play to rec (we already have that information), we need to know how the viewport dimensions changed. Since this behavior varies by camera model, just knowing the mode changed is not useful.
video recording mode -> this may prove difficult if there is special tricks to it. Sending it over PTP may affect the movie encoding process on device relying on their Digic to do the USB handling.Again, I was talking *specifically* about the behavior of the viewport when the camera is in movie mode. Example: on a540, which you switch to 640x480 video mode, the viewport buffer becomes 704x528 (vs the normal 704x240).
Digital zoom -> nasty thing... But I'm pretty confident the zoom value is one the the vendor-specific property, so DevicePropChanged should appear.Even if it does, this won't tell us how many lines the current viewport buffer has.
Camera aspect ratio setting -> same as #3.As above.
stitch mode ? You mean panoramas ? Aren't those supposed to be handled as associated objects in the PTP specifications ?Again, all I'm talking about is the behavior of the camera viewport buffer. When the camera is in stitch mode, the viewport buffer becomes smaller. This has absolutely nothing to do with anything in the PTP specification.
About the events : of course they're not supposed to give you actual informations about the viewport, since you hacked your way to its buffer, but you can expect actual changes on the viewport buffer when a property change is reported by the device.What use is this when you have to get and transfer the actual information anyway ? Sending a few extra ints with every 200k+ frame isn't going to be significant, and it makes things very simple. Plus the code to do it this way is pretty mostly already written and working on both the camera and client.
About the events : of course they're not supposed to give you actual informations about the viewport, since you hacked your way to its buffer, but you can expect actual changes on the viewport buffer when a property change is reported by the device.What use is this when you have to get and transfer the actual information anyway ? Sending a few extra ints with every 200k+ frame isn't going to be significant, and it makes things very simple. Plus the code to do it this way is pretty mostly already written and working on both the camera and client.
hello everyone,Assuming you are talking about CHDK live view over PTP, it should be the same as whatever the LCD display does. If it's not, then it means Canon is doing something different with the display in different modes, and someone would need to figure out what that was.
i have question about live view.
i am using SX 130 (also a few 120) with the live view.
if the camera is in manual mode, the live view does not reflect or resemble the final picture which will be taken. is it possible for the live view to approximate the final picture. eg if exposure is increased the live view becomming brighter (because the final picture will be brighter)
for the moment i am keeping the camera in Program mode - even when my PC software says Manual - only when the PC requires a shot i switch the camera to Manual, set the parameters, take the shot, download pic and switch the camera back to Program mode.
in Manual mode the Live View is often very dark(almost black)
am i doing it right or is there a better way
hello everyone,Assuming you are talking about CHDK live view over PTP, it should be the same as whatever the LCD display does. If it's not, then it means Canon is doing something different with the display in different modes, and someone would need to figure out what that was.
i have question about live view.
i am using SX 130 (also a few 120) with the live view.
if the camera is in manual mode, the live view does not reflect or resemble the final picture which will be taken. is it possible for the live view to approximate the final picture. eg if exposure is increased the live view becomming brighter (because the final picture will be brighter)
for the moment i am keeping the camera in Program mode - even when my PC software says Manual - only when the PC requires a shot i switch the camera to Manual, set the parameters, take the shot, download pic and switch the camera back to Program mode.
in Manual mode the Live View is often very dark(almost black)
am i doing it right or is there a better way
Some unfinished work in progress on the live view protocol, mostly for discussion on IRC
- gets rid of get_handler
- gets rid of lv_base_info / lv_vid_info distinction
- generic description for frame buffers
This doesn't currently implement the different aspect ratios on newer cams, which would theoretically go into visible_buffer_*offset
The old protocol is still there. The chdkptp patch won't talk correctly to previous builds, and lvdump record/playback isn't implemented
What I've called the "logical" screen is meant to represent the physical screen on the camera (I'm open to better names!) in "buffer dimensions" In other words, the logical height is how tall the buffer would be if it filled the whole screen. In the old implementation, this was essentially vid_get_viewport_max_height and vid_get_viewport_max_width, but they can actually change in various cases.
On cameras like the g12, it looks like the height of a full screen viewport is always 480, but on a540 it can be 528 (640x480 video mode) 240 (playback, most capture modes), or down to ~60 depending on digital zoom. On D10, max digital zoom is also drops to half width.
I'm still struggling to find a way represent this in a way that is convenient and doesn't require a ridiculous number of new lib.c functions.
I've been struggling with trying to come up with a good model for this all week - one thing I'm unclear on is how does the logical viewport size relate to the OSD bitmap screen size (and ultimately the live image view) when it gets displayed on the PC?We discussed this a bit in IRC, but I'll try to elaborate a bit.
@reyalpThat's useful info, I wondered if TV out would change it. Would also be worth checking modern cameras that have HDMI too. Maybe we don't have to worry about the ones with combined USB/video (can you make a splitter ?)
Sorry for this off-topic, but you always seem to mention that the A540's viewport resolution is 704 pixels wide inside the 720px wide buffer. The cameras I've seen change their viewport width from 720px to 704 when they sense that TV-out is in use (this happens on-the-fly). Do you have something plugged into that camera? :)
Maybe we don't have to worry about the ones with combined USB/video (can you make a splitter ?)http://chdk.setepontos.com/index.php?topic=5691.msg62525#msg62525 (http://chdk.setepontos.com/index.php?topic=5691.msg62525#msg62525)
I don't have anything plugged in.Interesting, I assumed all TV-out models behave the same regarding viewport width.
It turns out that plugging in the TV out does change the height though (probably to 528), and worse, doesn't change the value I've been using to get the actual viewport height.There's a reason why the buffers follow each other at a distance of >>240 lines.
edit:
Ooops, change in viewport height is in playback mode, which I have hard coded because that value never works in playback. In rec mode, it seems to be normal with TV out connected.
After our chat yesterday I worked on fixing things for the G1X and also stitch mode.Thanks, I'll take a look.
Essentially this adds two new offset values to define the position of the viewport on the LCD/liveview display - the existing offset values now only define the position of the viewport in the memory buffer.
I'm thinking that all the viewport_???_proper() functions should be replaced by a single function that populates the live view data structure with the correct values. These are only used in the live view code.I was thinking about that too, but having a bunch of really simple functions that can be overridden individually may be better. Not sure...
After our chat yesterday I worked on fixing things for the G1X and also stitch mode.Thanks, I'll take a look.
Essentially this adds two new offset values to define the position of the viewport on the LCD/liveview display - the existing offset values now only define the position of the viewport in the memory buffer.
However I cant find how do I see the live preview on the PC ! Hope someone can help !The live preview is still not integrated into the main CHDK. Sorry :(
Let me understand: you do patch the camera firmware or the external ptpcam dll ? Is the ptp livepreview a binary compiled thing or some kind of LUA hack to provide realtime buffers ?Liveview support requires code in CHDK on the camera. It also requires a client that understands the protocol.
In the livepreview do we get the viewfinder images or also the menus and everything on the LCD screen ?The UI (aka bitmap) and viewfinder (aka viewport) are separate framebuffers on the camera. Both are available in the current work-in-progress PTP protocol, although support is not complete for every camera. See http://chdk.wikia.com/wiki/Frame_buffers (http://chdk.wikia.com/wiki/Frame_buffers) for information about the various camera frame buffers.
Also, how do I check out latest version and am I able to compile with mingw ?For the client, see http://www.assembla.com/wiki/show/chdkptp (http://www.assembla.com/wiki/show/chdkptp) - The readme explains the prerequisites. mingw is the preferred toolchain for windows.
Finally, I would like to create a python wrapper and make this work both on windows and linux via libusb. I can use some functions to convert from yuv to rgb et.al so if I am able to access the raw buffers it would be nice.That should be possible. If you want to use libusb directly, you will need to make your own implementation of the CHDK PTP protocol. OTOH, chdkptp is cross platform and already has a Lua API (in fact, it's mostly written in Lua, the C code only provides some low level APIs), so you might just want to use lua. It includes code to convert the framebuffers to RGB.
After I start chdkptp, I turn on my camera (in play mode) and hit the connect button.reyalp will be better able to help you, but what happens if you start your camera before you start chdkptp ?
reyalp will be better able to help you, but what happens if you start your camera before you start chdkptp ?The same thing really. I don't see any difference trying it either way. Not sure what else I could try.
I've been reading this thread all day today hoping I could find someone who experienced the problem I am having. I can't seem to get chdkptp to connect for me. I've a SX130IS with firmware 1.01f. I've downloaded reyalp-ptp-live1871 and successfully built that within CHDK-Shell and installed it on my camera. I'm using chdkptp r230-win32 as my client.You will need to use the latest chdkptp from svn if you want to use the latest CHDK code from my branch. The reason it's in a private branch is so I don't have to worry about compatibility from version to version.
Wow, thanks! That made all the difference in the world! I don't have much time to play with it this morning but most functions I've tried seem to be working. Livevview doesn't though. I get this error: "WARNING: camera live view protocol not supported by this client, live view disabled" I may try building 1817 as you suggested and see how that goes. Thanks again!If you use the chdkptp r248 snapshot, then you need to use chdk 1873 (not the 1871 you built earlier, sorry) or later from my CHDK branch.
If there are no objections, I'm going to commit the live view stuff in the trunk tomorrow.I'm excited !!! Are the host clients ready ?
Are the host clients ready ?The most recent snapshot of chdkptp should work.
Checked in trunk changeset 1921 (http://trac.assembla.com/chdk/changeset/1921)Huge round of applause - nice job :)
The following ports should have pretty much full support:The SX30 works as well.
a540, d10 g12, g1x, sx40hs, g10, a590, ixus310_elph500hs
The sig finder finds bitmap and palette stuff for recent dryos. It would be worthwhile to do the same for the older versions.Would be good; but I wasn't able to see any usable patterns for finding this stuff in older DryOS versions.
Would be good; but I wasn't able to see any usable patterns for finding this stuff in older DryOS versions.I think the ones I'm using on d10 shouldn't be too hard (especially the bitmap one), but when I was looking at it, I though I found code equivalent code on newer cameras that gave different addresses from what your method is using (or found code equivalent to the new cameras on d10, I don't remember exactly.) I haven't had a chance to look into it further yet.
Checked in trunk changeset 1921 (http://trac.assembla.com/chdk/changeset/1921)
The following ports should have pretty much full support:
a540, d10 g12, g1x, sx40hs, g10, a590, ixus310_elph500hs
a720 & sx220 work well.Thanks for the report.
There is a little problem with ptpCamGui. The Gui don't know the new PTP version 2.3 and will not start. Not a big thing. We must add one line. But rudi and me are in vacation. When you need ptpCamGui, use the stable CHDK version. In some days we will update the ptpCamGui.Minor versions (like 2.0 to 2.3) are supposed to be backward compatible, so it should be safe for ptpCamGui to start if the minor version is greater than what it knows about.
Here's a weird one for you.I've reproduced this on a540 and d10. As expected, it also happens with ptpcam.
If I use chdkptp to upload files to the camera (in this case the compiled module files), and the module file length is 5592 bytes, and the filename+path is 24 bytes then the PTP upload crashes the PTP code in the camera. The total data size sent is 5592+24+4 = 5620 bytes.
One byte shorter or longer on the filename and it works fine.
The PTP code gets the correct data_size value but gets the wrong value for fn_len (gets 8 instead of 24).
I've tried using umalloc instead of malloc in the PTP code; but the result is the same.
I've tested this on the G12, SX40 and G1X all give the same result.
Any ideas?
con> !return con:exec('msg_shell:run()',{libs='msg_shell'})
=true
con> putm exec msg_shell.default_cmd=function(msg) write_usb_msg(msg) end
con 1> !con:write_msg('\24\0\0\0'..string.rep('x',5592+24))
... hangs ...
unexpected return code 0x2ff
con 1> !chdk.reset_device({dev="\\\\.\\libusb0-0002--0x04a9-0x311b",bus="bus-0"})
reset_device: dev \\.\libusb0-0002--0x04a9-0x311b bus bus-0
Device status OK
___> c
con> getm
... returns message, intact ??? ...
con> !con:write_msg('\24\0\0\0'..string.rep('x',5592+25))
... no hang ...
con> getm
... returns message...
con> !con:write_msg('\24\0\0\0'..string.rep('x',5592+23))
... ok ..
con> !con:write_msg('\24\0\0\0'..string.rep('x',5592+24))
... hangs ...
:blink:chdkptp -c -i
!m=require'msgtest'
!m.load()
!m.test(<start lenght>,<optional max length>,<optional increment>)
However, I wrote a script that sends messages from with every length from 1 to N and that seems to be crashing in a different way.Just a quick test: problematic lengths on the A410: 0x74, 0xb4, 0xf4, 0x134 (and so on, I guess). The assert (same as above) comes on the next ptp attempt.
Minor versions (like 2.0 to 2.3) are supposed to be backward compatible, so it should be safe for ptpCamGui to start if the minor version is greater than what it knows about.
On D10, I narrowed down where the error is returned from in data->recv_data (sub_FF9FA258) to the bne at FF9FA37C. The hang seems to happen in the preceding call to sub_FF869A1C__KerFlag_c (some kind of synchronization object timing out, I think)
Some of the preceding code seems to be related to the unknown parameter to recv_data that we always set to zero.I tried putting in a 1, but that didn't help.
edit:
testing error, setting the unknown param to 1 crashes very promptly when you try to use ptp. No surprise there ;)
Out of curiosity, does the problem still happen using the Rasberry PI version you posted about in another thread?It does the same thing from the pi. Not sure it's really any different than the linux laptop I already tested though...
If so that would pretty much make it a firmware problem, as your research seems to indicate. Or at least a problem in the way CHDK is using the firmware.
Other ideas
http://biot.com/blog/usb-sniffing-on-linux (http://biot.com/blog/usb-sniffing-on-linux)
http://lxr.linux.no/#linux+v2.6.28.8/Documentation/usb/usbmon.txt (http://lxr.linux.no/#linux+v2.6.28.8/Documentation/usb/usbmon.txt)
edit:
http://wiki.wireshark.org/CaptureSetup/USB (http://wiki.wireshark.org/CaptureSetup/USB)
uint16_t
ptp_usb_senddata (PTPParams* params, PTPContainer* ptp,
unsigned char *data, unsigned int size)
{
static uint16_t ret;
static PTPUSBBulkContainer usbdata;
/* build appropriate USB container */
usbdata.length=htod32(PTP_USB_BULK_HDR_LEN+size);
usbdata.type=htod16(PTP_USB_CONTAINER_DATA);
usbdata.code=htod16(ptp->Code);
usbdata.trans_id=htod32(ptp->Transaction_ID);
memcpy(usbdata.payload.data,data,
(size<PTP_USB_BULK_PAYLOAD_LEN)?size:PTP_USB_BULK_PAYLOAD_LEN);
/* send first part of data */
ret=params->write_func((unsigned char *)&usbdata, PTP_USB_BULK_HDR_LEN+
((size<PTP_USB_BULK_PAYLOAD_LEN)?size:PTP_USB_BULK_PAYLOAD_LEN),
params->data);
if (ret!=PTP_RC_OK) {
ret = PTP_ERROR_IO;
/* ptp_error (params,
"PTP: request code 0x%04x sending data error 0x%04x",
ptp->Code,ret);*/
return ret;
}
if (size > PTP_USB_BULK_PAYLOAD_LEN)
{
/* if everything OK send the rest */
ret=params->write_func (data+PTP_USB_BULK_PAYLOAD_LEN,
size-PTP_USB_BULK_PAYLOAD_LEN, params->data);
if (ret!=PTP_RC_OK) {
ret = PTP_ERROR_IO;
/* ptp_error (params,
"PTP: request code 0x%04x sending data error 0x%04x",
ptp->Code,ret); */
}
}
if ((usbdata.length & 0x1FF) == 0)
params->write_func(data, 0, params->data);
return ret;
}
Found it :)
While researching how USB PTP transfers work I came across this:
A bulk transfer is considered complete when it has transferred the exact amount of data requested, transferred a packet less than the maximum endpoint size, or transferred a zero-length packet.
When the chdkptp code sends a file/message that is exactly a multiple of the maximum end point size (512) the camera is still waiting for the bulk transfer to be completed. In this case it is necessary to send another 0 length packet to terminate the transfer.
Found it :)Excellent :)
While researching how USB PTP transfers work I came across this:
A bulk transfer is considered complete when it has transferred the exact amount of data requested, transferred a packet less than the maximum endpoint size, or transferred a zero-length packet.
Found it :)
While researching how USB PTP transfers work I came across this:
A bulk transfer is considered complete when it has transferred the exact amount of data requested, transferred a packet less than the maximum endpoint size, or transferred a zero-length packet.
When the chdkptp code sends a file/message that is exactly a multiple of the maximum end point size (512) the camera is still waiting for the bulk transfer to be completed. In this case it is necessary to send another 0 length packet to terminate the transfer.
Nice find. :)
I've tested it with 0x3f as the magic value (still testing with a USB1.1 (or full speed ?) camera), and it really fixes the issue in question. (I now get "unexpected msg type" sporadically while running reyalp's msgtest)
Am I correct that the current implementation assumes "high speed" connection? I've found a define named PTP_USB_BULK_HS_MAX_PACKET_LEN in ptp.h . According to my web search this originates from some libgphoto related code(?). I've also found PTP_USB_BULK_FS_MAX_PACKET_LEN on the web in a project named arducam-osd which assumes full speed instead (which is understandable since it's Arduino-related).
I guess the connection speed could be detected somehow, and that magic value could be based on the connection type. I do not know the libusb code enough to tell how to do that...
Attached is a patch to the current chdkptp trunk that attempts to get the maximum endpoint size from the libusb data and use that to determine if the dummy packet needs to be sent.The recognition of the max packet size works well for the two "full speed" cameras I tried.
Can you try this with your camera and see what happens.
I have no idea if this is the right way to do this sort if thing - I'm still trying to understand the whole USB PTP interface stuff.Looks good enough to me.
I don't see a live view function
or a way to control multiple cameras
I'd like to start implementing remote capture with remote target (i.e. PC). Some months ago I did some research, as you may rememberQuestion: Is this actually worthwhile compared to letting the camera save to disk and downloading the file ? It could be faster, but what is the use case where saving to disk is too slow, and this isn't ?
What I don't know is the PTP part of all this, and how should scripting be involved. Do you have an idea?To capture raw (and presumably the other options) you will need to do something like the how the raw hook in capt_seq blocks and waits for spytask. Since PTP transfers can only be initiated by the PC side, I think it will have to go like this (for raw, if you hook some other place the sequence should be similar):
Question: Is this actually worthwhile compared to letting the camera save to disk and downloading the file ? It could be faster, but what is the use case where saving to disk is too slow, and this isn't ?For me, the current remote capture solution feels like a workaround (it is a workaround, actually). Canon's "remote control" capability always included (AFAIK) a shooting option where the destination of the capture is the "PC". I would like to restore this functionality. My other concern is: why write to the card (causing wear) when the pictures should end up on the connected host anyway? And yes, it can be a little faster.
Practically, the overhead of individual send calls is such that you'd probably send entire lines.I don't quite get what you mean here.
Should the transferred data include meta data ? If so, how much in what form ? For raw, it might make sense to send the whole DNG exif ?It could be an option to send metadata or not. For JPEG, exif is available at the same time (it's stored in a separate buffer and is intended to be written first), for raw, probably the DNG exif-generator routine should be reused somehow.
If you want to send the whole buffer, you can do it with one call to data->send_data. Same if your subimage is full width but reduced height: you just adjust the start and total size. Live view does this for modes where the viewport isn't full height (e.g. 16:9 on a 4:5 screen)QuotePractically, the overhead of individual send calls is such that you'd probably send entire lines.I don't quite get what you mean here.
It could be an option to send metadata or not. For JPEG, exif is available at the same time (it's stored in a separate buffer and is intended to be written first), for raw, probably the DNG exif-generator routine should be reused somehow.Questions to think about:
If the subimage is narrower than the full image you have to call send_data for each line. I tried this for live view, because some cameras have unused borders (e.g. stitch window) but it was significantly slower than just sending the whole buffer.OK, now I get it. Don't break up data into tiny pieces because the USB DMA doesn't like it.
If you need some data, is it really worthwhile to make the rest optional ? With a 20mb raw, or even a 1mb jpeg, adding a few kb of exif isn't going to have a significant impact. It may be better to send it all the time and let the client discard it if they don't care.Good point.
to get the job doneJob?
I probably won't get a chance to look at it in detail before the the weekend.No problem. There is no hurry.
the final version should do raw for all cams, and the ifdef should just apply to the parts that require the filewritetask hook.Good idea.
This implies some way of checking which are available, preferably without trying to take a shot.Perhaps it could be part of some sort of "capabilities" request.
- the chdkptp side should expose the individual steps to lua. I can certainly understand doing it the way you are now for testing though, and assuming the CHDK code gets added, I can work on that part.Well, I had the feeling that the huge ptp_chdk_remoteshoot() in ptp.c is not really appropriate :)
but I'm concerned about the chdkptp-side implementation. Especially now that there's a brand new shoot command, I'm not sure how mine should be implemented.As long as the individual steps are available to lua (which as I said, I can add) and a scripted shoot will honor the remote capture settings, this doesn't seem like it should be much of a problem.
As long as the individual steps are available to lua (which as I said, I can add)OK :)
and a scripted shoot will honor the remote capture settings, this doesn't seem like it should be much of a problem.You mean setting file type(s), crop parameters independently of initing / uniniting the remote file target mode?
The client will have to know not to use a blocking call for a shoot script if it expects to get remote data.
Would it be useful to be able to set the remote capture parameters in lua, (equivalent to PTP_CHDK_RemoteCaptureInit) or is this redundant ? I'm not sure...
From an efficiency point of view, it would be preferable to add the remote capture data available flag to PTP_CHDK_ScriptStatus rather than giving it a unique command (PTP_CHDK_RemoteCaptureIsReady). That way, the existing wait_status command in chdkptp would be extended to work on that value, without the overhead of polling multiple different commands.OK, but the return value of PTP_CHDK_RemoteCaptureIsReady will need to be placed somewhere (param2?)
This seems like the way to go to me. (Ultimately, there should be a way to combine all the status bits with the live view polling, but that's another project)Guess you mean something like you did with ptp_chdk_download(). I'll leave that to you :D
How chdkptp side Lua should deal with getting the individual chunks so something of a question, since in some cases you'd want to write directly to disk (for low memory systems, you might want to not even store a whole raw) and in others you might want the data as an lbuf in Lua.
Are these steps similar to what those (unfinished/untested) functions in ptp.c (in the v2 patch) are supposed to do? (ptp_chdk_rcinit, ptp_chdk_rcisready, ptp_chdk_rcgetname, ptp_chdk_rcgetfile)Yes, that's what I had in mind. I still haven't had a chance to really look at it, but next weekend is looking good.
You mean setting file type(s), crop parameters independently of initing / uniniting the remote file target mode?Yes.
I've been thinking about a use case where the camera runs a motion detection script, and the client fetches the pictures (continuously) as they become available. In this case the camera would stay in remote file target mode for a while. Maybe this could be considered.Yes, it seems like this might be more convenient under script control. Question:
OK, but the return value of PTP_CHDK_RemoteCaptureIsReady will need to be placed somewhere (param2?)We have quite a few params and bits free. I also realized after posting that this would make it no longer really "script" status :-[
Also, for cameras with partial (RAW only) support, maybe the SDM method is the way to go (I haven't looked into that code, but I seem to recall that the RAW data is transferred over PTP and the JPEG - which is small - gets erased instantly). I think SDM fills the RAW buffer with some homogeneous value (0?) to achieve a really small JPEG.I haven't looked at SDM, but if we go this way, it seems like it should be something you can set for script, could be useful in other situations beyond remote camera. e.g nuke_next_jpeg()
I'm sorry, but I don't follow. What does "both" mean here? If it's about the format + cropping settings, I think a single method of setting them would be preferred.QuoteI've been thinking about a use case where the camera runs a motion detection script, and the client fetches the pictures (continuously) as they become available. In this case the camera would stay in remote file target mode for a while. Maybe this could be considered.Yes, it seems like this might be more convenient under script control. Question:
Should we allow both (could lead to odd situation if client uses both, also code complexity) or require that it be done in lua ? I'd be tempted to only do it in lua if we are going to allow lua at all, so PTP_CHDK_RemoteCaptureInit would be replaced by a lua function.
Since shooting requires sending lua anyway, it doesn't seem like this would be a big handicap. If anything would make the client simpler.
I wonder if it would make more sense to have this be a flag (remote capture chunk is available), and make PTP_CHDK_RemoteCaptureGetData just return the next chunk along with an ID of what it is ?I'll think about this. It would require new logic on the client side too.
A small step from here is to keep the full jpeg and transfer/erase it on demand :/ This would essentially do what current scripted solutions do. The number of possible options keeps growing...QuoteAlso, for cameras with partial (RAW only) support, maybe the SDM method is the way to go (I haven't looked into that code, but I seem to recall that the RAW data is transferred over PTP and the JPEG - which is small - gets erased instantly). I think SDM fills the RAW buffer with some homogeneous value (0?) to achieve a really small JPEG.I haven't looked at SDM, but if we go this way, it seems like it should be something you can set for script, could be useful in other situations beyond remote camera. e.g nuke_next_jpeg()
In some situations, keeping the all black jpeg could be useful, because it would still have the exif
Regarding development:I'm not against it. Tried my v2 patch, it still applies to current trunk code without change.
I understand you didn't want to use the trunk because it's moving too fast. What would you think about having this temporarily go in a branch of the current trunk ? I'd prefer to have it in svn somewhere to play with, and it's not going to go into the stable branch in the end.
I'm sorry, but I don't follow. What does "both" mean here? If it's about the format + cropping settings, I think a single method of setting them would be preferred.Yes, by "both" I meant the current PTP_CHDK_RemoteCaptureInit + an equivalent script function. And I agree a single method seems better, and the more I think about it, I think script is the better option.
This was just thinking out loud, maybe it doesn't make sense don't put a lot of work into it.QuoteI wonder if it would make more sense to have this be a flag (remote capture chunk is available), and make PTP_CHDK_RemoteCaptureGetData just return the next chunk along with an ID of what it is ?I'll think about this. It would require new logic on the client side too.
A small step from here is to keep the full jpeg and transfer/erase it on demand :/ This would essentially do what current scripted solutions do. The number of possible options keeps growing...If they keep the full file, then I don't think there's any need to use remote capture. They can download it as a file and decide what to do. Deleting can be done in script already.
OK, I think I'll do this.QuoteRegarding development:...I'm not against it. Tried my v2 patch, it still applies to current trunk code without change.
Do you plan to take over the chdkptp part?I could put it under an ifdef and do some work on it at least.
I took out the a410 hack for now, but I would like to make chdkptp do the right thing for these cameras at some point.I was too lazy to separate that.
Remote shoot with raw on a540 and d10 work. Transfer for full raw is reasonably quick, the entire rs command took 2 sec on a540, 3.8 sec on d10 (this includes pre-focus, shooting etc in addition to transfer)What do you think about the jpeg? Should it be minimized / blacked out?
This is a good question. I think it's OK to have in script though:I took out the a410 hack for now, but I would like to make chdkptp do the right thing for these cameras at some point.I was too lazy to separate that.
About replacing PTP_CHDK_RemoteCaptureInit with script command(s):
Can this be bulletproof? Currently, PTP_CHDK_RemoteCaptureInit is independent of scripting, so it can be issued any time. I'm asking because of its "uninit" functionality.
If script execution breaks for some reason and the camera keeps reporting "script is already running", can an emergency uninitialization be executed? If not, hooks will continue to be blocked, until they time out (which is not currently implemented).
What do you think about the jpeg? Should it be minimized / blacked out?Maybe nice to have, but I don't think it's a high priority.
Most of the stuff add to generic/capt_seq.c should be in thumb code somewhere.I added the code to generic/capt_seq.c because that seemed to be somewhat related. Since the thumb-arm barrier in CHDK is not entirely clear to me, is there something I should watch out for when moving functions into thumb code?
It might make more sense to do most of the stuff in capt_seq_hook_raw_here() within the regular raw hook (or the same spytask loop where the raw hook is called)
get_next_chunk_data_for_ptp also looks like it doesn't need to be in capt_seq, but not sure where it goes.
The main point is thumb code is smaller, so anything that's not tightly tied to the ARM asm could should usually be in thumb. If you are using a __attribute__((naked)) function to save and restore registers, that should be in arm.Most of the stuff add to generic/capt_seq.c should be in thumb code somewhere.I added the code to generic/capt_seq.c because that seemed to be somewhat related. Since the thumb-arm barrier in CHDK is not entirely clear to me, is there something I should watch out for when moving functions into thumb code?
It might make more sense to do most of the stuff in capt_seq_hook_raw_here() within the regular raw hook (or the same spytask loop where the raw hook is called)
get_next_chunk_data_for_ptp also looks like it doesn't need to be in capt_seq, but not sure where it goes.
note if you use ASM_SAFE in the call (or don't need to save extra registers), then you can go directly to the thumb function, e.g. core_spytask_can_start is a thumb function and most dryos cams call it with justI think that's what I wanted to know.
"BL core_spytask_can_start" in boot.c, and the toolchain converts it to core_spytask_can_start_from_arm
Yeah, I think sending an extra arg is fine. If I remember right we already send deta->recv_data more than (some ?) firmwares actually use.note if you use ASM_SAFE in the call (or don't need to save extra registers), then you can go directly to the thumb function, e.g. core_spytask_can_start is a thumb function and most dryos cams call it with justI think that's what I wanted to know.
"BL core_spytask_can_start" in boot.c, and the toolchain converts it to core_spytask_can_start_from_arm
Something (not) completely unrelated:
How should I handle this (http://chdk.setepontos.com/index.php?topic=8600.msg90038#msg90038) situation? Two of the earliest cameras are affected (I plan to release my Ixus30/SD200 port soon). send_resp() has only 2 parameters in every other camera. Would it be better to create a new #define in camera.h for this, or can I just do what I did here (http://chdk.setepontos.com/index.php?topic=8600.msg90081#msg90081)? That would cause a 2 or 4 byte size increase in every port, but - as I understand - would be safe otherwise.
Regarding remote capture, I was thinking about putting the init stuff in lua, but I don't want to duplicate effort.If you have time to do it, just go ahead.
I'd probably put the remote capture related stuff in it's own file (core/ remote_capture.c or something, some of the stuff in the main ptp.c could probably go there too). Without that, capt_seq_hook_raw_here is very small. capt_seq_hook_set_nr needs defines from the cameras capt_seq.cRegarding remote capture, I was thinking about putting the init stuff in lua, but I don't want to duplicate effort.If you have time to do it, just go ahead.
How about moving most of generic/capt_seq.c into (say) core/hooks.c?
I started to move code from generic/capt_seq.c to core/remote_capture.c in a temporary repoI started doing something similar when I tried to put the init stuff in Lua cleanly, ran into similar issues and ran out of time before I got to the point of having anything to check in.
( https://subversion.assembla.com/svn/chdk-s1.remotecapture/ ).
I changed some from static to volatile (to be able to access them from multiple source modules), and I created "setter" and "getter" functions for some others.I'd say getters/setters is probably cleaner, neither one is really safe to communicate across tasks, but for simple state variables it should be OK. Overall, the current code (in remote-capture-test) is quite convoluted with the different globals and statics. I think it can be made much more modular, but it's a bit tricky.
I'm not completely sure about naming the new file "remote_capture.c", because the filewritetask hook could possibly be used for more.Agreed. The direction I was going was to have minimal filewrite_main_hook in capt_seq.c which currently just called into the remote capture code, but could have other stuff later.
To do the separation, I had to move the declaration of the struct "cam_ptp_data_chunk" into platform_camera.h. As platform_camera.h only holds #define's in current CHDK code, this looks pretty ugly to me.If there is only one known variant, it might be better to just start out assuming it's always that way, and deal with it later if it turns out not to be. If there are only a few variants, they are likely to correspond to DryOS revs (old vxworks like a410 and more recent like a540 seem to be the same), so you could just have a few different ones using existing DryOS ifdefs.
"cam_ptp_data_chunk" may turn out to be the same in a large number of camera generations, but this is only a suspicion.
My later plan includes using the dng module to get the CHDK exif (optionally).I'm not sure it's even worth making this optional, if the client doesn't want it they can just throw it out.
I'd say getters/setters is probably cleaner, neither one is really safe to communicate across tasks
I'd say getters/setters is probably cleaner, neither one is really safe to communicate across tasks, but for simple state variables it should be OK.What I don't like about the getters/setters is the overhead (especially when the variable is shared between arm and thumb code).
Overall, the current code (in remote-capture-test) is quite convoluted with the different globals and statics. I think it can be made much more modular, but it's a bit tricky.I know ...
- code is ugly and imperfect (suggestions / improvements are welcome)
In my code, started putting in a function to copy from the platform defined one into the generic one in generic/capt_seq.cHow does it look like?
I'm not sure it's even worth making this optional, if the client doesn't want it they can just throw it out.I assumed that if somebody needs a subrange of the RAW, a header may not be needed anyway. Or do you think the output should be valid DNG? If so, the byteswapping should probably be done in the client.
On question does arise, if you do a sub-image, what dimensions go in the header ?
Another thing I noticed trying to work on the init stuff is that startline, numlines could have different possible values for raw vs YUV (raw buffer being a bit bigger than max size jpeg, and jpeg potentially being lower res.)It should be available. Otherwise the camera would need to decode the jpeg again to provide the recreview. I think I'm going to dump the RAM in the FileWriteTask hook to look for the buffer, its pointers and the image dimensions. Then look them up in the firmware.
Have you found / verified that a YUV buffer is actually available ? If not, maybe V1 should just be jpeg and raw, and not worry about any of the YUV stuff for now.
Why is that ?Which one ?
What I don't like about the getters/setters is the overhead (especially when the variable is shared between arm and thumb code).
How does it look like?Unfinished ;)
void filewrite_get_jpeg_chunk(remotecap_data_chunk *dest,void *src,int n) {
cam_ptp_data_chunk *pdc = src;
dst[n].address=pdc[n].address;
dst[n].length=pdc[n].length;
}
void filewrite_main_hook(char *name, cam_ptp_data_chunk *pdc)
{
remotecap_jpeg_available(name,MAX_CHUNKS_FOR_JPG,pdc);
...
but after I wrote that it seemed like it might be better to just leave the loop in filewrite_main_hook.I assumed that if somebody needs a subrange of the RAW, a header may not be needed anyway. Or do you think the output should be valid DNG? If so, the byteswapping should probably be done in the client.I agree byteswapping should be left to the client. For subframe, I think having the exif could be useful, but we could just send the full-frame exif and let the client adjust it or pad the missing data to the expected size.
Some preliminary trials show that the YUV source is indeed available during "recreview". Its pixel format is UYVY in A410 (and probably in every other camera too). Getting it correctly may requireQuoteHave you found / verified that a YUV buffer is actually available ? If not, maybe V1 should just be jpeg and raw, and not worry about any of the YUV stuff for now.It should be available. Otherwise the camera would need to decode the jpeg again to provide the recreview. I think I'm going to dump the RAM in the FileWriteTask hook to look for the buffer, its pointers and the image dimensions. Then look them up in the firmware.
The UYVY image seems to have only one use: source for the compression engine. Activating the record review (by pressing SET) causes immediate decoding of the JPEG to another buffer with y411 as pixel format. I see no point in getting that.So if I understand right, on this camera (a410 ?) the full res YUV buffer seems to be needed only for reduced size jpeg, otherwise the image processor does the raw->jpeg all in one step ? This would match well with the observation that reduced res jpeg is slower than full res on some (mostly older, digic II maybe digic III) cameras.
I started to move code from generic/capt_seq.c to core/remote_capture.c in a temporary repoI'm not really sure how to coordinate work with this branch and the one main CHDK svn repo.
( https://subversion.assembla.com/svn/chdk-s1.remotecapture/ ).
So if I understand right, on this camera (a410 ?) the full res YUV buffer seems to be needed only for reduced size jpeg, otherwise the image processor does the raw->jpeg all in one step ?It appears to be so.
I suppose a possible alternative is that the full res case does create a YUV buffer, but it's destroyed before the point you are making the dump ?I would think that's unlikely. At least I can't see any sign of that buffer. The RAW buffer seems to still contain the RAW data, and of course there is the JPEG data.
As an aside, the normal viewport buffer addresses don't seem to work in recreview mode. I'm not sure if this is true for all cameras, it is for d10 and a540. These cameras both have vid_get_viewport_live_fb, it's possible the correct address is returned by one of the other existing functions (I think someone tested this and found it wasn't but I'm not 100% sure.) If you find a reliable way to get it, we could make recreview work in ptp live view, which would be nice. Showing chdk zebra/histogram in recreview could also be useful.By looking at some dumps, there is an active viewport buffer (still talking about review mode right after shooting), vid_get_viewport_fb_d() points to it. In another dump, which was created after shooting AND pressing SET (recreview hold), vid_get_viewport_fb_d() still seems to be valid.
QuoteI started to move code from generic/capt_seq.c to core/remote_capture.c in a temporary repoI'm not really sure how to coordinate work with this branch and the one main CHDK svn repo.
( https://subversion.assembla.com/svn/chdk-s1.remotecapture/ ).
Regarding remote capture, I was thinking about putting the init stuff in lua, but I don't want to duplicate effort.and I didn't want to clash with that. Moreover, my code still sucks (not optimal, needs cleanup).
By looking at some dumps, there is an active viewport buffer (still talking about review mode right after shooting), vid_get_viewport_fb_d() points to it. In another dump, which was created after shooting AND pressing SET (recreview hold), vid_get_viewport_fb_d() still seems to be valid.OK, so we could check recreview in vid_get_viewport_active_buffer. I'll try that and report back.
Take a look at that repo. If you find that the code I committed there should go into ptp-remote-capture-test, then I'll update that repo too. I started this because you mentionedYes, I that confused me a bit. A branch in the same repo makes it easier to compare and move changes around, and doesn't really cost any space.QuoteRegarding remote capture, I was thinking about putting the init stuff in lua, but I don't want to duplicate effort.and I didn't want to clash with that. Moreover, my code still sucks (not optimal, needs cleanup).
BTW, I used a (full, unnecessarily) dump of the main CHDK repo (rev. 2133) for mine (I'm practicing with svn).
on d10, after switching the recreview_hold variable to the sig finder value, this seems to work. However, it only works in *hold* mode, you'd need some other variable to detect a fixed review time like 10 sec.By looking at some dumps, there is an active viewport buffer (still talking about review mode right after shooting), vid_get_viewport_fb_d() points to it. In another dump, which was created after shooting AND pressing SET (recreview hold), vid_get_viewport_fb_d() still seems to be valid.OK, so we could check recreview in vid_get_viewport_active_buffer. I'll try that and report back.
Updated patch.First (quick) impression: it works, the code looks much better organized now. I copied over YUV support from my version, also works, but it may have to be modified (it will produce 0 byte long files with cameras without explicit yuv support OR when L picture size is selected). I think you can commit this.
- initial lua based init implementation. Untested, need to set up the client stuff.
- much simplified ptp.c code. Error returns may change slightly, but current rs command seems to work
- now compiles if camera doesn't have filewritetask
...True. I haven't checked that yet.
However, it only works in *hold* mode, you'd need some other variable to detect a fixed review time like 10 sec.
First (quick) impression: it works, the code looks much better organized now. I copied over YUV support from my version, also works, but it may have to be modified (it will produce 0 byte long files with cameras without explicit yuv support OR when L picture size is selected). I think you can commit this.Committed in changeset 2135. Please feel free to work in this branch.
edit: there's still some cleanup to do (removed functions' declarations)Yes, there's still quite a bit to be done.
state_shooting_progress=SHOOTING_PROGRESS_DONE; //shoot() needs this...
In remotecap_get_data_chunk... since this can be called when remotecap is reset (on error or init(0)) it would happen at random times in the shooting process. Same for hook_raw_save_complete. I'm not clear exactly what the implications would be, but it seems like there could be problems.One thing that i'm not sure about is theThe reason for this was that when I first tried my monolithic rs command, I kept getting "script is already running" afterwards. If I remember correctly, this was needed for shoot() to finish (?). I'm going to take a look at this again. Maybe it's not needed anymore or is misplaced.Code: [Select]state_shooting_progress=SHOOTING_PROGRESS_DONE; //shoot() needs this...In remotecap_get_data_chunk... since this can be called when remotecap is reset (on error or init(0)) it would happen at random times in the shooting process. Same for hook_raw_save_complete. I'm not clear exactly what the implications would be, but it seems like there could be problems.
- hook_raw_save_complete() is only called after all the chunks are downloaded, meaning the the raw hook will be blocked until the jpeg is complete. This seems like it might have unwanted impacts, perhaps depending on where the raw hook is on a specific camera.I assumed the two hooks to never overlap. If they do, the port is IMHO broken (imagine what would happen if FileWriteTask's open stage found an open file). That was the reason I used the same variable and status codes for the two hooks.
- The current implementation of remotecap_get_data_chunk requires an extra ptp transaction for each file type, because you only find out you are done by requesting another chunk and getting an empty response. This probably doesn't matter much, but if we can cleanly flag "this is the last chunk" it will save a little time. The status returned by this function could also possibly be made clearer.Yes, the condition check could be shifted into remotecap_get_data_chunk() - it could return a value for example.
- should we actually prohibit calling init in playback mode ? Since it just sets up some flags, this doesn't actually cause any harm. You can initialize it and switch to playback mode, so it's still possible to have remotecap enabled in play.This again was necessary for my version of rs, with the integrated shoot().
- The current raw remote capture implementation bypasses raw_savefile() completely, but this function also does some other stuff beyond raw saving, like handling bracketing in continuous mode and shot_histogram. It might be better to remove the remote_capture stuff to raw save_file. This would mean you'd block spytask, which in turn would block capt_seq thoughtThe reason of the bypass was exactly what you say, it would be complicated to do otherwise.
OK, the current issue is probably a problem in my understanding of the old code and change then. filewrite now doesn't use the raw status variable at all (more clear, I think), so hook_raw_save_complete can probably be called when the last raw chunk is collected.Quote- hook_raw_save_complete() is only called after all the chunks are downloaded, meaning the the raw hook will be blocked until the jpeg is complete. This seems like it might have unwanted impacts, perhaps depending on where the raw hook is on a specific camera.I assumed the two hooks to never overlap. If they do, the port is IMHO broken (imagine what would happen if FileWriteTask's open stage found an open file). That was the reason I used the same variable and status codes for the two hooks.
hook_raw_size() is sometimes different than CAM_RAW_ROWS*CAM_RAW_ROWPIX*CAM_SENSOR_BITS_PER_PIXEL/8Do you know which cameras have this ? It seems like it should not ever be true. DNG will probably be corrupt if this happens (more / less data than given in exif...)
A470, the difference is 6 lines. I haven't tested RAW shooting on it too much ;)Quotehook_raw_size() is sometimes different than CAM_RAW_ROWS*CAM_RAW_ROWPIX*CAM_SENSOR_BITS_PER_PIXEL/8Do you know which cameras have this ? It seems like it should not ever be true. DNG will probably be corrupt if this happens (more / less data than given in exif...)
raw has to be finished before jpeg startsNow that I think of it... What happens in continuous mode? There has to be some caching somewhere. Could it be a filesystem cache (meaning Open(), Write(), Close() finish before the data actually ends up on the card)?
A470, the difference is 6 lines. I haven't tested RAW shooting on it too much ;)OK, this should be investigated. It may be that someone thought it was a good idea to discard the blank lines at the bottom of the sensor, but the values should be consistent.
Now that I think of it... What happens in continuous mode? There has to be some caching somewhere. Could it be a filesystem cache (meaning Open(), Write(), Close() finish before the data actually ends up on the card)?When we write raw in continuous mode, I'm pretty certain it blocks until the write is finished, and very little if any caching is done with write().
:-[ Canon DSLRs do this sort of thing (and maybe compacts of some other manufacturers). My mistake. Haven't found any PowerShot with this feature (caching).QuoteNow that I think of it... What happens in continuous mode? There has to be some caching somewhere. Could it be a filesystem cache (meaning Open(), Write(), Close() finish before the data actually ends up on the card)?When we write raw in continuous mode, I'm pretty certain it blocks until the write is finished, and very little if any caching is done with write().
void filewrite_set_discard_jpeg(int state);
void filewrite_get_jpeg_chunk(char **ardr,unsigned *size, unsigned n);declarations in core/remotecap.c be extern?state_shooting_progress=SHOOTING_PROGRESS_DONE;in remotecap_get_data_chunk() is in an unfortunate place though...
A question:For functions, it doesn't make any difference, but I guess we mostly use extern elsewhere (including in the headers), so it would be more consistent.
Shouldn't theseCode: [Select]void filewrite_set_discard_jpeg(int state);declarations in core/remotecap.c be extern?
void filewrite_get_jpeg_chunk(char **ardr,unsigned *size, unsigned n);
state_shooting_progress normally gets this value in core/main.c, which is not happening when remote shooting is active.That makes sense, and I agree that it's not ideal at the moment. Especially in the case where it could get set at an arbitrary time to clear hooks.
It's true thatCode: [Select]state_shooting_progress=SHOOTING_PROGRESS_DONE;in remotecap_get_data_chunk() is in an unfortunate place though...
Index: core/remotecap.c
===================================================================
--- core/remotecap.c (revision 2141)
+++ core/remotecap.c (working copy)
@@ -212,7 +212,7 @@
// allow raw hook to continue
// TODO could do like filewrite wait
hook_raw_save_complete();
- state_shooting_progress=SHOOTING_PROGRESS_DONE; //shoot() needs this... TODO
+ state_shooting_progress=SHOOTING_PROGRESS_PROCESSING;
}
}
orIndex: platform/generic/capt_seq.c
===================================================================
--- platform/generic/capt_seq.c (revision 2141)
+++ platform/generic/capt_seq.c (working copy)
@@ -50,10 +50,12 @@
}
else if ( (remotecap_get_target() & 2) > 0 ) //chdk raw requested
{
+ state_shooting_progress=SHOOTING_PROGRESS_PROCESSING;
remotecap_raw_available();
}
else
{
+ state_shooting_progress=SHOOTING_PROGRESS_PROCESSING;
raw_save_stage = RAWDATA_SAVED;
}
#else
- should the remote capture selection only apply to the next shot, like exposure overrides, or should you need to explicitly cancel it ?My initial code used the latter (rs sets it up at start and resets it when it has finished). Probably both approach should deal with the possibility of PTP connection problems.
If it's left on after the PTP connection goes away, that could confuse users. We don't explicitly know whether there is a PTP connection, but I guess we could check USB power is present (and clear remotecap if it's not found ?).Could be good enough, until somebody decides to allow usb remote and ptp at the same time if that's possible at all (physw bit override or eventprocs like ConnectUSBCable, etc.)
Setting it once per shot shouldn't be much of a burden if you are using script to shoot, but I guess you could potentially set it once and have the user manually shooting with the shutter ?Maybe. There are also some G series cams with proper remote control socket.
In chdkptp r301, I've started making a lua-based remoteshoot command. It's pretty messy at the moment, but it seems to work. This uses remotecap_init() on the camera to set up.Will try it soon.
I understand the "script is already running" problem a bit a better now, of course the 'shoot()' command doesn't finish until the remote capture is done, so you can't cancel with script. The jpeg timeout works at least ;) In the new command, I get around this by using button presses instead of shoot, so the script can end as soon as shoot_full has been clicked.If this approach is the way to go, can we get rid of setting state_shooting_progress?
...hy\CHDK\LiveCapture\chdkptp-r291-win32\lua\chdku.lua:162: attempt to index local 'opts' (a nil value)
stack traceback:
...hy\CHDK\LiveCapture\chdkptp-r291-win32\lua\chdku.lua:162: in function 'mdownload'
...CHDK\LiveCapture\chdkptp-r291-win32\lua\gui_tree.lua:110: in function 'do_dir_download_dialog'
...CHDK\LiveCapture\chdkptp-r291-win32\lua\gui_tree.lua:273: in function <...CHDK\LiveCapture\chdkptp-r291-win32\lua\gui_tree.lua:272>
(tail call): ?
[C]: in function 'Popup'
menu.lua:15: in function 'popup'
...CHDK\LiveCapture\chdkptp-r291-win32\lua\gui_tree.lua:288: in function <...CHDK\LiveCapture\chdkptp-r291-win32\lua\gui_tree.lua:241>
(tail call): ?
[C]: in function 'MainLoop'
...aphy\CHDK\LiveCapture\chdkptp-r291-win32\lua\gui.lua:673: in function <...aphy\CHDK\LiveCapture\chdkptp-r291-win32\lua\gui.lua:659>
(tail call): ?
...phy\CHDK\LiveCapture\chdkptp-r291-win32\lua\main.lua:226: in main chunk
[C]: in function 'require'
[string "require('main')"]:1: in main chunkHowever, on PTP Cam, I can copy the files1) when I press "shoot" to capture a picture, the shot is taken but the camera freezes and the viewfinder on PC becomes white noise (pic attached). The only thing can be done is to shut the camera off manually.Do you have review mode set to "hold" on the camera ? Or a long time ?
Same thing occurs in PTP Cam, except of course, there's no viewfinder.
2) I can't view files under the files tab. I tried copying the contents and got the following error message:This bug is fixed in the latest chdkptp svn. If you replace lua/chdku.lua with the one found here http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293 (http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293) it should be fixed.Code: [Select]...hy\CHDK\LiveCapture\chdkptp-r291-win32\lua\chdku.lua:162: attempt to index local 'opts' (a nil value)
stack traceback:
This bug is fixed in the latest chdkptp svn. If you replace lua/chdku.lua with the one found here http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293 (http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293) it should be fixedI replaced the file, but it didn't solve the problem. Now I get this error message:
Thanx for the replyThat is very strange. Are you using non-latin character set on the camera or PC ?
yes I had my review mode set to "hold", disabling this got things working :)QuoteThis bug is fixed in the latest chdkptp svn. If you replace lua/chdku.lua with the one found here http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293 (http://trac.assembla.com/chdkptp/browser/trunk/lua/chdku.lua?rev=293) it should be fixedI replaced the file, but it didn't solve the problem. Now I get this error message:
"error: A/ں뫧ں§~1.PPT: error"
I tried enteringYou need to enter that line in the text box beside the Execute button, and of course hit enter afterwards (or click Execute).
ls -l
in the console. Nothing happened
Now I get this error message:Are you sure you don't have a PowerPoint file on that card (copied there with a card reader)?
"error: A/ں뫧ں§~1.PPT: error"
=return os.listdir('A/')?int register_pt_hooks() {
#if (!CAM_DRYOS) || defined(CAMERA_s5is)
/*
* System.Create -> SystemEventInit
* SS.Create -> ?
* PT_CompleteFileWrite -> no equivalent?
*/
return -1;
#else
if(_ExecuteEventProcedure("System.Create") == -1) {
return 1;
}
if(_ExecuteEventProcedure("SS.Create") == -1) {
return 2;
}
if(_ExecuteEventProcedure("ExportToEventProcedure","PT_CompleteFileWrite",overridden_PT_CompleteFileWrite) == -1) {
return 3;
}
//_LogPrintf(0x120,"pt hook(s) registered");
return 0;
#endif
}All DryOS >= r23 cameras seem to have System.Create, SS.Create, PT_CompleteFileWrite. This is based on searching through a few dozen firmware files I have on my HDD.All DryOS >= r23 cameras seem to have System.Create, SS.Create, PT_CompleteFileWrite. This is based on searching through a few dozen firmware files I have on my HDD.Calling the eventprocs in the sequence I used and checking for -1 should be OK, the romlog code does the same thing.
Which (kind of) functions are not supposed to be called from thumb?The firmware code doesn't always return with a BX LR, so it might return to your thumb code without switching back to thumb mode. This is most common on vxworks cameras, but happens in some cases newer cameras do it too. So generally you shouldn't call a firmware function directly without a wrapper.
So in this case, we don't need to fall back to the older registration functions, unless we decide to use other eventproc "hooks".I haven't found anything obvious. Close() could be wrapped in filewritetask, but that obviously needs that task hook implemented.
The firmware code doesn't always return with a BX LR, so it might return to your thumb code without switching back to thumb mode. This is most common on vxworks cameras, but happens in some cases newer cameras do it too. So generally you shouldn't call a firmware function directly without a wrapper.I see, just looked at a main.dump.
I actually meant for other purposes, not detecting file write complete. I agree, none of the others look like good candidates for that.So in this case, we don't need to fall back to the older registration functions, unless we decide to use other eventproc "hooks".I haven't found anything obvious. Close() could be wrapped in filewritetask, but that obviously needs that task hook implemented.
Note we should be able to detect if the camera has PT_CompleteFileWrite. The original firmware version (on D10 at least) returns 0. If we call it before registering our own, -1 will indicate the firmware doesn't support, while 0 will indicate it does.I implemented this in changeset 2212.
On the A470, the event procedure registration consumes 5352 bytes according to core_get_free_memory() - tested with the first version of register_pt_hooks().That's not too bad, but if we end up using this, it the final version should probably only register the first time it's needed, rather than in startup. I wouldn't worry about this for now.
You may have already noticed, there's a typo in changeset 2212.Thanks for pointing that out. I didn't notice since a540 doesn't have PT_CompleteFileWrite, and I haven't implemented filewrite task in D10 yet.
I am CHDK beginner so can somebody please update needed sources for my cam?Please be a bit patient. I also have this camera and, out of curiosity, created the necessary modifications for it. However, remote shooting appears to eat up its memory for some reason (unlike on other cameras), and causes it to crash after a few shots. As I'm curious whether it's my fw version (1.00e) that makes problems, I'll make the necessary mods for your fw too - soon.
As I'm curious whether it's my fw version (1.00e) that makes problemsI think it's very unlikely that the Canon firmware version would make a difference.
True, it was a silly idea. Incidentally, the 1.00e fw revision for this camera has a bad bug that made Canon issue a firmware upgrade (to 1.01a).As I'm curious whether it's my fw version (1.00e) that makes problemsI think it's very unlikely that the Canon firmware version would make a difference.
The event procedure registration costs 0 bytes of RAM on this cam, according to calls to core_get_free_memory().
Keep in mind that core_get_free_memory returns the size of the largest free block currently in the heap, not the total amount of free memory available.Re-did the test with core_get_free_memory() adjusted to return camera_meminfo.free_size.
If the event registration required a small amount of memory and there was a small free block on the heap available you would not see any change in the return value from core_get_free_memory.
Cameras with DryOS >= r50 have a more complicated filewritetask. This means that a protocol change and a change of the hooking method will be needed to successfully retrieve those files.In changeset 2226 (https://trac.assembla.com/chdk/changeset/2226/) I have checked in support code for this. It hasn't been tested on a DryOS r50 camera yet, so it may contain some errors.
After analyzing the code (and the debug data provided by nafraf) I concluded that there must be some multitasking issue between the ptp and filewrite tasks. If that's the case, changeset 2243 (https://trac.assembla.com/chdk/changeset/2243/) should fix it.The change has now been tested, and it appears to work correctly. Many thanks to nafraf for helping me to make this happen!
hello phil,
i am still using your CHDKPTPRemote (posted by you around page 38 - around a year back) along with Mweerden's liveview patch.
as liveview is already included in the trunk and as it has been implemented as a top level patch, i have been unable to make the old code (c# solution) work with the new live view. if you still using the code and if it is not to much effort can you upload a new version.
thanks
thanks,
but link gives me a 404
Sorry, I'd like to know how to use the CHDK PTP interface available in https://www.assembla.com/spaces/chdkptp/documents (https://www.assembla.com/spaces/chdkptp/documents) with my Canon PowerShot SX130 IS. I need to use the remote capture feature, but I don't know which procedure to follow.You don't need do any programming. Assuming you are using windows:
Thankfully :)
I have tried to do this but when I click on the button "Connect" in the interface, the message "error: no devices available" appears.Perhaps you haven't successfully installed the libusb driver.
I need some advice. I'm trying to make DNG available as RAW format for ptp remote capture. Doing this requires the use of the DNG module. However, existing exported functions are not appropriate for this use, so I needed to create some new ones.
What would be the better solution?
- Extend the DNG module with the new functions.
- Create a new module using the source of the DNG module (with #ifdefs, etc.). The problem is that the DNG module is treated special by other code (modules.c for example), so doing this seems a bit hard.
If you mean you want to call functions in the DNG module from the core CHDK code then just add them to the end of the 'struct libdng_sym' structure.Thanks, I'll do it like this then.
Remember to update both the definition in dng.h and the instance in dng.c.
If you also update the version in the libdng variable in dng.c you can check the version in your code before calling any of the functions to make sure the new module version is loaded.
I followed the installation steps, but when I run the chdkptp executable and I click any button, the message "unexpected return code 0x2005" appears. Is there any incompatibility with my camera model (SX 130 IS)?You can get error 0x2005 when you try to connect to a camera on which CHDK is not loaded. Download a fresh CHDK release from the autobuild server (http://mighty-hoernsche.de/), and install it on the camera. Use the "Bootable SD card method" (http://chdk.wikia.com/wiki/Bootable_SD_card).
But there's a problem: for use the Bootable SD Card Method I need to set the card's lock switch to the "LOCK" position.Let's start again.
My later plan includes using the dng module to get the CHDK exif (optionally).I have implemented dng as raw format in changeset 2270 (https://trac.assembla.com/chdk/changeset/2270/). It completely relies on CHDK settings, i.e. DNG has to be activated in CHDK. If DNG support is switched off, only the usual RAW buffer will be transferred. I did it like this because scripts can alter CHDK settings when needed.
I don't think #ifdef code in a module is a good idea. As far as possible the modules should be platform and version independent.Done in changeset 2271 (https://trac.assembla.com/chdk/changeset/2271/).
Some of you may not like the way I changed dng.h and dng.c (I mean the added #ifdefs), if I should change it, just ask :)
I execute chdktp and click in "connect". So, the message "error: no devices available".You get this when the libusb driver is not installed for your camera. You need to install it for every camera(model) you'd like to use on the PC.
I already did it :( but didn't workCan you find a "libusb-win32 device" in the device manager when the camera is connected? Is it your SX130IS?
@reyalpIn fact, the ptpcam based ptp.c code did not fill in this value (see ptp.c ptp_usb_getresp)
Somehow ptp.Nparam doesn't get updated after ptp_transaction().
Parameters are always 4 bytes in length. The number of parameters can be determinedI've added this code in chdkptp changeset 312. If anything relied on Nparam continuing to hold the number of input params after a transaction, it will be broken, but I didn't see anything like that from a quick inspection.
from the Container length. (Length – 12)/4
Thanks for that, I wanted to use Nparam for an additional check, not sure whether it will still be needed when the protocol becomes final.@reyalpIn fact, the ptpcam based ptp.c code did not fill in this value (see ptp.c ptp_usb_getresp)
Somehow ptp.Nparam doesn't get updated after ptp_transaction().
According to Universal Serial Bus Still Image Capture Device Definition, 7.1.4 Response Block Payload StructureQuoteParameters are always 4 bytes in length. The number of parameters can be determinedI've added this code in chdkptp changeset 312. If anything relied on Nparam continuing to hold the number of input params after a transaction, it will be broken, but I didn't see anything like that from a quick inspection.
from the Container length. (Length – 12)/4
Thanks for that, I wanted to use Nparam for an additional check, not sure whether it will still be needed when the protocol becomes final.Yeah, I'm not sure varying the number of return parameters is a great idea, but it seemed like something the PTP code should provide.
I'm considering the simplification of filewrite_main_hook()'s arguments. If the FileWriteTask data block is mapped to a camera-specific structure, only a pointer would be needed as a single argument. It would also make the assembly (glue) part simpler and the code less ugly.Reducing the amount of glue sounds good.
int fwt_open(char *name,int flags,int mode) {
if(ignore_current_write) {
current_write_ignored = ignore_current_write;
return 0xff;
}
return _Open(name,flags,mode);
}
not promising I understood the current asm correctly... which is a good reason to do it in CReducing the amount of glue sounds good.You're probably right. Those functions started out as part of the "glue" and I wanted full control over the saved registers, so I haven't even thought about doing them in C :)
A question about fwt_open etc. Is there a reason these can't be C code? Since they are called as a replacement for a normal function call, it seems like you should just be able to make normal C function with the appropriate number of parameters. The registers will be preserved or not according to the ARM ABI, just like the replaced call.
eg:Code: [Select]int fwt_open(char *name,int flags,int mode) {not promising I understood the current asm correctly... which is a good reason to do it in C
if(ignore_current_write) {
current_write_ignored = ignore_current_write;
return 0xff;
}
return _Open(name,flags,mode);
}
Some more general thoughts, (thinking out loud, not saying anything has to change)Well, this part is a bit neglected at the moment, with only one of the cameras supporting it. It's true that it's of no use to most of the expected user base (which may not be very high either), and getting the YUV is a bit troublesome (it's not present when the JPEG is L or W sized, needs activated review to not get erased before transfer). I thought it would be interesting to have it, because 1) you get to see which part of the image deterioration is the result of the camera's JPEG compression 2) if you'd like to pre-process the image somehow (say, you want some text or image overlay on it) before storing or using it, you evidently lose quality when your source is already JPEG compressed 3) it might be good sometimes that the camera is doing demosaicing for you (sadly excluding full size).
YUV:
Is this really worth the effort? What situations would the jpeg not be good enough, but the raw too much much? I know it is relatively easy to get, but it's still several functions that have to be added to each lib.c and then it isn't available in all shooting modes. I guess the YUV may be nice because some third party libraries would already be able to deal with it directly, with even less overhead than jpeg. I'm not against including it, just wondering if there is a specific use that makes it important.
DNG:Yes, the current code is the first try (I think there are comments indicating this possibility)
For applications that want to maximize speed, it would be useful to be able to get the DNG header without having the DNG data bytes reversed.
This could also provide a way for dealing with reduced cropped images. You would get the full header (with the dimensions set normally) and the client application could reverse the bytes and pad it to the desired size, or adjust the dng dimensions.I probably wanted to avoid dealing with both sides of this, to keep the first implementation simple and not having to code more :)
I'm not sure I like relying the chdk UI DNG setting to decide if raw returns plain raw or DNG, especially since you have to use set_config_value / get_config_value to modify it from script. It seems like the client should explicitly ask for what it wants.No objections, I did it so, because it seemed easy for a shooting script to include set_config_value. Should this be handled by a new script command setting some static variables, or ...?
I don't know how the compiler decides over which registers should be preserved. If you think it's safe to do, then I'll re-write them.In general if you are replacing a canon firmware function call with another function call, it should be safe. Which registers are preserved is defined by the ABI, the compiler doesn't look at the called/calling code (it can't since they could be in different object files.) The ARM function call ABI (http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0042e/index.html) is pretty standard (+/- different floating point modes and arm/thumb interworking) unlike the PC world where different compilers have used wildly different conventions.
I probably wanted to avoid dealing with both sides of this, to keep the first implementation simple and not having to code more :)Understood, and agreed that it's better to get something simple working first :)
No objections, I did it so, because it seemed easy for a shooting script to include set_config_value.Yes I see the attraction of this. It's not clear how this would work if we wanted to allow the DNG header separately or byte reversing control though.
Should this be handled by a new script command setting some static variables, or ...?I would expect it to be part of init_remotecap. Logically I'd expect it to just be a format option, but having raw and dng be separate bitflags doesn't really make sense.
No YUV, not sure if it exists on digic > 2, or how to find it.I plan to do this on a DryOS cam sooner or later. I simply dumped all RAM from the filewrite hook routine, used the usual hex editor with the visualization, and after that I looked up the found address in fw. That said, it's easier to do on a camera with 16MB of RAM than on another with 64 or 128... .
I also wasn't clear how MAX_CHUNKS_FOR_JPEG is determined. 3 seems to work. I assume the structure is OK, since I get a valid jpeg.Again, I dumped that specific piece of RAM from the filewrite hook (on D10, sub_FFA26268_my gets a pointer to it in R0), length is less than 0x80. Actually, that pointer is received by the task via ReceiveMessageQueue (sub_FF826C30).
I was thinking about removing the all in one "remoteshoot" command and associated functions from chdkptp, so we only have to keep the lua based one up to date. Let me know if this is a problem for you.No problem, just do it.
For YUV, there's libyuv (http://code.google.com/p/libyuv/). Yeah, it's just a library, and I haven't checked whether it actually supports the YUV variants we deal with here.It would be nice to get a proper conversion of this YUV to see how it really compares to jpeg. Given that it's only available in reduced size (for cameras so far), it seems like it's unlikely to be better than full res jpeg. I can imagine automation projects that wanted to look quickly look at a small part of the images might like YUV, since we can't really return a sub-range of jpeg. OTOH, the YUV actually comes out bigger than the correspond raw...
I'm considering the simplification of filewrite_main_hook()'s arguments. If the FileWriteTask data block is mapped to a camera-specific structure, only a pointer would be needed as a single argument. It would also make the assembly (glue) part simpler and the code less ugly.It looks like this (camera specific capt_seq.c):
typedef struct {
unsigned int address;
unsigned int length;
} cam_ptp_data_chunk; //camera specific structure
#define MAX_CHUNKS_FOR_JPEG 7 // filewritetask is prepared for this many chunks
/*
* fwt_data_struct: defined here as it's camera dependent
* unneeded members are designated with unkn
* file_offset, full_size, seek_flag only needs to be defined for DryOS>=r50 generation cameras
* pdc and name are always needed
*/
typedef struct
{
int unkn1;
int file_offset; // needed for DryOS>=r50
int full_size; // needed for DryOS>=r50
int unkn2, unkn3, unkn4;
cam_ptp_data_chunk pdc[MAX_CHUNKS_FOR_JPEG];
int seek_flag; // needed for DryOS>=r50
char name[32];
} fwt_data_struct;
#define FWT_MUSTSEEK 2 // value of seek_flag indicating seek is required
#define FWT_CONDSEEK 3 // value of seek_flag indicating seek is required when file_offset not 0
Example of an earlier DryOS cameratypedef struct {
unsigned int address;
unsigned int length;
} cam_ptp_data_chunk; //camera specific structure
#define MAX_CHUNKS_FOR_JPEG 3 //model specific
/*
* fwt_data_struct: defined here as it's camera dependent
* unneeded members are designated with unkn
* file_offset, full_size, seek_flag only needs to be defined for DryOS>=r50 generation cameras
* pdc and name are always needed
*/
typedef struct
{
int unkn1, unkn2, unkn3, unkn4, unkn5;
cam_ptp_data_chunk pdc[MAX_CHUNKS_FOR_JPEG];
char name[32];
} fwt_data_struct;
Which makes the assembly glue simpler://place hook here
"STMFD SP!, {R4-R12,LR}\n"
"MOV R0, R4\n"
"BL filewrite_main_hook\n"
"LDMFD SP!, {R4-R12,LR}\n"
//hook end
I guess it's better. I have a feeling that cam_ptp_data_chunk will go into a common header as it's always the same.
I guess it's better. I have a feeling that cam_ptp_data_chunk will go into a common header as it's always the same.Agreed. I would expect fwt_data_struct is very likely tied to dryos rev, so could be in a common header with a minimal number of ifdefs.
It usually works after the camera is switched to rec mode, but fails with timeout after the camera goes to powersave mode (rs wake up the cam as usual and the half press phase is OK, but then nothing happens). I don't remember experiencing this with the monolithic rs, hopefully it's not related to the completefilewrite replacement. OTOH the A470 is working normally, no problems.One difference is that this one uses shoot_half, wait for get_shooting etc, instead of shoot() Full camera side code is in lua/rlibs.lua rs_shoot
There's another small bug in rs, it fails to append the received file name when a target directory is given (I can probably figure out how to solve this one).Whoops, I left that half finished, should be better in changeset 321.
I seem to remember having had other weird problems when the camera goes to screen off mode, I usually disable it when I'm doing ptp stuff.I don't remember anything weird in connection with powersave mode. During earlier testing I left the cameras in rec mode sleeping for hours and rs always succeeded when executed later. I guess I'll investigate this stuff.
Whoops, I left that half finished, should be better in changeset 321.Thanks :)
rs (no file) = use camera filename
rs foo = if foo is a directory, use foo/<camera filename>, otherwise foo.jpg, foo.raw
rs foo/ = create foo if it doesn't exist, use foo/<camera filename>
Note that CAM_CHDK_PTP_REMOTESHOOT can't be turned off at the moment, because ptp.c is now in platform independent code. I'm not sure it's worth keeping this as an option at all, since all cameras should at least support raw. It would save a small amount of memory.
If it needs to be turned off because the functionality is not available then it should be done at runtime. Add a new entry to camera_info.h/.c and check that in the code.The minimum functionality (raw only) is available on any camera, so there's really no point in turning it off at runtime. The only reason to keep the #ifdef would be to reduce the core executable size slightly. This can be done, it would just require moving some of the code out of ptp.c and stubbing out the public functions when it's not available. I don't really think this is worthwhile.
Also done some tests, and found that the new rs command doesn't work reliably with the A410.It's puzzling, but after putting in larger amounts of debug code, it appears that the problem is related to flash. In other words, it's probably the same issue that skrylten is experiencing (http://chdk.setepontos.com/index.php?topic=9118.0). I usually disable flash when testing, but haven't always done so recently, hence the issue...
The only reason to keep the #ifdef would be to reduce the core executable size slightly.Not fully related, but I'd consider putting the whole filewritetask related code inside #ifdefs, including the copied task code and the lines related to task replacement.
function rs_shoot(opts)
local rec,vid = get_mode()
if not rec then
return false,'not in rec mode'
end
if type(init_remotecap) ~= 'function' then
return false, 'remotecap not supported'
end
if bitand(get_remotecap_support(),opts.fformat) ~= opts.fformat then
return false, 'unsupported format'
end
if not init_remotecap(opts.fformat,opts.lstart,opts.lcount) then
return false, 'init failed'
end
press('shoot_half')
local shtimeout = get_tick_count()
repeat
sleep(10)
until ( get_shooting() ) or ( (get_tick_count()-shtimeout)>5000 )
reset_shoot_ok()
press('shoot_full')
local gsrtime = get_tick_count()
repeat
sleep(10)
until ( get_shoot_ok() ) or ( (get_tick_count()-gsrtime)>5000 )
release('shoot_full')
return true
endreset_shoot_ok() and get_shoot_ok() are new functions. They use a new static variable which is set in Some news.Unless there is some really compelling use for the YUV, this doesn't seem like it would justify adding a new task hook. Even at full size, I'm have trouble seeing one.
I've been trying to implement YUV support for the A470. Looks like it's a no-go with the filewrite hook. It appears that by the time the filewritetask hook is reached, the YUV source of the JPEG is overwritten by the decompressed Y411 version of the fresh JPEG (unless 640x480 size is selected). So, either a new hook or no YUV support on this camera...
For the rs "bug" I came up with the following (it's probably influenced by lapser's findings)did you check get_flash_ready() ?
...
reset_shoot_ok() and get_shoot_ok() are new functions. They use a new static variable which is set in the raw hook. This approach appears to work on the A410 and 470 - even with flash. My trials included putting arbitrary delays after pressing shoot_full, but that method was unreliable (unless the delay was set to like 2 seconds).
did you check get_flash_ready() ?Yes.
function rs_shoot(opts)
local rec,vid = get_mode()
if not rec then
return false,'not in rec mode'
end
if type(init_remotecap) ~= 'function' then
return false, 'remotecap not supported'
end
if bitand(get_remotecap_support(),opts.fformat) ~= opts.fformat then
return false, 'unsupported format'
end
if not init_remotecap(opts.fformat,opts.lstart,opts.lcount) then
return false, 'init failed'
end
press('shoot_half')
local shtimeout = get_tick_count()
local gfrtime = 0
repeat
sleep(10)
if get_flash_ready() and (gfrtime==0) then
gfrtime=get_tick_count()-shtimeout
end
until ( get_shooting() and ( get_flash_ready() or (get_flash_mode()==2) ) ) or ( (get_tick_count()-shtimeout)>5000 )
shtimeout=get_tick_count()-shtimeout
reset_shoot_ok()
press('shoot_full')
local gsrtime = get_tick_count()
repeat
sleep(10)
until ( get_shoot_ok() ) or ( (get_tick_count()-gsrtime)>5000 )
gsrtime=get_tick_count()-gsrtime
release('shoot_full')
return true, 'get_shooting: ' .. shtimeout .. 'ms , gfr: ' .. gfrtime .. 'ms, gsr: ' .. gsrtime .. 'ms'
endAbove version (I haven't removed the debug related lines) works, tried on the A410 and A460 this time.function rs_shoot(opts)
local rec,vid = get_mode()
if not rec then
return false,'not in rec mode'
end
if type(init_remotecap) ~= 'function' then
return false, 'remotecap not supported'
end
if bitand(get_remotecap_support(),opts.fformat) ~= opts.fformat then
return false, 'unsupported format'
end
if not init_remotecap(opts.fformat,opts.lstart,opts.lcount) then
return false, 'init failed'
end
press('shoot_half')
local shtimeout = get_tick_count()
local gfrtime = 0
repeat
sleep(10)
if get_flash_ready() and (gfrtime==0) then
gfrtime=get_tick_count()-shtimeout
end
until ( get_shooting() and ( get_flash_ready() or (get_flash_mode()==2) ) ) or ( (get_tick_count()-shtimeout)>5000 )
shtimeout=get_tick_count()-shtimeout
local gsrtime = get_tick_count()
click('shoot_full')
return true, 'get_shooting: ' .. shtimeout .. 'ms , gfr: ' .. gfrtime .. 'ms, gsr: ' .. gsrtime .. 'ms'
endOf course I'm not trying to say that my "solution" is idealPlease don't check anything into ptp-remote-capture-test. I'll delete it soon (only removes it from the list of current branches)I have removed the old branches.
It might make more sense to split the chdk specific stuff into separate headers.
... but the live view does not display the screen correctly. The picture is distorted, showing 1/4 of the picture four times. Is this a known issue?Now it is. Can you show a screenshot?
... but the live view does not display the screen correctly. The picture is distorted, showing 1/4 of the picture four times. Is this a known issue?Now it is. Can you show a screenshot?
Yeah, here is the screen shot of what is happening...I see. The reason is that not all liveview related code is implemented in this port. I can probably help with this, but since I don't have access to this camera, I will need feedback. Are you ready to do some testing?
I updated the remote capture branch with philmoz module re-work. It seems to work, but I haven't tested it carefully. This will probably make it hard to merge back (because there were changes needed to make it build checked in at the same time), but I guess we'll deal with that when we get there...Had second thoughts about this, made a new branch ptp-remote-capture-test3. Future changes should go there.
I would like to "land" the remote cap branch fairly soon.Would it make sense to implement querying/reporting camera related quirks, so that the client can take appropriate measures to avoid known problems?
I've also had weirdness / failed connections when the camera goes into powersave mode.- when the camera can't tolerate remote shooting in too fast succession (ixus870)
- PTP (or the whole USB subsystem) gets shut down somehow when the number of "camera objects" grows beyond a certain limit. This might be related to that "communication error" message which is mentioned in the manual (I personally have not seen it yet).Yes, I saw the "communication error" when I was chasing the Lua bug. It's annoying, would be nice if we could block the camera from whatever triggers this. I guess building the list of PTP objects?
- at least on DryOS cameras, a certain amount of exmem (1MB in cases I've seen) gets allocated when usb is connected which may or may not interfere with record mode.Yes, EXMEM_COM is in the firmware exmem view output.
In a separate note, recent trunk changes can't be easily merged into the v3 remote capture branch.Yeah, would be good to land this branch soon. I'll try to have a look at updating.
For fuse, there's this: http://www.gphoto.org/proj/gphotofs/ (http://www.gphoto.org/proj/gphotofs/)Yes, this sort of thing is what brought up the discussion in the first place. The idea is to expose the full filesystem, vs what is already accessible via regular PTP. Also, as I mentioned earlier it would be a lot more flexible than the current options. However, this isn't something I think is urgent to do now, it's just an idea that I've had kicking around for a while. I posted it here because nafraf expressed some interest in working on it.
Yes, I saw the "communication error" when I was chasing the Lua bug. It's annoying, would be nice if we could block the camera from whatever triggers this. I guess building the list of PTP objects?I don't know how these object representations are related (the camera's internal list of its known files vs the same for ptp). If my memory is correct, the camera hasn't even shown up as USB device when plugged in, so it got overwhelmed before any ptp connection attempt.
Yes, EXMEM_COM is in the firmware exmem view output.PTP acts unstable on certain cameras (regardless of movie recording), but I have no clue why, I can never reproduce.
At a minimum, the cameras that have quality issues when CHDK is in exmem will probably suffer the same problem. Worse if CHDK is already in exmem.
I wonder if video recording clobbers the EXMEM_COM section like it does CHDK allocations? If not, maybe we can figure out how. And if it does, I assume PTP will become very unreliable if you try to record video
I'll try to have a look at updating.Not too important at the moment, I mentioned it as a hint why I haven't done the update.
If my memory is correct, the camera hasn't even shown up as USB device when plugged in, so it got overwhelmed before any ptp connection attempt.When I got the "communication error" it appeared when plugging the camera in, and the camera was not recognized at all on USB.
In a separate note, recent trunk changes can't be easily merged into the v3 remote capture branch. I was able to - sort of - merge it into current trunk code, with the exception of platform *.S files, but I would need a full rebuild with all fw dumps to correct it. The usual "cherry-picking" method seems more painful.
/tmp/ccxjW2Jc.s: Assembler messages:
/tmp/ccxjW2Jc.s:166: Error: invalid literal constant: pool needs to be closer
/tmp/ccxjW2Jc.s:175: Error: invalid literal constant: pool needs to be closer
/tmp/ccxjW2Jc.s:176: Error: invalid literal constant: pool needs to be closer
make[2]: *** [capt_seq.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [batch-zip-complete] Error 1
and thank you very much for doing that.+1
Is this 4.4.3 toolchain available somewhere?https://www.box.com/s/2bs0i77srru9d68cy6j5 (https://www.box.com/s/2bs0i77srru9d68cy6j5)
Thanks. Unfortunately it's x86-64, so I can't currently use it. Can you show the output ofIs this 4.4.3 toolchain available somewhere?https://www.box.com/s/2bs0i77srru9d68cy6j5 (https://www.box.com/s/2bs0i77srru9d68cy6j5)
arm-elf-objdump -v
[build]$ arm-elf-objdump -v
GNU objdump (GNU Binutils) 2.18
Copyright 2007 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.
arm-elf-gcc -v
[build]$ arm-elf-gcc -v
Using built-in specs.
Target: arm-elf
Configured with: ../configure --target=arm-elf --prefix=/home/hacki/newenv/ --enable-languages=c --disable-shared --disable-newlib --disable-libssp
Thread model: single
gcc version 4.4.3 (GCC)
Thanks. Unfortunately it's x86-64, so I can't currently use it.FWIW, I have a x64 system and a similar generation toolchain I built, so I should be able to check on the pool issue.
I wonder whether a toolchain with these binutils and gcc versions will act the same as the autobuild's...FWIW ... the values I posted are from the toolchain that hacki uses for the autobuild - or at least they were when he shared them with me.
This whole remote capture branch has been a disaster. I am unsure what to do about it. IMO, the current implementation is too much of a mess to accept as is, it should not be in the trunk. It needs significant work both in the protocol and on the client side.Easy for me to say this, but sometimes its just worth chucking the code and using the knowledge gained to start again cleanly?
Easy for me to say this, but sometimes its just worth chucking the code and using the knowledge gained to start again cleanly?I don't think that's the case here. The code we have is stuff we need, there is just more to be done.
Seeing the turbulent changes in CHDK trunk, it makes me wonder whether it would have been a better idea to continue development on the stable 1.1 codebase.Or it might be time to bump the revs? The UI changes in 1.2.0 are probably enough to justify moving it to "stable" status, retiring 1.1.0, rewriting the user guide, and putting the ptp stuff into a newly created 1.3.0?
Probably.Seeing the turbulent changes in CHDK trunk, it makes me wonder whether it would have been a better idea to continue development on the stable 1.1 codebase.Or it might be time to bump the revs? The UI changes in 1.2.0 are probably enough to justify moving it to "stable" status,
rewriting the user guide,Badly needed, but :blink:
putting the ptp stuff into a newly created 1.3.0?Without restructuring the rc code, it will become the same pain to maintain.
- Remove functionality that is unneeded at this point: YUV. I was going to say DNG too, since the RAW/DNG code has always been file-centered, and I had to butcher an existing DNG function (done in January(?), many things have happened since) to make DNG available for rc.I agree about YUV. It's a bit of a pity having done the work, but I don't see any real use for it, especially considering you can't get it full res.
- Split every possible piece of rc related code into separate files - looks like choosing capt_seq.c for holding the additional code was a bad idea.I was thinking about this today too. Putting filewrite in it's own C file would make sense, but then we need somewhere for the generic filewrite stuff.
- The rc related variables may need reorganization.Agree, there's some cleanup that could be done.
- Remove the usage of event procedures. It's AFAIK not needed for rc to work.Agreed.
Client side:That's understood, I have been slow working on this, and some changes need to happen simultaneously on both sides.
The problem is, any advanced usage of Lua is beyond me, which prevents me from doing any useful work on CHDKPTP.
Seeing the turbulent changes in CHDK trunk, it makes me wonder whether it would have been a better idea to continue development on the stable 1.1 codebase.I don't think so, getting it into the trunk would have been an even bigger problem then. We might have been better trying to to keep updated with the trunk, but that also would have made the final merge horrendous.
I still think that this kind of remote capture support is much better than retrieving the most recently shot image from the card.I agree. However, getting from the card is needed as a fallback unless we implement filewrite for every camera.
I still think that this kind of remote capture support is much better than retrieving the most recently shot image from the card.
Maybe so, and this is an interesting excercise for those who enjoy code development, but realistically what is the PTP feature actually being used for in real life (rather than day-dreaming) and what percentage of CHDK users would be inconvenienced if it did not exist ?I would argue that once you get past basic setup & configuration issues, the next most popular requests on this forum relate to projects that need to be able to coordinate activities between a PC and the camera. So ptp support is a lot more than just an interesting coding concept - there is as strong interest and demand for the capability. And as that capability will apparently not be available in any other hacks, that just leaves CHDK.
as that capability will apparently not be available in any other hacks, that just leaves CHDK.
Maybe so, and this is an interesting excercise for those who enjoy code development, but realistically what is the PTP feature actually being used for in real life (rather than day-dreaming) and what percentage of CHDK users would be inconvenienced if it did not exist ?I would argue that once you get past basic setup & configuration issues, the next most popular requests on this forum relate to projects that need to be able to coordinate activities between a PC and the camera. So ptp support is a lot more than just an interesting coding concept - there is as strong interest and demand for the capability. And as that capability will apparently not be available in any other hacks, that just leaves CHDK.
I agree about YUV. It's a bit of a pity having done the workWell, my assumption about it turned out to be not true - it's not available in every camera anyway. Seeing ML's progress with EDMACs, this may change one time in the future.
I've removed the PT_ hooks along with the imagesavecomplete logic and YUV support. I've left some parts of this commented in case we want to use it later. I think the imagesavecomplete stuff and the PT hooks might be useful at some point in the future, but they don't need to be done to finish the remote capture stuff.Thanks for doing all this work. My a470 also works, including dng.
I also split the filewrite code into separate files. I have built all the affected cameras but only tested a540 and d10 of course.
Splitting the filewrite code into it's own file does appear to resolve the literal pool issues.I think that too large contiguous blocks of naked asm functions cause this kind of error.
I also split the filewrite code into separate files. I have built all the affected cameras but only tested a540 and d10 of course.I tested on a495(100f) and a810(100d), it works.
]I think that too large contiguous blocks of naked asm functions cause this kind of error.I agree. I suspect older versions of GCC are not smart enough to put pools between functions, while the new ones are.
The code in filewrite.c and remotecap.c is still kind of messy. The most irritating part is the huge number of #ifdef's, but I have no idea how to get rid of those.Yes, agreed. Splitting is only a first step. Somehow the ifdefs and variables need to be cleaned up, but it's not obvious how.
- DNG header is an option like raw and jpeg. This is independent from the CHDK menu DNG option, and at the protocol level, independent of whether raw is requested.Makes sense, no separate state variable needed.
- No thumbnail is included, but header is generated normally (as if there were a 128x96 thumb). Client can create a thumb or blank data.This will require work on the dng module - create_dng_header() is not suitable for this, unless it gets changed. Making a copy of it would again create maintainability problems.
- Raw data is always returned in native byte order, no bad pixel patching etc
- Client is responsible for constructing a valid DNG if desired.
- If a sub-image is requested, the client can adjust the header, or pad the image to full size.
This keeps the options and protocol very simple, and minimizes overhead on the camera side. It requires more work in the client code, but it feels like the right way to do it.An additional downside is that whoever implements a new client, may not feel like messing with this - but at least headerless RAW will always be available.
Issues:The module's unload checker currently bases its decision on the two RAW related config values. What else should it check when rc is involved? Play mode? Timeout? USB presence?
- Need to ensure the dng module gets loaded if needed, and preferably unloaded when no longer needed. It seems like the logical place to do this is when the option is set, but loading/unloading the module for very shot is undesirable.
- DNG version of the header? If the DNG option is independent of the menu, it seems like this should be set explicitly as well. It would be tempting to just pick one, but I think we might regret that later.
set cli_verbose=2
rs -f=3
that would be helpful.This will require work on the dng module - create_dng_header() is not suitable for this, unless it gets changed. Making a copy of it would again create maintainability problems.I think it will be fairly simple to make create_dng_header() take some options.
An additional downside is that whoever implements a new client, may not feel like messing with this - but at least headerless RAW will always be available.and sample code will be available in chdkptp, though likely split between C and Lua which may not be clear to many users.
-The module's unload checker currently bases its decision on the two RAW related config values. What else should it check when rc is involved? Play mode? Timeout? USB presence?Yes, this is the question. USB present seems like the most logical, but we really want to know if there is a USB connection, not USB power. A timeout that resets whenever a DNG remote shoot is requested should be fairly simple.
My proposal is they get pure raw data + a DNG header, not DNG data. Since the client is responsible for reversing, it could just as easily patch badpixels too.Bad pixel treatment* depends on this, so it does matter which version is chosen. I don't think that this(*) task belongs to the client side.You wrote above that "no bad pixel patching". Hmmm. That's DNG 1.3
The other way of completing RAW/DNG could be:Yes, I thought about a few variants of this, but they all seemed to end up messy.
DNG and its version is sent as an option (these bits would need to be treated differently, not as another data type of course). This would require a ptp-local state variable (where should these be kept?). Leave create_dng_for_ptp() almost as it is now (byte reversal and badpixel patching would become optional). In case of partial raw, the client patches the header.
In chdkptp changeset 350 (http://trac.assembla.com/chdkptp/changeset/350/trunk) I've moved the handling of chunks into lua. This is in preparation for changing DNG, but the protocol isn't changed yet.
I would like to know if this works on the cameras that require seek in the jpeg chunks.
If someone with one of these cameras can show me the output ofCode: [Select]set cli_verbose=2that would be helpful.
rs -f=3
connected: Canon PowerShot A810, max packet size 512
con> set cli_verbose=2
con> rs -f=3
ERROR: not in rec mode
con 1> rec
con 2> rs -f=3
got name IMG_0042
rcgetfile IMG_0042.raw 1
rcgetchunk IMG_0042.raw 1
rcgetchunk size:1536 offset:nil last:false
rcgetchunk IMG_0042.raw 1
rcgetchunk size:36864 offset:nil last:false
rcgetchunk IMG_0042.raw 1
rcgetchunk size:24724224 offset:nil last:true
rcgetfile IMG_0042.jpg 0
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1042944 offset:17920 last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:993297 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:17920 offset:0 last:true
libdng->capture_data_for_exif();
butint _module_can_unload()
{
return (running == 0) && ((conf.save_raw == 0) || (conf.dng_raw == 0));
}
will cause it to unload right away if raw isn't on in the menu.Thanks nafraf.
It turns out the raw disabled issue happens due to a module loading problem. The dng module will be autoloaded withCode: [Select]libdng->capture_data_for_exif();butCode: [Select]int _module_can_unload()will cause it to unload right away if raw isn't on in the menu.
{
return (running == 0) && ((conf.save_raw == 0) || (conf.dng_raw == 0));
}
So we need to address this issue anyway, even without my dng proposal.
Alternately, we could just require raw to be enabled, and fail init_remotecap if it isn't, but that seems pretty ugly. I wouldn't want to set it in the config there, since it would change the users saved values unless we also save and restore it...
if (conf.save_raw && conf.dng_raw && is_raw_enabled())
{
libdng->capture_data_for_exif();
}
So capture_data_for_exif is only called when raw & dng are enabled - so module unloading doesn't happen and isn't an issue.Right, and the remotecap variant is called in a remotecap.c remotecap_raw_savefile and doesn't do this, which is the problem.
You also need to take care of the modes where raw/dng doesn't work - currently handled by the is_raw_enabled() call above.That's a good point. Which leads to the question of whether remotecap raw should honor or ignore the "raw exceptions" settings. I would say no, but that means that is_raw_enabled() would need to be split into mandatory and preference conditions.
Two patches to add remote capture support for a1300 and a4000.Thanks. Added in changeset 2853 (http://trac.assembla.com/chdk/changeset/2853).
- a1300 (tested on A810 100d) - this includes small change to a810 boot.c, to use task_FileWrite name in taskHook()
- a4000 (tested on A4000 101b by rkomar)
DNG header is an option like raw and jpeg. This is independent from the CHDK menu DNG option, and at the protocol level, independent of whether raw is requested.I've started implementing this in chdk changeset 2861 (http://trac.assembla.com/chdk/changeset/2861) chdkptp changeset 352 (http://trac.assembla.com/chdkptp/changeset/352/trunk)
I still think this is the right direction.I'm sorry if I caused the impression that I'm objecting against this rework. I'm fine with the changes.
On a semi-related note, the whole business of having the remotecap have it's own butchered variant of raw_savefile called from capt_seq also bothers me. raw_savefile does more than just saving, it's involved in bracketing, shot histogram, curves etc. Not all of those make sense for remotecap, but it's an ugly duplication.I agree, I just wanted to spare changing another source file when I did that. Wasn't even sure about which calls to keep in that function.
I'm sorry if I caused the impression that I'm objecting against this rework. I'm fine with the changes.No, you didn't. I just meant it still seems ok after I started coding it. Sometimes ideas that sound good in theory end up being ugly when you actually try to implement them.
I agree, I just wanted to spare changing another source file when I did that. Wasn't even sure about which calls to keep in that function.Ok, that makes sense. I will look at integrating it in raw_savefile, though there's some weird cases there too.
On a separate note, you can correct my comments and variable names in case they're confusing or inappropriate (there are quite a few misspelled words / sentences in the CHDK source, I suspect that they are not corrected due to respect :) ).Probably more lazyness than respect ;) I did change notlastchunk to morechunks, because negatives in variable names make my head spin...
I want to sent PTP over USB to control focus, zoom, ISO, etc. My plan is to use an Arduino host shield over USB. Is that possible? I am planning to shoot time lapse by controlling various camera settings over USB.Yes - its very possible. Although you might be able to do everything you need with a CHDK script and save yourself a lot of work.
I want to sent PTP over USB to control focus, zoom, ISO, etc. My plan is to use an Arduino host shield over USB. Is that possible?Yes - its very possible.
Could you suggest a beginning point to investigate how to use PTP with CHDK?http://chdk.wikia.com/wiki/PTP_Extension (http://chdk.wikia.com/wiki/PTP_Extension)
This thread is four years old, but I can't find any tutorials or detailed instructions anywhere.Technically, this thread has been running for four years. The information in the recent posts is current.
What CHDK build is recommended for this? What is the best source of information on how to pursue this?The latest build of the dev trunk is usually best - but there is a dedicated build for dev testing as well.
Could you suggest a beginning point to investigate how to use PTP with CHDK? This thread is four years old, but I can't find any tutorials or detailed instructions anywhere.In addition to the wiki page waterwingz linked, here are some other useful references
What CHDK build is recommended for this?The 1.1 stable build should work fine. The 1.2 trunk should also work, but doesn't currently add any additional PTP functionality.
One oddity I noticed is that the jpeg code will sometimes return a zero size chunk as the last chunk. This isn't specifically a problem by itself, but I'm not sure how it crops up.Looks like a check inside filewrite_get_jpeg_chunk() was not complete. Following patch should correct that.
Index: platform/generic/filewrite.c
===================================================================
--- platform/generic/filewrite.c (revision 2877)
+++ platform/generic/filewrite.c (working copy)
@@ -34,6 +34,9 @@
return 0; // last chunk
}
}
+ else {
+ return 0; // last chunk
+ }
return 1; // not last chunk
#else
if ( jpeg_chunks == NULL ) { //do we have a valid queue?
You can now use the continuous mode remotecap with bracketing if you want.I was afraid of continuous mode when remote shooting, so I never even tried that before. Now, after seeing what a dryos r51 camera is doing, it looks like there's a caching layer between filesystem operations (open, close, read, write) and real card accesses. The whole thing can change if Canon decides to cache the raw data before/during processing, which is not currently happening on P&S models AFAIK.
This was the last significant code reorganization I wanted to do before merging to the trunk.Thanks for doing that.
Being able to turn off jpeg is something that has been requested in the past, unrelated to remote capture. So it might be tempting to make it an option and/or scriptable setting.Makes sense, although I think that throwing away the jpeg in favour of dng is not a good idea for the average shooter.
I can also imagine situations where someone would want to use remoteshoot for some data and keep some on the card.I think there may be problems with simultaneous sd card AND usb transfer (in a series of early tests nafraf's a810 was acting unstable when I managed to enable file operations while remote shooting on that test build), so we should avoid that.
Right now, if you request only dng header, all the image data is thrown away. OTOH, maybe people who data on the card should just use shoot+download?That should still work, so yes. I think Canon's official remote shooting support included both scenarios (remote file only, remote+local file).
The timeouts for the camera waiting for the PTP client to fetch data are currently hard coded. Does this need to be adjustable?I can imagine situations where it could be needed (slow usb host, slow transfer, 16MP raw + jpeg).
One more thing I've been thinking about: Is it worth sending the filename? It would be simpler to just send the exposure number as a param with each chunk (or as part of the RemoteCaptureIsReady response?), and that's essentially all we are using right now anyway. It would include the directory name/number, but that can be obtained lua if someone really wants it.Makes sense, one round trip less. I originally thought that this task is responsible for writing native raw too, but that assumption turned out to be false (according to an experiment done by SticK last year).
Of course. I was a bit surprised that continuous mode worked, but it does on my cams at least. I think in general it should work with raw, because we hold up the whole shooting process until it's done. jpeg... I'm less sure.You can now use the continuous mode remotecap with bracketing if you want.I was afraid of continuous mode when remote shooting, so I never even tried that before. Now, after seeing what a dryos r51 camera is doing, it looks like there's a caching layer between filesystem operations (open, close, read, write) and real card accesses. The whole thing can change if Canon decides to cache the raw data before/during processing, which is not currently happening on P&S models AFAIK.
Agreed. To save space, you could just use a small size / low quality jpeg. Time saved by skipping jpeg isn't very significant. People have asked for it a few times, but maybe it doesn't need to be added with remotecap.QuoteBeing able to turn off jpeg is something that has been requested in the past, unrelated to remote capture. So it might be tempting to make it an option and/or scriptable setting.Makes sense, although I think that throwing away the jpeg in favour of dng is not a good idea for the average shooter.
I can imagine situations where it could be needed (slow usb host, slow transfer, 16MP raw + jpeg).Good point. I'll add this. I think one timeout value for all the phases is sufficient.
On the other hand, nothing requires the client to ask for the file name.QuoteOne more thing I've been thinking about: Is it worth sending the filename? It would be simpler to just send the exposure number as a param with each chunk (or as part of the RemoteCaptureIsReady response?), and that's essentially all we are using right now anyway. It would include the directory name/number, but that can be obtained lua if someone really wants it.Makes sense, one round trip less.
I'm a bit concerned about calling get_target_file_num() directly in PTP_CHDK_RemoteCaptureIsReady. Some cameras have a flaky counter (or the implementation of getting that number is not optimal), file numbering may become less reliable (my method was to store the counter at the start of the raw hook).You're right, it probably shouldn't call this when no datatypes are available yet. I'll change this in the next version.
I'm currently in process of re-organizing the hook wait / timeout / cancel code, since after I implemented the adjustable timeout I found the behavior wasn't well defined. I'll probably break something...This is done (or at least started...) in changeset 2917 (http://trac.assembla.com/chdk/changeset/2917)
=set_remotecap_timeout(10)
rs
This should shoot a shot without any delay and return something like=set_remotecap_timeout(10000)
=init_remotecap(1) ; shoot()
This should shoot, and then wait 10 seconds, doing whatever LED activity would be normal when writing a jpeg.=set_remotecap_timeout();init_remotecap(1);press('shoot_half');repeat sleep(10) until get_shooting();click('shoot_full');sleep(3000);init_remotecap(0)
This should shoot, wait 3 seconds and then finish. No files should be written.Calling after the first one becomes available should be OK, since that always has to happen after the raw hook is called.This counter is currently only used for raw (in vanilla CHDK), and is probably 'optimized' for that. I think that on at least a few Vx ports a delay of a few tens of msecs will produce a much less reliable result (you can take a look at the respective platform function of my ports for example (a410..) and the note beside it)
long get_file_next_counter() { //looks like this hack is needed for old vxworks
return ((get_file_counter()>>4)+1)<<4;
}You could put in a duplicate file (or counter) check in the client - at least for debug reasons.PTP_CHDK_RemoteCaptureIsReady currently returns a special value if you call when no target is selected. I wonder if it should use PTP_RC_GeneralError instead. On the other hand, remotecap can get turned off by various errors, so maybe it should be different.Since errors usually produce PTP_RC_GeneralError, maybe this one should also. Would it be possible (at some point) to return a more descriptive error sub-code?
The ptp task will continue transferring data, which could potentially get corrupted. There will be no error status, unless the client expected to get more chunks or data types. I'm not really sure how we could make this safe. I don't think this is likely to crash or corrupt memory on the cam, but I could be wrong.I don't have a clear overview over this ATM, but:
It's also exposed in script as get_exp_count(). But you've convinced me, I'll save it in the raw hook so it will behave the same as raw.Calling after the first one becomes available should be OK, since that always has to happen after the raw hook is called.This counter is currently only used for raw (in vanilla CHDK), and is probably 'optimized' for that. I think that on at least a few Vx ports a delay of a few tens of msecs will produce a much less reliable result (you can take a look at the respective platform function of my ports for example (a410..) and the note beside it)
Since errors usually produce PTP_RC_GeneralError, maybe this one should also. Would it be possible (at some point) to return a more descriptive error sub-code?Good idea. Something to look into.
I don't have a clear overview over this ATM, but:The current code gets the address and size once for each chunk. If the timeout expires while that chunk is being transferred, the transfer will continue using that address/size (which may no longer contain valid data). If the chunk wasn't the last chunk, it will get an error because the remotecap state has been cleared.
In case of jpeg, the chunk descriptor structure is part of the filewritetask's message structure. If the task continues, it may get corrupted. We could either make a copy of it and use that, or we could plant some kind of an 'emergency brake' ... uhmm ... somewhere. We never send information about the file size, so it's up to the user to recognize a truncated file...
Can't help with r50+, but your examples appear to work on the a470.Nafraf reported it worked on sx160. But I'm thinking that the situation above may have additional problems, because there could be calls to the filewrite hook after the remotecap state is cleared... but maybe it's ok because the ignore write flag is still set? I'll have to look at this again.
You have a comment in remotecap.c:I'm not sure I follow. This would get called from the ptp task at the end of it's chunk. Meanwhile, filewritetask is waiting. When filewritetask is finished waiting, it will immediately set jpeg_chunks null. I guess there could be another call from ptp before it gets there?
remotecap_jpeg_chunks_done(); // make jpeg_chunks NULL, immediately. TODO needed?
This was (and probably is) needed for r50+ cameras, due to multitasking issues (can't be done from another task, because it's too late by then).
The current code gets the address and size once for each chunk. If the timeout expires while that chunk is being transferred, the transfer will continue using that address/size (which may no longer contain valid data). If the chunk wasn't the last chunk, it will get an error because the remotecap state has been cleared.OK then, I thought that it can't halt in the middle of a certain datatype. In this case, the user will get corrupted/truncated file.
Nafraf reported it worked on sx160.That's good :)
But I'm thinking that the situation above may have additional problems, because there could be calls to the filewrite hook after the remotecap state is cleared... but maybe it's ok because the ignore write flag is still set?To prevent
I'm not sure I follow. This would get called from the ptp task at the end of it's chunk. Meanwhile, filewritetask is waiting. When filewritetask is finished waiting, it will immediately set jpeg_chunks null. I guess there could be another call from ptp before it gets there?Can't recall what exactly happened, but this (https://trac.assembla.com/chdk/changeset/2243) change was the solution (the commit is a bit noisy).
In this case, the user will get corrupted/truncated file.Yes. Not good, but better than crashing. I'm tempted to add a timeout flag, so we can check after the send_data is complete and set a ptp error. Or maybe check if the data available flag is cleared? I'm worried it will get too complicated...
I think the 'current_write_enabled' flag should take care of this, as it only gets cleared when all data chunks are 'written' from the filewritetask's perspective.Yeah, I think that is right.
connected: Canon PowerShot A810, max packet size 512
con> rec
con 1> =set_prop(require("propcase").FLASH_MODE,2)
con 2> rs -jpg
con 5> =set_prop(require("propcase").FLASH_MODE,1)
con 6> rs -jpg
ERROR: timed out
A495 works without problems in both cases. Am I missing something?
I tested on A810, without flash, remote capture works. If flash enabled it returns timeout. Same behavior on sx160. It seems that if flash is not ready, capture is interrupted.I had similar experience on some cameras, see http://chdk.setepontos.com/index.php?topic=4338.msg95368#msg95368 (http://chdk.setepontos.com/index.php?topic=4338.msg95368#msg95368) .
Can the raw hook be entered for the next shot before the jpeg from the previous is downloaded?This should probably be tested. Cameras with 2 raw buffers could in theory take advantage of the extra buffer, but... wouldn't that kind of parallelism already cause problems with CHDK raw/dng? We block filewritetask before its "open" stage, so anything that would signal "next shot is available" from there is also blocked. BTW, there are event procedures that could be used for testing (one or more of those around 'completefilewrite').
I tested on A810, without flash, remote capture works. If flash enabled it returns timeout. Same behavior on sx160. It seems that if flash is not ready, capture is interrupted.Does shooting raw/dng without using PTP work in this situation?
press('shoot_half')
repeat sleep(10) until get_shooting()
click('shoot_full')
I can confirm this happens on both a540 and d10. Both only have a single raw buffer, so I guess this means the final jpeg buffer does not overlap the raw buffer. The canon firmware is happy to start a new exposure before the previous one has finished writing.Can the raw hook be entered for the next shot before the jpeg from the previous is downloaded?This should probably be tested. Cameras with 2 raw buffers could in theory take advantage of the extra buffer, but... wouldn't that kind of parallelism already cause problems with CHDK raw/dng? We block filewritetask before its "open" stage, so anything that would signal "next shot is available" from there is also blocked. BTW, there are event procedures that could be used for testing (one or more of those around 'completefilewrite').
Index: D:/chdk/remotecap4/core/remotecap.c
===================================================================
--- D:/chdk/remotecap4/core/remotecap.c (revision 2921)
+++ D:/chdk/remotecap4/core/remotecap.c (working copy)
@@ -133,6 +133,16 @@
extern long hook_raw_size(void); // TODO should use camera_sensor, but see note on size mismatch!
void remotecap_raw_available(char *rawadr) {
+// temp sanity check
+ while(remotecap_get_available_data_type()) {
+ int i;
+ for(i=0; i<10; i++) {
+ started();
+ msleep(250);
+ finished();
+ msleep(250);
+ }
+ }
// TODO this should probably just be noop if hook doesn't exist
#ifdef CAM_HAS_FILEWRITETASK_HOOK
filewrite_set_discard_jpeg(1);
In chdkptp:con 1> rec
con 2> =return init_remotecap(1)
3:return:true
con 3> .shoot(); press('shoot_half'); repeat sleep(10) until get_shooting(); click('shoot_full') sleep(100)
I will add the timeout logic I described earlier.Does shooting raw/dng without using PTP work in this situation?Yes, it works.
How about a (non-ptp) script that just doesIt does not work. but changing sleep(10) to sleep(500) it works. ???Code: [Select]press('shoot_half')
repeat sleep(10) until get_shooting()
click('shoot_full')
It does not work. but changing sleep(10) to sleep(500) it works. ???
press('shoot_half')
repeat sleep(10) until (get_flash_ready() and get_shooting())
click('shoot_full')
On my a540 get_flash_ready() only goes true if the flash is enabled and would fire, so "real" usage requires a more complicated check.
This one works better (using the a410 this time), but still not completely reliable.Code: [Select]press('shoot_half')
repeat sleep(10) until (get_flash_ready() and get_shooting())
click('shoot_full')
This one works better (using the a410 this time), but still not completely reliable.What if you use a
I think this issue can be simulated by hand, the camera sometimes fails to notice short shutter presses when flash is involved. It's also camera dependent, happens only rarely on the a470.
What if you use aIt doesn't shoot (so it's worse than the previous script). It's still the a410, and I'm commanding it through ptp. Increasing the delay increases the success rate (1000 ms sometimes works, 5000ms probably always works...).
press('shoot_full')
sleep(500)
release('shoot_full')
It doesn't shoot (so it's worse than the previous script). It's still the a410, and I'm commanding it through ptp. Increasing the delay increases the success rate (1000 ms sometimes works, 5000ms probably always works...).In case it wasn't clear, I meant the above instead of click('shoot_full') in the previous code.
In case it wasn't clear, I meant the above instead of click('shoot_full') in the previous code.I see. That seemed to have increased the success rate, but still tends to fail when it wakes the camera from 'sleep'. I wonder what nafraf's cameras will do.
I'm testing with script. 15 shoots on each camera (a810, sx160) 100% worked.In case it wasn't clear, I meant the above instead of click('shoot_full') in the previous code.I see. That seemed to have increased the success rate, but still tends to fail when it wakes the camera from 'sleep'. I wonder what nafraf's cameras will do.
- luaCB_get_flash_ready() is calling shooting_is_flash(); but should be calling shooting_is_flash_ready()Ah, that makes sense.
Requests for filewrite support on specific cameras are also welcome.Can you please make a filewrite support for a590 101b, when you have free time.
edit:Checking PROPCASE_IS_FLASH_READY on a540 and D10, it appears to behave pretty much as expected. I would expect there's some situations where the additional checks in shooting_is_flash_ready() are required.
Ugh, it gets more complicated:
shooting_is_flash returns:
shooting_get_prop_int(PROPCASE_IS_FLASH_READY), which is what the old ubasic code did.
shooting_flash_is_ready checks more stuff.
Ugh, it gets more complicated:
shooting_is_flash returns:
shooting_get_prop_int(PROPCASE_IS_FLASH_READY), which is what the old ubasic code did.
shooting_flash_is_ready checks more stuff.
* Make PTP not use 50% of available RAM for buffer if there's lots available.I did some investigation. Conclusion is basically that download performance is better with larger buffer size, up to at least 1mb or so. I tested both downloading a DNG file (which does file IO in units of the "buffer size") and using GetMemory (which still sends in "buffer size" units, even though it doesn't do any actual buffering)
set cli_xferstats=true
set cli_time=true
To adjust the (max) size of the buffer=set_usb_buf_max(1024*1024)
note it will still be capped at 1/2 ram, so you need to check how much you have free. For 1mb or more, exmem is recommended...d -nolua DCIM/100CANON/CRW_nnnn.DNG
!con:getmem(0x1900,1024*1024)
I did notice that remote capture is using send_ptp_data (which always writes in buffer_size chunks) rather than calling data->send_data like live view. I suspect it could be sped noticeably using the latter, unless it all has to fit in some internal buffer.I changed this to use data->send_data, which makes dng remoteshoopt about 200ms quicker on my D10.
I think the original reason for send_ptp_data using fixed data size was on the theory that it might have a limited size buffer somewhere, and not know what to do with larger values.:-[ Actually, I don't know why I used that call. It was there since my first implementation. I probably copy-pasted some lines, and hadn't noticed the possible adverse effects. So, thanks for fixing it.
get data 16
rc file IMG_7576.jpg 1
rc chunk get IMG_7576.jpg 1 0
rc chunk size:7680 offset:nil last:false
rc chunk get IMG_7576.jpg 1 1
rc chunk size:1536116 offset:nil last:true
time 8.6325 0.4710
get data 17
rc file IMG_7578.jpg 1
rc chunk get IMG_7578.jpg 1 0
rc chunk size:7680 offset:nil last:false
rc chunk get IMG_7578.jpg 1 1
rc chunk size:1533559 offset:nil last:true
time 9.4095 0.7770
get data 18
rc file IMG_7578.jpg 1
rc chunk get IMG_7578.jpg 1 0
rc chunk size:0 offset:nil last:true
ERROR: d:\chdk\chdkptp\lua\chdku.lua:1015: invalid offset
Each get data should be an image. Note that 16 is image #7576, 17 is #7578 (skipping one) then 18 is 7578 again, but zero sized with last set, which triggers the actual error. The actual error is caused by lbuf:fwrite trying to write a zero sized chunk. This is likely not related to the actual problem, but may have been missed before because it used to allow 0 sized chunks.con 89> rs
time 21.3564
ERROR: timed out uninit a script is already runningEvery command sent after that gets the script "already running error" and no photo is taken. After a power off/on the camera works again.
To reproduce the problem, just run separate rs commands many time in a row.I had experienced something similar on the Ixus870/SD880. That camera has relatively small available memory. When I was attempting to use rs from a script, it failed in a few shots due to excessive memory consumption (something ate up the available RAM, without releasing it). With delays (1-2 sec?) between the rs attempts, the gradual memory leak seemed to vanish. I don't know whether your problem has anything to do with this though.
When I was attempting to use rs from a script, it failed in a few shots due to excessive memory consumptionCould you do rs commands after the error or did you have to power the camera off and on first?
Could you do rs commands after the error or did you have to power the camera off and on first?The memory was permanently lost, I think the cam usually crashed when the rs cycle was left running.
remote shoot in continuous mode seems to fail frequently on a540.I'm afraid I can't be much help, but I suspect that the cause is a task synch issue - something isn't happening in the expected order.
I'm afraid I can't be much help, but I suspect that the cause is a task synch issue - something isn't happening in the expected order.No problem. I just posted here so people can be aware of the issue. I'll try to have a closer look this weekend.
Try to print the available camera memory after every shot.When I after each photo run this (found earlier in this thread)
!return con:execwait('return get_meminfo()')I get this con 142>
=true,{
start_address=1363912,
chdk_start=2212572,
free_size=204752,
chdk_size=147072,
allocated_size=996624,
total_size=1201376,
allocated_peak=999872,
name="system",
allocated_count=1364,
chdk_malloc=true,
free_block_max_size=202696,
end_address=2565288,
free_block_count=15,
}The values doesn't seem to change. They are the same right before the error appears as they were many shots earlier. Is there some other memory command I should use?
I have done some (not proper) testing. A410 (USB speed limited), S size images (around 64kB each).I've had instances like this where the script didn't start shooting at all, I'm not clear why it happens yet. I don't think it's directly remoteshoot related. The current remoteshoot camera side script in chdkptp doesn't handle this well either, I'll try to check in a better behaved one in a bit.
I did get errors like
> rs /tmp -cont=1000
WARNING: timed out waiting for shot script
wait time 31,3551
# truncate the existing stdout
=f=io.open('A/stdout.txt','wb');f:write('init\n');f:close()
# make the camera log buffer larger, so we get a reasonable number of lines
# should return 3 zeros if everything worked. ShowCameraLogInfo records the new camera log config in stdout
=return call_event_proc('StopCameraLog'),call_event_proc('StartCameraLog',0x20,0x4000),call_event_proc('ShowCameraLogInfo')
# shoot
rec
rs -cont=20
# download the ouput
d stdout.txt
> ... my log wasn't big enough to see the start of this shot
00150350: DSI_ControlShootInfo( 0x44, 0 )
00150360: LogicalEvent:0x311e:adr:0x0,Para:0
> filewrite hook signals image 107 is available
00150370: jpg av 107 A/DCIM/106CANON/IMG_0107.JPG ign 1
> raw hook signals raw 108 is available
00150590: raw av 108
> raw hook sees jpeg is still pending, so it stalls
00150590: raw av pend fmt 1 ign 1
> jpeg junk requested
00150630: rc chunk 1 ign 1
> completed with status "more chunks"
00150640: rc send comp 1 status 1 ign 1
00150650: rc chunk 1 ign 1
> second chunk completed, status last chunk, this clears the pending flag
00150660: rc send comp 1 status 2 ign 1
> raw hook sees pending flag clear after 6 sleep iterations,
00150670: raw av pend done wait 6 fmt 0 ign 1
> raw hook exits, this has called filewrite_set_discard_jpeg(1) for shot 108
00150670: raw av done no raw
> filewrite hook finally notices it is done with 107. After this, it will clear ignore_current_write!
00150720: jpg av done 108 A/DCIM/106CANON/IMG_0107.JPG ign 1
00150750: DSI_ControlShootInfo( 0x27, 0 )
> ... next shot (#109)
00150830: SS:ShootPicture
00150840: DSI_ControlShootInfo( 0x43, 0 )
00150850: ShootSeqToUI:0x2008:adr:0x272040,Para:2564160
00150850: _EntryPrepareRecreviewOff
00150850: ShtCon_StopReview
00150850: StopRecReviewController
00150860: SS:StopRecreview
00150860: LogicalEvent:0x3120:adr:0x0,Para:0
00150860: ShtCon_StartReview
00150860: SS:StartRecreview
00150870: ShootSeqToUI:0x2001:adr:0x211470,Para:2167920
00150880: ShootSeqToUI:0x2021:adr:0x0,Para:0
00150880: LogicalEvent:0x311f:adr:0x0,Para:0
00150880: Sht_CancelStrobeChargeTimer
00150880: DSI_ControlShootInfo( 0x2b, 0 )
00150880: DSI_ControlShootInfo( 0x44, 0 )
00150890: LogicalEvent:0x311e:adr:0x0,Para:0
> filewrite signals image 108 available, but notice the ignore write flag is zero. IMG_0108 was saved on the card
00150900: jpg av 108 A/DCIM/106CANON/IMG_0108.JPG ign 0
00151040: rc chunk 1 ign 0
00151050: rc send comp 1 status 1 ign 0
00151060: rc chunk 1 ign 0
00151080: rc send comp 1 status 2 ign 0
00151080: jpg av done 108 A/DCIM/106CANON/IMG_0108.JPG ign 0
> filewrite hook ended quickly this time, before raw hook
00151120: raw av 109
00151120: raw av done no raw
The solution to this seems to be to clear the jpeg pending flag from filewrite hook, after ignore_current_write is cleared.
Test code with proposed fix attached.Thanks, will test when I get some time.
Could it be that the wrapping happens earlier than 9999?From what I understand the canon firmware can create a new folder before 9999 (depending on number of images present, I think?)
From what I understand the canon firmware can create a new folder before 9999 (depending on number of images present, I think?)I don't know. This amount of pictures could be unexpected by the fw, thus the weird numbering?
It's interesting (and nice) that this seems to avoid running into limit where having ~1000 images on the card kills Canon PTP...I wonder how. Perhaps I should test non-continuous mode too.
nafraf sent me compiled test versions for A490 and A2300 that included reyalp's remotecap-cont-debug-8.patch However the problem with rs not working after a variable number of photos is still there in tests with both cameras.The debug patch doesn't have any fixes that aren't in the trunk, it just has some debug logging calls, and to use those, you need DEBUG_LOGGING defined in platform.h have the _LogPrintf function found.
Based on my testing and feedback from nafraf in irc, I've checked a version of this without the debug code (+ fixed vxworks compile error in the previous patch) into the trunk changeset 3026 (http://trac.assembla.com/chdk/changeset/3026). I'd like to get a bit more testing before moving it over to the release branch.I merged this and the a490 changes into the 1.2 branch in changeset 3034 (http://trac.assembla.com/chdk/changeset/3034)
From the multi-thousands of images I've taken with my time lapse script, I've figured out the number scheme, at least on all my cameras.Could it be that the wrapping happens earlier than 9999?From what I understand the canon firmware can create a new folder before 9999 (depending on number of images present, I think?)
I'm not sure if this resets numbering. Honestly I'm not totally clear how all this is supposed to work, and it's a bit annoying to find out by testing.
If you are referring to the error described in http://chdk.setepontos.com/index.php?topic=4338.msg103854#msg103854 (http://chdk.setepontos.com/index.php?topic=4338.msg103854#msg103854) this seems like it is just the shooting part of the script failing to take a shot and getting stuck (discussed earlier with the flash related issues). This shouldn't happen with the current chdkptp code in svn, though the shot would still fail. I'll try to get a new chkdptp snapshot build up soon.Post when you do and I'll try it. nafraf helped me narrow down the problem a bit more. It only appears when sending separate rs commands in a row. So doing rs -jpg pretty rapidly over and over will cause the problem sooner or later. But rs -jpg -cont=200 works fine. But I wish to send single photo rs -jpg commands and vary the time inbetween each command somewhat so -cont=200 won't do.
Hi, I have tested the new chdkptp-r426 with the A2300. The problem with timeout after a variable number of rs commands remains. I get error "WARNING: capture_get_data error timed out".This means that chdkptp never saw remote capture data become available.
I then have to restart the camera to get any commands to work again.What happens if you try to use other commands? Is it the "script is already running" error, or something else?
It looks like thatNative RAW files are also written by filewritetask. In RAW + JPEG case, native RAW is saved first - the code doesn't currently handle this, rs -jpg will simply retrieve the native RAW file and let the camera save the JPEG to the card.
- filewritetask only writes jpeg files (!), native raw is written elsewhere
I have to correct my earlier statement.Late reply :-[ I guess it would be nice to support this. Right now we don't have a consistent way to tell if a port supports canon raw, or if it is enabled. We'd also need a way of selecting Canon raw for remote shoot, while running through the same code jpeg.QuoteIt looks like thatNative RAW files are also written by filewritetask. In RAW + JPEG case, native RAW is saved first - the code doesn't currently handle this, rs -jpg will simply retrieve the native RAW file and let the camera save the JPEG to the card.
- filewritetask only writes jpeg files (!), native raw is written elsewhere
It seems you're right. Fwt data blocks below, sampled in filewrite_main_hook(), a3200First chunk, but last=true?This sounds like it could match multiple invocations of filewrite, where the first "session" only has one chunk.
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 09 00 00 00 00 00 00 00 01 03 00 00 9A BB 24 53 ............š»$S
00000010 00 00 20 09 00 64 A6 43 00 E4 13 00 00 F4 74 43 .. ..d¦C.ä...ôtC
00000020 B9 36 1D 00 00 00 00 00 00 00 00 00 00 00 00 00 Ä…6..............
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000050 41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F A/DCIM/219___03/
00000060 49 4D 47 5F 30 30 32 38 2E 4A 50 47 00 00 00 00 IMG_0028.JPG....
Bad JPEG, 4 fwt invocationsOffset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 09 00 00 00 00 00 00 00 01 83 00 00 A8 BB 24 53 ............¨»$S
00000010 00 00 20 09 00 F4 74 43 00 AA 22 00 00 00 00 00 .. ..ôtC.Ş".....
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000050 41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F A/DCIM/219___03/
00000060 49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00 IMG_0029.JPG....
00000000 09 00 00 00 00 00 00 00 09 80 00 00 A8 BB 24 53 .........€..¨»$S
00000010 00 00 20 09 00 A0 97 43 00 A8 22 00 00 00 00 00 .. .. —C.¨".....
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000050 41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F A/DCIM/219___03/
00000060 49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00 IMG_0029.JPG....
00000000 09 00 00 00 00 00 00 00 09 80 00 00 A8 BB 24 53 .........€..¨»$S
00000010 00 00 20 09 00 F4 74 43 49 FB 09 00 00 00 00 00 .. ..ôtCIű......
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000050 41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F A/DCIM/219___03/
00000060 49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00 IMG_0029.JPG....
00000000 09 00 00 00 00 00 00 00 01 00 00 00 A8 BB 24 53 ............¨»$S
00000010 00 00 20 09 00 A0 97 43 00 40 00 00 00 00 00 00 .. .. —C.@......
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 ................
00000050 41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F A/DCIM/219___03/
00000060 49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00 IMG_0029.JPG....The trick is that the first chunk starts with garbage, the exif block will be written there - in the final fwt invocation for that file.Right now we don't have a consistent way to tell if a port supports canon raw, or if it is enabled. We'd also need a way of selecting Canon raw for remote shoot, while running through the same code jpeg.'Native RAW support' will probably be a define, there are at least 2 propcases that have to do with native raw / raw+jpeg selection.
Proposed fix for cameras with multipass JPEG writes.I'm still getting partial JPEGs occasionally, it's either an unhandled saving pattern or a bug... :(
It seems like it might be possible to clean up the ifdef code a bit so there is less duplicated code. It might also be clearer to have CAM_FILEWRITETASK_SEEKS automatically define multipass, so the common parts only check one ifdef.Will do this in the next patch.
It was a bug. New (still incomplete) patch is attached, along with a new test build for that camera.Proposed fix for cameras with multipass JPEG writes.I'm still getting partial JPEGs occasionally, it's either an unhandled saving pattern or a bug...
It was a bug. New (still incomplete) patch is attached, along with a new test build for that camera.Patch tested on A495, I did not find problems.
Turns out, there's still something wrong. Last chunk is not recognized when the cam is set to continuous mode...It looks like this can't be solved easily. The code is working when in single shot mode. In continuous mode, there is simply nothing in fwt_data_struct that would indicate whether there will be further fwt invocations for a given file. Therefore, the current code always expects another invocation and causes every rs attempt to fail
In continuous mode, there is simply nothing in fwt_data_struct that would indicate whether there will be further fwt invocations for a given file.:(
- fail every rs attempt in continuous mode for these camsBetter than nothing I guess, but unfortunate, because rs in continuous is actually pretty useful for a some applications.
- add code and fw hooks to better deal with the issue (this could end up pretty hacky)Not sure what this entails? An example of what it looks like when it fails might be helpful.
- rewrite the code to mirror filesystem operations (ouch)Does that even help if we don't know when it's done? Maybe I'm not understand the the suggestion.
small fileQuote- add code and fw hooks to better deal with the issue (this could end up pretty hacky)Not sure what this entails? An example of what it looks like when it fails might be helpful.
con 19> rs
rs_init
rs_shoot
get data 1
rc file IMG_0197.jpg 1
rc chunk get IMG_0197.jpg 1 0
rc chunk size:321042 offset:nil last:false
rc chunk get IMG_0197.jpg 1 1
RemoteCaptureGetData: unexpected return code 0x2002
WARNING: capture_get_data error rcgetchunk failed
script wait time 1.5299
ERROR: rcgetchunk failed
large file, fwt (and its related tasks) locked after thiscon 15> rs
rs_init
rs_shoot
get data 1
rc file IMG_0207.jpg 1
rc chunk get IMG_0207.jpg 1 0
rc chunk size:2271744 offset:nil last:false
rc chunk get IMG_0207.jpg 1 1
rc chunk size:2271232 offset:nil last:false
rc chunk get IMG_0207.jpg 1 2
rc chunk size:1473745 offset:nil last:false
rc chunk get IMG_0207.jpg 1 3
rc chunk size:16384 offset:nil last:false
rc chunk get IMG_0207.jpg 1 4
RemoteCaptureGetData: unexpected return code 0x2002
WARNING: capture_get_data error rcgetchunk failed
script wait time 1.5360
ERROR: rcgetchunk failed
etc...fwt doesn't seek in this camera generation. It opens and closes the file multiple times (either in append or in overwrite mode) when needed.Quote- rewrite the code to mirror filesystem operations (ouch)Does that even help if we don't know when it's done? Maybe I'm not understand the the suggestion.
Other ideas:As far as I know, fwt doesn't have information about the total file size. It may exist somewhere else, didn't try to check yet...
Is the total size available somewhere else in memory so we can check if we are done?
Could we use a timeout to signal the last chunk? If we are waiting for a jpeg, what is the max time between one chunk completing and the next being available? It seems like it could potentially be pretty short.A timeout was one of my ideas that I referred to as "hacky". Other "hacks" include signaling from the raw hook.
Reverse engineer the code that sends the chunks, there must be some loop or variable that knows when it's done.I gave up on understanding "filescheduletask", it looks much more difficult than the filewritetask code.
Here's an a490 100f build for jpq5588
By the way, what is the command to apply .diff file to a local copy of a trunk ? (with chdk-shell win32 and gcc)I don't use that shell, but it has a "source tools" button. "patch" should be available from there.
By the way, what is the command to apply .diff file to a local copy of a trunk ? (with chdk-shell win32 and gcc)I don't use that shell, but it has a "source tools" button. "patch" should be available from there.
Attached the 'completed'(= for all affected cams) patch for this problem - which is, as the last posts show, only a partial fix. The patch may not apply cleanly, as it's a bit dated.
- CAM_FILEWRITETASK_MULTIPASS is not needed on the G12, only one call to 'filewrite_main_hook' is made per file (regardless of mode)"multipass" is relatively hard to trigger on my a3200, a very detailed scene and/or superfine quality override is needed for it to happen.
- OFLAG_APPEND (not sure this is a good name) is always set in continuous mode, and never set in single shotthe name (which can be changed) is based on the variable's single mode behaviour, I only noticed the continuous mode ill-behaviour later
- 'fwt_close' is never called in continuous mode, this seems to be a major source of problems and lockupsThere are like 3 different places in the Canon code to close the file, continuous mode uses a different one.
Might also depend on camera resolution the G12 is only 10MP compared to the A3200 14MP.- CAM_FILEWRITETASK_MULTIPASS is not needed on the G12, only one call to 'filewrite_main_hook' is made per file (regardless of mode)"multipass" is relatively hard to trigger on my a3200, a very detailed scene and/or superfine quality override is needed for it to happen.
Quote- OFLAG_APPEND (not sure this is a good name) is always set in continuous mode, and never set in single shotthe name (which can be changed) is based on the variable's single mode behaviour, I only noticed the continuous mode ill-behaviour laterQuote- 'fwt_close' is never called in continuous mode, this seems to be a major source of problems and lockupsThere are like 3 different places in the Canon code to close the file, continuous mode uses a different one.
Thanks for the notes, btw.
Might also depend on camera resolution the G12 is only 10MP compared to the A3200 14MP.Could be, or some related Canon code can be different. My identification was based on the presence of '0x8000' in that part of the fwt code. I managed to make a multipass JPEG at the camera's 7MP setting - I'm using a fine black/white grid displayed on the monitor to torture the JPEG engine.
In limited testing today it seems to work ok - except after using chdkptp 'rs' or 'rsint' the camera crashes when switching to playback mode with 'unidentified image' displayed.I have updated nafraf's table (http://chdk.wikia.com/wiki/User:Nafraf/RemoteShootIssues).
Might also depend on camera resolution the G12 is only 10MP compared to the A3200 14MP.Could be, or some related Canon code can be different. My identification was based on the presence of '0x8000' in that part of the fwt code. I managed to make a multipass JPEG at the camera's 7MP setting - I'm using a fine black/white grid displayed on the monitor to torture the JPEG engine.
So the problem is how to determine if this is a single or multi-pass file saveNo, the problem is with continuous mode (http://chdk.setepontos.com/index.php?topic=4338.msg112314#msg112314). My last patch is already able to deal with single shot mode (at least I believe so).
So the problem is how to determine if this is a single or multi-pass file saveNo, the problem is with continuous mode (http://chdk.setepontos.com/index.php?topic=4338.msg112314#msg112314). My last patch is already able to deal with single shot mode (at least I believe so).
However, if you don't ever get a multipass JPEG, your cam may not be affected by any of these issues. The playback mode crash is a separate thing, unrelated to our fwt code (reyalp mentioned somewhere that it's also happening when JPEGs are erased after shooting).
Thanks for the idea, I'll see whether it can be used in practice.
Side note: The 'ff d8 ff e1' byte sequence appears more than once in JPEGs of some new models (ixus115: twice, sx280: 3 times).
I have to correct my earlier statement.QuoteIt looks like thatNative RAW files are also written by filewritetask. In RAW + JPEG case, native RAW is saved first - the code doesn't currently handle this, rs -jpg will simply retrieve the native RAW file and let the camera save the JPEG to the card.
- filewritetask only writes jpeg files (!), native raw is written elsewhere
A single-pass save would have:A small update on this: I have encountered a variation in which a multipass JPEG starts with a big chunk of the final JPEG (with its exif in place). Of course no flags help with the indication (oflags is 0x8301 for the 1st pass).
- oflags & 0x300 == 0x300
- a valid JPEG header in the first chunk of memory
A multi-pass file save would have:
- oflags & 0x300 == 0x300 on the first invocation
- an in-valid JPEG header in the first chunk of memory
- oflags & 0x308 == 0x008 for each append to the file
- oflags & 0x308 == 0 for the last invocation that re-writes the header
A single-pass save would have:A small update on this: I have encountered a variation in which a multipass JPEG starts with a big chunk of the final JPEG (with its exif in place). Of course no flags help with the indication (oflags is 0x8301 for the 1st pass).
- oflags & 0x300 == 0x300
- a valid JPEG header in the first chunk of memory
A multi-pass file save would have:
- oflags & 0x300 == 0x300 on the first invocation
- an in-valid JPEG header in the first chunk of memory
- oflags & 0x308 == 0x008 for each append to the file
- oflags & 0x308 == 0 for the last invocation that re-writes the header
Is it the correct EXIF for the image (correct segment length and filename)?The question is good. Some of the exif data seems final, but some of its parts are missing: the halfword after the first ff d8 ff e1 is zero, the thumbnail image is not there yet. The "file number" (as exiftool calls it) is there and is correct. Since this was the first photo taken in that session, it may not happen regularly (apparently, chunks of the JPEG are allocated separately).
There's a chance the memory could contain the data from a previous save - which would make things somewhat difficult.Seems unavoidable (unless we erase some sensitive parts after transfer), but wouldn't happen very often.
Another thought I had was maybe we could move the problem to the host computer instead of the camera.I don't know. So much trouble to support this camera generation.
If this still holds, I'd be tempted to say use this code for 1.3 and disable remote capture in continuous mode. It turns out that most of the benefits of continuous mode can be achieved by holding half press and clicking full, so the need for continuous mode is less. Assuming this doesn't also trigger the bug... which I guess we need to test.So the problem is how to determine if this is a single or multi-pass file saveNo, the problem is with continuous mode (http://chdk.setepontos.com/index.php?topic=4338.msg112314#msg112314). My last patch is already able to deal with single shot mode (at least I believe so).
I have since made some changes to my experimental code (plus it's stuffed with debug lines), I'll try to determine which one works better.If this still holds, I'd be tempted to say use this code for 1.3 and disable remote capture in continuous mode. It turns out that most of the benefits of continuous mode can be achieved by holding half press and clicking full, so the need for continuous mode is less. Assuming this doesn't also trigger the bug... which I guess we need to test.So the problem is how to determine if this is a single or multi-pass file saveNo, the problem is with continuous mode (http://chdk.setepontos.com/index.php?topic=4338.msg112314#msg112314). My last patch is already able to deal with single shot mode (at least I believe so).
It turns out that most of the benefits of continuous mode can be achieved by holding half press and clicking full, so the need for continuous mode is less. Assuming this doesn't also trigger the bug... which I guess we need to test.Can you suggest a way to test this situation?
We'll also have to determine somehow which cameras need the fix...We could just apply it to cameras known to suffer from the problem.
The new rs -quick=n option described here http://chdk.setepontos.com/index.php?topic=6231.msg116242#msg116242 (http://chdk.setepontos.com/index.php?topic=6231.msg116242#msg116242) should do it.It turns out that most of the benefits of continuous mode can be achieved by holding half press and clicking full, so the need for continuous mode is less. Assuming this doesn't also trigger the bug... which I guess we need to test.Can you suggest a way to test this situation?
The new rs -quick=n option described here http://chdk.setepontos.com/index.php?topic=6231.msg116242#msg116242 (http://chdk.setepontos.com/index.php?topic=6231.msg116242#msg116242) should do it.Done, the camera does not show the continuous mode anomalies (as expected) when using that command.
We could just apply it to cameras known to suffer from the problem.The patch includes support for these cams: a2200, a3200, a490, a495, ixus1000_sd4500, ixus200_sd980, ixus230_elph310, s90, sx220, sx230, sx30.
it will affect anyone who happens to leave their camera in continuous mode, even if they are only trying to take single shots. Nothing we can do about that, and still better than having it fail completely.We might be able to switch drive mode with UI properties - once/if the necessary research is done.
Should remotecap_set_target only fail if they requested jpeg? Presumably CHDK raw should be fine, but I guess the jpeg would have to be written to the card and filewrite might get more complicated.I did not try, but I think the current approach of suppressing JPEG may fail to work in continuous mode for multipass cams. So, IMHO all targets should fail.
Another question is if it should affect the return value of remotecap_get_target_support. I'm not sure, currently it's returns what the port is capable of regardless of the current state (e.g if you are in playback or movie mode it returns the supported formats)The code could be reorganized by moving the "possibilities" check from the beginning of remotecap_set_target() to remotecap_get_target_support() and making the latter function accept a parameter. remotecap_get_target_support() could then be used for querying both "what does the cam support" and "what does the cam support right now".
it will affect anyone who happens to leave their camera in continuous mode, even if they are only trying to take single shots. Nothing we can do about that, and still better than having it fail completely.We might be able to switch drive mode with UI properties - once/if the necessary research is done.
A495 - Propset 3 - 0x8009
A810/SX160 - Propset 5 - 0x800D
A1400 - Propset 6 - 0x800D
SX170/SX510 - Propset 6? - 0x800E
0=Single, We might be able to switch drive mode with UI properties - once/if the necessary research is done.That would be useful in a lots of other situations too.
I did not try, but I think the current approach of suppressing JPEG may fail to work in continuous mode for multipass cams. So, IMHO all targets should fail.Yeah, makes sense.
The code could be reorganized by moving the "possibilities" check from the beginning of remotecap_set_target() to remotecap_get_target_support() and making the latter function accept a parameter. remotecap_get_target_support() could then be used for querying both "what does the cam support" and "what does the cam support right now".Hmm. That could work. Somewhat tempting just stick with your original patch to keep it simple, but allowing the client to distinguish between not implemented and something that just needs a mode change would be more user friendly. The Lua function could just return both bitmasks, like
Somewhat tempting just stick with your original patch to keep it simpleThat would be my intention for now, the code reorganization can be done later. The check will then have to be extended to disallow rs when custom timer is active - on all cameras.
A495 patch tests.Pretty big files for a 10MP cam and many small chunks. Do small picture sizes / higher compression also work? Are the files OK (including their exif)?
A495 tests: L, M2 and S size. Fine and Superfine quality.Thanks. Unless there's objection, I'll check in the above patch (limited to a3200, a490, a495 for now) in a few days.
JPGs are fine. Logs and exif data in attachment.
Thanks. Unless there's objection, I'll check in the above patch (limited to a3200, a490, a495 for now) in a few days.OK with me.
A495 tests: L, M2 and S size. Fine and Superfine quality.FWIW, when testing it's probably a good idea to use a scene with a lot of detail.
Done: https://www.assembla.com/code/chdk/subversion/commit/3611 (https://www.assembla.com/code/chdk/subversion/commit/3611)Thanks. Unless there's objection, I'll check in the above patch (limited to a3200, a490, a495 for now) in a few days.OK with me.
Hello,Are you trying to use both cameras simultaneously, or does one camera always work and the other not?
I have a setup where I use a Powershot S110 connected to an Odroid, controlled by the chdkptp code. I activate 'remoteshoot -cont'. I have two cameras. In one camera the 'remoteshoot -cont' works fine. In the other I can use the 'remoteshoot' command but when I try to use it in continuous mode (i.e. with the -cont option) the camera makes noise of shooting two images and then stops. No image is transferred to the Odroid. After half a minute the chdkptp throws the following error message:
ERROR: .../chdkptp/lua/chdku.lua:1339: timed out
Any clue why there would be a difference between the cameras?
I have got CHDK 1.3.0-4029 installed and the latest chdkptp version.
Its hard to tell what firmware was on the working camera as it was completely destroyed in a drone crash yesterday :).If you have a photo taken with destroyed camera, it is possible to determine its fw version from exif data.
Btw, the jpeg I used (for the determining the firmware version) was taken while the camera had a CHDK card in it. Can this interfere with the firmware detection?No, it won't interfere.
So now I have only one camera with firmware 102b and CHDK 1.3.0, where 'remoteshoot -cont' doesn't work ('remoteshoot' in non continuous mode works).Out of curiosity, does remoteshoot -quick work?
What CHDK settings can interfere with the operation? And how can I debug this?I don't know any off the top of my head, my suggestion was that if two identical cameras behave differently, compare the settings in both CHDK and the canon firmware.
For example can I change the firmware of the camera and test another port of CHDK?No.
Regarding the settings of the destroyed camera. I don't remember its settings, but from what I remember the camera dial button was set on 'Tv' (or maybe 'Av'). This is strange as the chdkptp docs clearly state that the camera should be set to continuous mode, which is in the 'SCN' dial mode. Maybe this gives some clue?Yes, don't use that scene mode. There is a problem with the terminology, we use the word "mode" for many different things. What we call 'continuous mode' is actually available in many camera modes, including Av, Tv, M, P. The manual calls it "Continuous Shooting", press SET when the camera is in shooting mode and find it in the menu that appears.
I have another question regarding a different model, the Powershot N. We are considering buying it also for the drone. I have posted a question on the 'Powershot N porting thread' but got no replySorry - I guess that I'm as close as you get to the resident expert on the N as I did the port. However I'm currently traveling without my computer or much access to the internet until the second week of March.
. Does anyone have experience with this camera? i.e. I need to know if it is possible, with 'chdkptp', to:I've used the CHDK port for the N extensively with chdkptp. Off hand, I see no reason this will not work. And if it doesn't, we will fix it. Having said that, i can't remember if MF works - there may be a comment in the poriting thread for the N or the large thread on fixing MF for 1.3.0. (sorry - no links- to hard to do on my mobile device)
1) Control the focus (no manual focus in the camera)
2) Control the exposure (no manual mode in the camera)
3) Use it with 'remoteshoot -cont'
Is it possible to set the datetime of the camera (specifically S110) using chdkptp?You can probably do this using event procs (http://chdk.wikia.com/wiki/Event_Procedure). To use event procs, you need "native function calls" in the CHDK miscellaneous menu.
I've tried to set the datetime of the camera Canon S110 using event procs as suggested in the previous comment and received -1.Can you copy and paste the lines from the chdkptp console? Would be useful to know which step failed.
Thank you! Edge overlay is great thing.There is no feature like this in chdkptp. If you have some programming skill, you might be able to implement it. The live view drawing is mostly done with IUP and CD from Lua, see lua/gui_live.lua. You do not need to re-build chdkptp if you modify the Lua code. You can also interactively execute Lua from the console with the ! command.
But I still interest of onion skin in chdkptp or make visible canon SX 230 hs in Dragonframe or Stop motion Pro.
But if it is impossible I'll use edge overlay :)
I haven't any programmer skills :( But I found a way to make something like onion skin using chdkptp and ghostwin. It's not perfect and takes a lot of time to make 1 photo.If there is a simple way chdkptp could make it easier to do what you want, feel free to suggest it in the chdkptp thread http://chdk.setepontos.com/index.php?topic=6231.780 (http://chdk.setepontos.com/index.php?topic=6231.780)
Here photo :)
I am trying to perform remote capture with PowerShot SX170IS (Firmware Version: 1.01, CHDK Version: 1.4.1 and SVN build of CHDK-PTP).Are you using linux? Does the physical camera screen go black when you plug in the camera?
Other commands are working but when I type rec, the chdk-ptp terminal returns error:switch failed and camera hangs. It does not even turn off.
I was wondering if it is possible to remote shoot in burst mode. If yes how?If you mean shoot several shots quickly without re-focusing, you can use -quick=N or -cont=N. N is the number of shots to take. Quick simulates holding the shutter half way and clicking full. This should work on all cameras that support remoteshoot. Cont uses Canon continuous mode. This requires that you set the camera to continuous mode in the Canon UI first, and has problems on some cameras. If you are using quick, you should turn off review in the canon settings.
reyalpI didn't spend much time on it, but it appears to me that the canon send_data function fails on NULL pointer. On G7X, If I take out the param2 == 0 check in PTP_CHDK_GetMemory I still get the same
I am asking you again to enable memory reading from address 0.
d -nolua MEM<start addr>,<length> [memdump.bin]where <start addr> is the starting address in decimal or hex, <length> is the dump length in bytes.Index: core/ptp.c
===================================================================
--- core/ptp.c (revision 4794)
+++ core/ptp.c (working copy)
@@ -495,6 +495,60 @@
free(temp_data.str);
temp_data_kind = 0;
+ // hack, parse filename for special operation (gets camera memory range)
+ // format: MEM<start addr>,<length>
+
+ char *comma = 0;
+ // check for special file name (starts with MEM)
+ if (strstr(fn, "A/MEM"))
+ {
+ comma = memchr(fn+5, ',', temp_data_extra-5);
+ }
+ if (comma)
+ {
+ *comma = 0;
+ unsigned long start = strtoul(fn+5, 0, 0);
+ unsigned long length = strtoul(comma+1, 0, 0);
+
+ // some protection against typos
+ if (length > camera_info.maxramaddr+1)
+ {
+ length = camera_info.maxramaddr+1;
+ }
+
+ buf = (char *) malloc(buf_size);
+ if ( (buf == NULL) || (length == 0) )
+ {
+ // send dummy data, otherwise error hoses connection
+ send_ptp_data(data,"\0",1);
+ ptp.code = PTP_RC_GeneralError;
+ if (buf)
+ {
+ free(buf);
+ }
+ free(fn);
+ break;
+ }
+ tmp = t = length;
+ while (t > 0)
+ {
+ r = (t<buf_size)?t:buf_size;
+ t -= r;
+ memcpy(buf, (void*)start, r);
+ start += r;
+ // cannot use send_ptp_data here
+ data->send_data(data->handle,buf,r,tmp,0,0,0);
+ tmp = 0;
+ }
+ free(fn);
+ free(buf);
+ ptp.num_param = 1;
+ ptp.param1 = length;
+ // we're finished
+ break;
+ }
+ // hack end
+
f = fopen(fn,"rb");
if ( f == NULL )
{
edit:I wasn't convinced of the need for code changes to read address zeroEvery time I work with RAM dumps made with chdkptp I need to recalculate addresses. This is inconvenient.
Quick hack that lets getting camera memory from places that are not PTP friendly (I wanted to get a copy of the 0xbfe10000 TCM content). It uses buffered read. Still not suitable for reading MMIO because it uses memcpy.One slight complication http://pubs.opengroup.org/onlinepubs/009695399/functions/memcpy.html
If copying takes place between objects that overlap, the behavior is undefined.This will obviously happen if you are dumping the whole RAM using a buffered copy. Whether this applies to dryos or actually matters is unclear. Obviously no RAM dump is going to be fully consistent instantaneous snapshot.
TBH, I did this because I needed it, and wanted to get it done fast. As you say, this (in its current form) may not be what Ant wants, because it can mess up the copied version of the Canon heap.QuoteIf copying takes place between objects that overlap, the behavior is undefined.This will obviously happen if you are dumping the whole RAM using a buffered copy. Whether this applies to dryos or actually matters is unclear. Obviously no RAM dump is going to be fully consistent instantaneous snapshot.
TBH, I did this because I needed it, and wanted to get it done fast. As you say, this (in its current form) may not be what Ant wants, because it can mess up the copied version of the Canon heap.Yes, understood and thanks for posting it. I wasn't criticizing the hack, just trying to figure out what to add to the protocol.
Ideally you'd want GetMemory to automatically identify the different regions and use DMA, memcpy or word by word as required, but analyzing the CP15 values at run time seems excessive.Perhaps allowing the user to specify the method would be the optimal solution.
Has there been any progress on the PTP event front beyond mweerden's findings (http://www.mweerden.net/chdk_ptp2.html)?Note that I am aware of.
!chdk.reset_device(con.condev)!con:write_msg(('x'):rep(0x23f4))
If the bug isn't present, you will immediately getERROR: call failed:no script runningERROR: call failed:I/O error
!m=require'extras/msgtest'
!m.test{size=1,sizemax=0x3000,sizeinc=1,gc='step'}
It will stop if it hits the error. I would be interested to know if cameras similar to D10 are affected (DryOS 31, 2009, Digic IV)Tested my r31 and r23 cams. The one affected is the ixus110/sd960 (r31). Unaffected: sx100 (r23), a470 (r23), g10 (r31), ixus870/sd880 (r31). The a470 crashed once, but not at those sizes and I could not reproduce.
Thanks. I assume you were using the msgtest test when a470 crashed? If there's a romlog, I'd be interested to see it.I would be interested to know if cameras similar to D10 are affected (DryOS 31, 2009, Digic IV)Tested my r31 and r23 cams. The one affected is the ixus110/sd960 (r31). Unaffected: sx100 (r23), a470 (r23), g10 (r31), ixus870/sd880 (r31). The a470 crashed once, but not at those sizes and I could not reproduce.
I assume you were using the msgtest test when a470 crashed?Yes. Romlog attached (a470 102c) (edit: date is wrong, of course). My theory is that one of the Canon libraries had a bug that was relatively quickly fixed and only affected a few models.
> r31 cams could be interesting too, after D10, the next oldest I have is SX160 (DryOS 51, digic IV, 2012)a490 (r43), a3200 (r47): no failures, tested range 0x2300...0x3000.
Yes. Romlog attached (a470 102c) (edit: date is wrong, of course).Thanks. That's deep in the Canon PTP code, which suggests it could be an actual PTP problem, rather than say, running out of memory or some other unrelated CHDK issue.
My theory is that one of the Canon libraries had a bug that was relatively quickly fixed and only affected a few models.Or chdkptp is doing something not quite correct that other versions tolerate. There are some weird rules about sending a zero-length packets when the data length is an exact multiple of the max packet size.
00024920: Op.[0x1002], P1[0x00000001]
00024930: Res.Comp
00024930: Op.[0x1001]
00024930: SendData,Len=385
00024930: Res.Comp
00024930: Op.[0x9999], P1[0x00000000]
00024940: Res.Comp
00035250: Op.[0x9999], P1[0x0000000b]
00035250: ReceiveData,Len=9204
00035250: WaitNullPacket fNullEnd = 1, Total=9216, Current=512, SendLen=87
00048970: DoTransactionCanceled()
Op 0x9999 P1 = 0xb is PTP_CHDK_WriteScriptMsg
00028010: Op.[0x9999], P1[0x0000000a]
00028010: SendData,Len=758
00028010: Res.Comp
00028020: Op.[0x9999], P1[0x00000008]
00028020: Res.Comp
00028020: Op.[0x9999], P1[0x0000000a]
00028020: SendData,Len=4
00028030: Res.Comp
00028030: Op.[0x9999], P1[0x00000008]
00028040: Res.Comp
00033370: Op.[0x9999], P1[0x0000000b]
00033370: ReceiveData,Len=9204
00033370: WaitNullPacket fNullEnd = 1, Total=9216, Current=512, SendLen=87
00033370: Res.Comp
00041990: Op.[0x1003]
00041990: Res.Comp
00042050: DoTransactionCanceled()
00044170: DoTransactionCanceled()
ixus110, has the issue:00016630: Op.[0x9999], P1[0x00000008]
00016630: Res.Comp
00016790: Op.[0x9999], P1[0x00000008]
00016790: Res.Comp
00016790: Op.[0x9999], P1[0x0000000a]
00016790: SendData,Len=667
00016790: Res.Comp
00016800: Op.[0x9999], P1[0x00000008]
00016800: Res.Comp
00016800: Op.[0x9999], P1[0x0000000a]
00016800: SendData,Len=4
00016800: Res.Comp
00016810: Op.[0x9999], P1[0x00000008]
00016810: Res.Comp
00022040: Op.[0x9999], P1[0x0000000b]
00022040: ReceiveData,Len=9204
00022040: WaitNullPacket fNullEnd = 1, Total=9216, Current=512, SendLen=87
00036370: DoTransactionCanceled()
00038320: DoTransactionCanceled()
On the Ixus, USB communication after the test message is disrupted. I'm not 100% sure, but I think the first DoTransactionCanceled() appears after the (unsuccessful?) chdkptp disconnection attempt, the second one comes after the USB physical disconnection.00026150: Op.[0x1002], P1[0x00000001]
00026150: Res.Comp
00026150: Op.[0x1001]
00026150: SendData,Len=417
00026160: Res.Comp
00026160: Op.[0x9999], P1[0x00000000]
00026160: Res.Comp
00040640: Op.[0x9999], P1[0x0000000b]
00040640: ReceiveData,Len=9203
00040640: Res.Comp
00051980: Op.[0x9999], P1[0x0000000b]
00051980: ReceiveData,Len=9204
00051980: WaitNullPacket fNullEnd = 1, Total=9216, Current=512, SendLen=87
00051980: Res.Comp
00055710: Op.[0x1003]
00055710: Res.Comp
00055720: DoTransactionCanceled()
00058660: DoTransactionCanceled()
(1001 = PTP_OC_GetDeviceInfo, 1002 = PTP_OC_OpenSession, 1003 = PTP_OC_CloseSession)con> !con:write_msg(('x'):rep(512*18-12))
ptp_usb_sendreq trans=5 len=0x14 (20) code=0x9999 nparam=2 param1=0x0000000b
USB_BULK_WRITE returned 20, size=20
ptp_usb_senddata trans=5 len=0x2400 (9216) code=0x9999
USB_BULK_WRITE returned 512, size=512
USB_BULK_WRITE returned 8704, size=8704
USB_BULK_WRITE returned 0, size=0
usb_bulk_read: No error
ERROR: call failed:I/O error
Elph130 is similar except it except it doesn't end with IO error.On the Ixus, USB communication after the test message is disrupted. I'm not 100% sure, but I think the first DoTransactionCanceled() appears after the (unsuccessful?) chdkptp disconnection attempt, the second one comes after the USB physical disconnection.I think the double DoTransactionCanceled() in both cases is due to the chdkptp disconnection code issuing usb_reset(). Commenting this this out seems to eliminate it, without obvious ill effect. (I've actually meant to make this optional for a long time...)
On the g10, I'd guess Op.[0x1003] initiates the disconnection and DoTransactionCanceled() appears when disconnect is done and again when the cam is physically disconnected.
Would it be possible to send that null packet?The log in my previous post seems to indicate that it is sent, and the whole sequence succeeds from the host POV, and the I/O error comes from reading the response. I guess ptp.c ptp_usb_senddata could be modified to send another one if the data is the magic length (to avoid messing up the transactions required to make the connection).
sudo dpkg-reconfigure wireshark-common
... check allow regular users to capture...
sudo usermod -a -G wireshark <user>
and re-logging in, but this didn't resolve the issue.dumpcap -i usbmon1 -w file.pcap
I could then open the file in wireshark. usb.addr contains "1.9"
The actual destination will show up as 1.9.x where x is the endpoint for the specific traffic. addr means it matches either source or destination.I guess ptp.c ptp_usb_senddata could be modified to send another one if the data is the magic length (to avoid messing up the transactions required to make the connection).I tried this, and the write for the second NULL packet timed out.
I confirmed that forcing the camera to do recv_data in chunks smaller than the problem size avoids the problem.I was about to check in a workaround based on this, but it turns out it only delays the problem to 0x45f4 >:(
env LANG=C gphoto2 --debug --debug-logfile=my-logfile.txt ...gphoto2 supports upload, so I thought I could get the problem size by padding the exif description of a 640x480 camera jpeg. Unfortunately, attempting to upload an unmodified jpeg from the camera fails with "Invalid Object Format Code"File upload with gphoto used to work on old cameras. Back in the days, I successfully used it on my first PowerShot (A200). I tested it on my s3is now, with success (card root was not accepted, but one of the image dirs worked). Same thing doesn't work on a more recent cam (g10).
I have no experience with the Windows built-in ptp/mtp support or Canon's utilities, does upload work with them?Not windows 10 built in at least, haven't tried Canon yet.
The 0x100c handler can be found by looking up references to add_ptp_handler, the 0x100c one is located in a table.Yeah, looking at those was part of reason I thought it wasn't just disabled, since it's fairly complicated. But maybe it's dead code.
Yeah, looking at those was part of reason I thought it wasn't just disabled, since it's fairly complicated. But maybe it's dead code.Found the following in it (using ghidra's decompiler).
if (local_c0 == 0x3002) {
uVar5 = (uint)local_c0;
if (param_7 != -1) {
uVar5 = local_9c;
}
if (param_7 == -1 || uVar5 == 0xffffffff) {
local_ec[0] = 0x201a;
goto LAB_ff8c3ce4;
}
}
else {
if (local_c0 != 0xbf01) {
local_ec[0] = 0x200b;
goto LAB_ff8c3ce4;
}
}
local_c0: seems to be the ptp object format code (this code is loaded in other local vars too, but those copies seem unreferenced)#define PTP_OFC_Script 0x3002
#define PTP_RC_InvalidObjectFormatCode 0x200B
#define PTP_RC_InvalidParentObject 0x201AGoogle found 0xbf01 in our svn, tools/patches/libptp2-1.1.0-canon-upload.diff+// ((int*)data)[1]=0xbf01; // canon firmwareSo, could it be that uploading firmware files is supported? The upload patch is from vitalyb and pre-dates DryOS cameras.
So, could it be that uploading firmware files is supported? The upload patch is from vitalyb and pre-dates DryOS cameras.PS2 files are listed in PTP (even on windows), with type 0xbf01. I tried uploading with gphoto2 and windows, but it presumably doesn't set the type.
!newfi2={ Filename="TEST.FI2", ObjectFormat=48897, ObjectCompressedSize=80448 }
!return con:ptp_send_object_info(newfi2)
=65537,0,262185
!con:ptp_send_object(('x'):rep(80448))
The 3 numbers are storage ID, parent handle and object handle, but we don't need them. SendObject just needs to send the size specified in the earlier SendObjectInfo.The same sequence of commands on D10 crashes with ASSERT!! OpObjHdl.c Line 2645 in PTPSessionTA0, regardless of size :-[I tested a few DryOS cameras (did not test file content).
!info={} for i,id in ipairs(con:ptp_get_storage_ids()) do info[i]=con:ptp_get_storage_info(id) end return info
!handles=con:ptp_get_object_handles()
!info={} for i,h in ipairs(handles) do info[i]=con:ptp_get_object_info(h) end return info
SendObject has some logic to do buffering with multiple reads, but it appears to have a pretty large buffer so it only used one in the problem case. It's possible this buffer is dedicated and could be re-used by our PTP code without taking away regular RAM, but I didn't track it down. d10 sub_FF8C9374 seems to be connected to the read (and presumably, buffer) size.The first routine that has a ComMemMan.c assert allocates 1MB uncached exmem (type 1, probably EXMEM_COM). I'd guess the buffer is there.
The first routine that has a ComMemMan.c assert allocates 1MB uncached exmem (type 1, probably EXMEM_COM). I'd guess the buffer is there.Thanks for that.
I wonder if that allocation can clash with those video buffers that need to be skipped when CHDK is loaded in exmem - most new-ish models don't support native remote shooting (except for the newest models that do).Good point. Seems likely since we weren't able to find a way to avoid those when allocating exmem. I guess if CHDK is already using EXMEM, the allocation for EXMEM_COM should fall outside it, but non-exmem enabled cams could see corruption.
Small patch for chdkptp to fix a crash on MacOS with libusb.Thanks. Added in r859
Also fixes a compiler warning in properties.c
Using uncached memory for the receiving buffer appears to fix the issue. This can be handled in recv_ptp_data by fiddling the cache bit, but we need a data cache flush function.This turns out to be more annoying than I had hoped. The cached address passed to recv_ptp_data isn't necessarily aligned to a cache line, so simply flushing or cleaning and flushing can corrupt neighboring data.
tl;drWe could also make a umalloc implementation that could use all memory available to CHDK, and use that on cameras thought to be affected (replacing mallocs in ptp code, if that's even possible). The change you made in recv_ptp_data() is quite large (took out that while block), but I can't say I'm familiar with that code. One of the ifdefs has a typo.
It's probably better to use dedicated buffers or umalloc. This is kind of unfortunate, since transfers of files or large scripts can use a significant fraction of free memory. umalloc may also have less memory available, since it only allocates from the Canon heap.
It's also annoying that this only appears to be need for a small subset of camera, and only at specific transfer size. I guess we could use a small uncached buffer only on those cameras when needed, but that seems pretty hokey. ptp.c is also current platform independent code.
The change you made in recv_ptp_data() is quite large (took out that while block), but I can't say I'm familiar with that code. One of the ifdefs has a typo.Yeah, that loop was the "buffering" logic I mentioned earlier. It was a leftover that didn't really make sense: in practice size was almost always <= buffer_size (half of the largest free block at the start of the PTP operation), and there appears to be no reason to operate in smaller chunks.
!m=require'extras/msgtest'
!m.test{size=1,sizeinc=1,count=10000,gc='step'}
!m.test{size=500,sizeinc=128,count=400,gc='step'}
The first tests single byte increments. The second covers a larger range including the problem (0x23f4 +n*512) values. It might fail on low memory cams trying to send a ~50k message.I took a look at the ComMemMan routines. The first(?) two inits and uninits the communication memory, the following two look like an allocator and a de-allocator. But, as previously mentioned, we don't necessarily want to use memory from there due to corruption risk.Agreed. I guess we could maybe use a similar approach to CHDK exmem and allocate an unused amount to cover the video buffers (if it allocates in a predictable way), or only use it in playback, but neither seems attractive for the amount of work.
We could also make a umalloc implementation that could use all memory available to CHDK, and use that on cameras thought to be affected (replacing mallocs in ptp code, if that's even possible).It's not that bad, but in the case of script messages, the memory is passed on and freed elsewhere. When I was testing, I made a version of memmgmt.c free() that detected uncached memory and called ufree on it. Any processing on the allocated buffer would be slow, but it's likely insignificant compared to the transfer. Script is compiled directly from the buffer. Messages are copied into Lua strings.
Now, I did not check whether the old and new #3 are doing exactly the same, but if they differ in operation that might be the reason behind the problem.Nice catch. I had sort of assumed it was a hardware difference, but could easily be a difference in the code that handles the transfer. I'll take a closer look at those cache functions later.
In case they do operate the same, this could still be used to identify the possible problem models, I think.
MCR p15, 0, R1,c7,c10, 1
MCR p15, 0, R1,c7,c6, 1
while g10 uses the combined instruction:MCR p15, 0, R1,c7,c14, 1
It seems like this difference should be cosmetic, but maybe there's a subtle difference with interrupts or something.MCR p15, 0, R4,c7,c10, 4
MCR p15, 0, R2,c7,c14, 2
G10 just does flush and clean. Both drain write buffer at the end. Draining for every iteration seems weird to me.The only time the trunk code does multiple reads is for file uploads that are > 1/2 max free block.Perhaps it's time to see how the 100d handler is doing it? I tried uploading files that are larger than the fw's usb buffer, and it worked. No idea whether it suffers from similar problems though.
Perhaps it's time to see how the 100d handler is doing it? I tried uploading files that are larger than the fw's usb buffer, and it workedYeah, that's definitely something to look at. I didn't try SendObject on A540 when I was playing with it earlier, because it doesn't list FIR files as objects, so it's not obvious what the ObjectInfo should contain. But I didn't dig into it much.
!m=require'extras/msgtest'
!m.test{size=1,sizeinc=1,count=10000,gc='step'}
occasionally crash (often after 3000-5000 iterations, but variable, sometimes it goes 10k without problems)send 7445 size:7445 0x1d15...Protocol error: data expected
After one of those, the camera crashed with an "Assert Memory.c line 67" which appears to be from malloc returning 0.I think it's possible to remove that handler at runtime and add our own.Another thing I've been thinking about is hooking the ptp_data functions, either by hooking TrnsCtrlTask which sets them up,
!newf={ Filename="TEST.DAT", ObjectFormat=0xBF01, ObjectCompressedSize=1024*1024*4 } return con:ptp_send_object_info(newf)
!return con:ptp_send_object(('x'):rep(newf.ObjectCompressedSize))
I think it's possible to remove that handler at runtime and add our own.This works. Attached patch hooks opcode 0x100d (SendObject) for A540.
On a540, the SendObject handler appears to have 256K buffer in uncached memory. Every call to recv_data gets the same address (0x11f80000).I experimented a bit on the a3200, and found that there are code paths using cached and uncached memory, so both are "legal" there. The uncached buffer is also 256k. The return value of recv_data is checked (bit0).
It does not appear to have the n*512 - 11 problem our code has.How can I test this?
To test if it happens with CHDK code, you can use the attached patch, which buffers reads in TBUF_SIZE chunks. You can switch between malloc/free and umalloc/ufree.QuoteIt does not appear to have the n*512 - 11 problem our code has.How can I test this?
m.test{size=1,sizeinc=1,count=10000,gc='step'}
will fail at 1013 with a stalled connection and IO error on the chdkptp side. If you change the size to 1024, it will then fail at 1525, and so on for multiples of 512.!newf={ Filename="TEST.DAT", ObjectFormat=0xBF01, ObjectCompressedSize=256*3*1024-11 } con:ptp_send_object_info(newf) con:ptp_send_object(('x'):rep(newf.ObjectCompressedSize))
which did not fail.con 1> =return call_func_ptr(0xFF8C9374,0)
2:return:8192
con 2> =return call_func_ptr(0xFF8C9374,1)
3:return:2048
con 3> =return call_func_ptr(0xFF8C9374,2)
4:return:16384
con 4> =return call_func_ptr(0xFF8C9374,3)
5:return:235520
con 5> =return call_func_ptr(0xFF8C9374,4)
6:return:524288
con 6> =return call_func_ptr(0xFF8C9374,5)
7:return:0
con 7> =return call_func_ptr(0xFF8C9374,6)
NHSTUB(get_ptp_buf_size, 0xFFE38264)
NHSTUB(get_ptp_file_buf, 0xFFE57498)
On d10 (without CHDK exmem enabled) the buffer address is 0x43E42E00If there's no typo here, the difference of the two addresses is only 0xc0000.
Comments in makefile.inc indicate the video buffer starts at 0x43F02E00, exactly 1MB after the start of the PTP buffer.
Good catch. It was mistake. The exmem buffer is at 0x43e02e00. TheOn d10 (without CHDK exmem enabled) the buffer address is 0x43E42E00If there's no typo here, the difference of the two addresses is only 0xc0000.
Comments in makefile.inc indicate the video buffer starts at 0x43F02E00, exactly 1MB after the start of the PTP buffer.
A540 (and likely other vxworks cameras) appear to have another, unrelated problem, present in the current trunk. Rapid fire messages likeI worked on this a bit more. It appears that protecting Canon malloc/free/GetMemInfo with a semaphore avoids the issue.Code: [Select]!m=require'extras/msgtest'occasionally crash (often after 3000-5000 iterations, but variable, sometimes it goes 10k without problems)
!m.test{size=1,sizeinc=1,count=10000,gc='step'}
It turns out that on vxworks cams, the compiler canon used pretty much always put 0x1000 or 0x9000 in one register, and then used ORR or ADD to make the operation ID for a bunch of subsequent add_ptp_handler calls.Attached modified patch adds (incomplete) support for this - vx only. Might be duplicated effort.
Attached modified patch adds (incomplete) support for this - vx only. Might be duplicated effort.Nice :D
I found an ID (0x9028) that seems to have two different handlers in some firmwares.The thumb2 sig finder ends up with some event procs and tasks like this too. I was thinking about making it rename them like foo_FW_2 or something.
The function to get the buffer address (d10 FF9F6B28, a540 FFE57498) seems a bit more complicated to match, so any clever ideas for that are welcome.No code this time, but
!newfi2={ Filename="BAD.DAT", ObjectFormat=0xbf01, ObjectCompressedSize=1024*256+512-11}
!return con:ptp_send_object_info(newfi2)
!return con:ptp_send_object(('x'):rep(newfi2.ObjectCompressedSize))
Triggers the assert, and size 1024*256+512 triggers the stalled connection, while 1024*257 works fine. I previously thought native operations weren't affected, but my test was incorrect. This is reproducible without CHDK running.!fh=io.open('TEST.DAT','wb') fh:write(('x'):rep(1024*255+1525-14)) fh:close()
u test.dat
The -14 accounts for the file name and file name length.!con:write_msg(('x'):rep(1024*255+1525))
Same experience (crash, hang) on s80 (older vx).Code: [Select]!newfi2={ Filename="BAD.DAT", ObjectFormat=0xbf01, ObjectCompressedSize=1024*256+512-11}Triggers the assert, and size 1024*256+512 triggers the stalled connection
!return con:ptp_send_object_info(newfi2)
!return con:ptp_send_object(('x'):rep(newfi2.ObjectCompressedSize))
chdkptp -c -e'exec require"camtests".run("xferbug_0x23f4")'
chdkptp -c -e'exec require"camtests".run("xferbug_0x1f5")'
original d10 bug (recv_data to uncached at 0x23f4 +N*512)Both of these tests execute successfully on a460.Code: [Select]chdkptp -c -e'exec require"camtests".run("xferbug_0x23f4")'
Unresolved A540 bug (multiple recv_data calls, total size = the first 512*N-11 that results in multiple recv_data)Code: [Select]chdkptp -c -e'exec require"camtests".run("xferbug_0x1f5")'
If anyone can get another program to PTP upload a 262645 byte file, that would be interesting.I don't know whether gphoto qualifies. Tried on ixus65_sd630 (old vx with working gphoto upload), cam crashed with that size. Assert is the same as on your a540.
gphoto2 -u /tmp/camou.fir --folder=/store_00010001/DCIM/100CANON
Both of these tests execute successfully on a460.Weird. Early DryOS like sx100 and a470 might be interesting.
I don't know whether gphoto qualifies. Tried on ixus65_sd630 (old vx with working gphoto upload), cam crashed with that size. Assert is the same as on your a540.I think the gphoto code is related to ptpcam, but it's better than nothing.Code: [Select]gphoto2 -u /tmp/camou.fir --folder=/store_00010001/DCIM/100CANON
Early DryOS like sx100 and a470 might be interesting.sx100: xferbug_0x23f4 executes normally, xferbug_0x1f5 causes ASSERT!! BulkTrns.c Line 1568 .
Next step I guess is to either hook and log PTP functions and/or capture USB traffic.I went with hooking. Canon UploadFirmware.exe doesn't trigger the crash because it uses OC 0x9019, identified as "PTP_OC_CANON_SendPartialObject" in the gphoto source.
00011830 DBG:ptp h:0x001fb6ac d:0x00008d94 oc:0x100c s:0x1 t:0x21 0x0 0x0 0x0 0x0 0x0
00011870 DBG:recv h:0x001bb09c b:0x0021e490 s:0x8d (141) u1:0x0 u2:0x0
00011960 DBG:ptp h:0x001fb6ac d:0x00008d94 oc:0x9019 s:0x1 t:0x22 0x0 0x40000 0x1 0x0 0x0
00012000 DBG:recv h:0x001bb09c b:0x11f80000 s:0x40000 (262144) u1:0x0 u2:0x0
00012220 DBG:ptp h:0x001fb6ac d:0x00008d94 oc:0x9019 s:0x1 t:0x23 0x40000 0x1f5 0x3 0x0 0x0
00012270 DBG:recv h:0x001bb09c b:0x11f80000 s:0x1f5 (501) u1:0x0 u2:0x0
May or may not be related: the low end a4xx series has crippled USB ("full speed"), up to and including a470.Wow, I thought that was only the really old ones. Interesting.
Given that Canon uses this approach, the it seems likely our only fix would be to extend the CHDK protocol to allow similarly splitting uploads over multiple transactions.Thinking about this more, I realized it should be possible to work around without resorting multiple transactions or protocol changes. My previous observation was
The crash happens whenSo if the size is in the magic range (0x...1f4 < size < 0x...3f4), reducing the size of the next-to-last read to ensure the final one is > 0x3f3 should avoid the problem.
1) data->recv_data is called more than once
2) total size is (512*n) - 11 to (512*n) - 1 i.e. 0x...1f5 through 1ff.
Additionally, when using the native buffer (maybe any uncached) a stalled connection happens for something like 512*n to 512*(n+1)-13. Larger sizes appear to be OK
chdkptp -e"exec require'camtests'.runbatch{bench=true,xfersizebugs=true,filexfer=true}"
Note that the xferbug_0x1f5 test may take something like a minute on some cameras.I'd appreciate anyone who can test with the current svn chdkptp running the following:a410 result attached (I used the GUI). I get the same log (with execution times differing somewhat) with or without CAM_PTP_USE_NATIVE_BUFFER. The ixus65_sd630 passes the test.Code: [Select]chdkptp -e"exec require'camtests'.runbatch{bench=true,xfersizebugs=true,filexfer=true}"
a410 result attached (I used the GUI). I get the same log (with execution times differing somewhat) with or without CAM_PTP_USE_NATIVE_BUFFER. The ixus65_sd630 passes the test.Thanks. The initial failure is not in one of my new tests, but on sending a 52 byte message in the basic message test. Can you try
!m=require'extras/msgtest'
!m.test{size=52,count=1}
If that works, try!m=require'extras/msgtest' m.test{size=1,sizeinc=1,count=100}
It would also be good to know if builds before 5235 have the same problem.I used the GUIDo you mean you ran it using an executable with GUI support in CLI, or actually ran it in the GUI?
Can you tryCode: [Select]!m=require'extras/msgtest'
!m.test{size=52,count=1}
___> !m=require'extras/msgtest'
___> c
connected: Canon PowerShot A410, max packet size 64
con> !m.test{size=52,count=1}
testing 1 messages size 52
send 1 size:52 0x34...I/O error
aborted at send 1 size:52 0x34, communication error
quit failed I/O error
ran 1 fail 1 time 10.0164 send time 0.0109It would also be good to know if builds before 5235 have the same problem.I get the same behaviour with r5204.
Do you mean you ran it using an executable with GUI support in CLI, or actually ran it in the GUI?I copied this
exec require'camtests'.runbatch{bench=true,xfersizebugs=true,filexfer=true}to the gui window and pressed execute. chdkptp.sh starts the GUI by default.
I don't actually expect you to fix all ptp problems on this model, I only picked it for this test because it has a history of ptp issues.Understood. It would be good to fix or understand if we can, but this seems like a different problem from the other cameras. You could run without the 'bench' option to see if the others pass, like
exec require'camtests'.runbatch{xfersizebugs=true,filexfer=true}
I copied thisFWIW, if you pass any command line arguments without -g, it should start in CLI mode as long as the .sh script passes the arguments through like the one in SVN does. I really doubt this has anything to do with the error, but it should be easy to try.Code: [Select]exec require'camtests'.runbatch{bench=true,xfersizebugs=true,filexfer=true}to the gui window and pressed execute. chdkptp.sh starts the GUI by default.
You could run without the 'bench' option to see if the others pass, likeThat executes with no failures.Code: [Select]exec require'camtests'.runbatch{xfersizebugs=true,filexfer=true}
I really doubt this has anything to do with the errorMe too. My previous post shows the results of the msgtest, run via cli.
That executes with no failures.Thanks. One more
!m=require'extras/msgtest'
!m.test{size=513,sizeinc=-1,count=512}
here (https://www.magiclantern.fm/forum/index.php?topic=24385.0).Hmm. I've always operated on the assumption that it's game over if the PTP host is compromised. CHDK is obviously vulnerable: PTP_CHDK_CallFunction PTP_CHDK_SetMemory, Lua poke and likely a lot of vulnerable Lua functions are all readily available. These could all be executed over PTP/IP too.
One more
___> !m=require'extras/msgtest'
___> c
connected: Canon PowerShot A410, max packet size 64
con> !m.test{size=513,sizeinc=-1,count=512}
testing 512 messages size 513-1, inc -1 gc=step
send 1 size:513 0x201...ok
send 2 size:512 0x200...ok
(...)
send 461 size:53 0x35...ok
send 462 size:52 0x34...I/O error
aborted at send 462 size:52 0x34, communication error
quit failed I/O error
ran 462 fail 1 time 26.5785 send time 6.2933
We could offer an option to disable CHDK PTP functionality, checking a conf setting and returning an error in handle_ptp would be easy.We could, although I'm not sure if it's worth it. The cameras only activate their wireless interface when the user enters the Wifi related dialogs. I find it hard to imagine that anyone (other than some 3-letter agencies and the like) will spend their time exploiting these firmware bugs. And only a very low percentage of cameras run CHDK.
send 461 size:53 0x35...okThanks.
send 462 size:52 0x34...I/O error
aborted at send 462 size:52 0x34, communication error
We could, although I'm not sure if it's worth it. The cameras only activate their wireless interface when the user enters the Wifi related dialogs. I find it hard to imagine that anyone (other than some 3-letter agencies and the like) will spend their time exploiting these firmware bugs. And only a very low percentage of cameras run CHDK.Yeah, this doesn't seem a high risk. OTOH, someone making a stink about "CHDK undoes Canon security fix" could be unfortunate.
I posted the links because the descriptions mention the problematic PTP handlers by name. And that we'll get to see the fixes (M3 and M10 are on the list).That could be interesting.
It may become more of an issue in the future, @srsa_4c suggested DryOS 59 extended the stat structure to support 64 bit sizes. >4 GB files would require a lot of changes, though probably not too common for CHDK until exFAT boot or multi-partition is supported.I don't think there is a PowerShot firmware based camera that records movies exceeding the 4GB barrier - but I can't be sure as this limitation is often omitted from various literature.
I don't think there is a PowerShot firmware based camera that records movies exceeding the 4GB barrier - but I can't be sure as this limitation is often omitted from various literature.Agreed, my impression is that they don't. My thought was extending the stat structure might be a hint that it was coming in future models, though of course it could be for other products that use DryOS, and of course it's possible future models will not be supportable by CHDK anyway.
My current thinking on >2gb files is just to provide a clean error, probably GeneralError in the protocol, something like "file too large" in chdkptp in cases where it would know.It's not like I see 2...4GB files very often, but they can be created on camera. I checked SendObject, it uses Open() rather than one of the fopen() variants. Download of >2GB files does work via libgphoto based software.
How about using open/read instead of fopen/fread when dealing with big files? I agree that files bigger than 4GB don't need to be supported (64bit versions of open etc. may not even exist in firmware).My impression was that the problems were in the PTP layer, not file IO, but I didn't dig deeply. If sendobject works, it might be worth investigating further.
My impression was that the problems were in the PTP layer, not file IO, but I didn't dig deeply. If sendobject works, it might be worth investigating further.It's possible that you're right and the problem is indeed caused by transferring >=2GB of data in one go. I guess this could be addressed by transferring them in multiple pieces, but that would not work with clients not implementing that addition to the protocol (in case that's a problem).
Interesting Canon issued a fix for https://chdk.setepontos.com/index.php?topic=13872.0 and left this ;)No CVE - no headache, I guess. They did make a step toward securing this backdoor, so current models are not in "danger". Or at least publicly not.
the decompiled code tells that PTP_OC_CANON_InitiateEventProc0 0x9050 and PTP_OC_CANON_InitiateEventProc1 0x905c do register the function, as you already mentioned but I don't know what the difference of them is. Both functions do call the same subroutine with the same arguments. The only difference is, that some some buffer on the stack is initialized with 1 or 0. I wasn't brave enough to call these functions on camera. Have you already tried this? Do you know what the difference between the functions is?No, I have not tried going after the changes they made to the registration function(s). I've seen that some magic sentences are involved (I'd prefer them not be published as is) and 0x9050/0x905c now has a data phase. Before DIGIC 8, the 0x9050/0x905c handlers were one and the same (as far as i've seen).
However, I've tried to register PTP_OC_CANON_ExecuteEventProc 0x9052 manually, which worked but the camera crashed when trying to call a function, so I think the initate functions do some necessary initialization to make it work properly.If that is so then they really did put in some effort. Perhaps those CVEs were the motivation.
I also don't know what the difference between PTP_OC_CANON_TerminateEventProc_051 0x9051 and PTP_OC_CANON_TerminateEventProc_05D 0x905d is. Those are probably the counterparts for the initiate pairs.Yes, they again used to be equal. They are still equal in sx740.
Hello Hello %s
So the first argument gets passed twice. My chdkptp implementation based on srsa_4c's behaves the same (in fact, I hadn't tried demo.py until I ran into this)srsa_4c thanks for posting your findings, I've implemented it in my mlinstall tool: https://github.com/petabyt/mlinstall/FWIW, I used the following for P&S. I think these not useful on EOS, just posting since I I also needed test cases...
Do you know of any quick evprocs with parameters to test it on?
3) The long arg handling appears to fail with multiple long args (tested on elph130 and elph180).Fails here also, thanks for noticing. I have a custom eventproc for testing, but I did not try multiple long args. I was happy that memcpy succeeded and considered the case done. I'll try finding the missing piece.
Do you know of any quick evprocs with parameters to test it on?I used memcpy due to its usefulness.
Another unrelated FWIW, the 9050 905c etc opcodes are also present on some Digic DV cams, such as xf605Nice... I wonder if they used the same API and whether the eventprocs (if any) include known ones.
The only difference is, that some some buffer on the stack is initialized with 1 or 0.On sx740, that seems to be an argument passed to an underlying function, its value appears to influence the way the challenge is evaluated.
IsEventProcRunning 0x9057I tested this a bit on elph130. it returns:
Self-explanatory name, but I did not try calling it. Can return up to 3 params.
Nice... I wonder if they used the same API and whether the eventprocs (if any) include known ones.notes (https://chdk.setepontos.com/index.php?topic=1641.msg147757#msg147757)
3) The long arg handling appears to fail with multiple long argsThanks to a different firmware and reyalp's help, I could fix this. I updated (https://chdk.setepontos.com/index.php?topic=4338.msg147738#msg147738) the original "announcement". A patch for my script in petabyte's ptp-tool (https://github.com/petabyt/sequoia-ptpy) is attached to show the fix.
Thanks to a different firmware and reyalp's help, I could fix this. I updated (https://chdk.setepontos.com/index.php?topic=4338.msg147738#msg147738) the original "announcement". A patch for my script in petabyte's ptp-tool (https://github.com/petabyt/sequoia-ptpy) is attached to show the fix.Nice work, with the new information long args are working in my chdkptp code. I've added "proper" support in chdkptp r1118-1120. Documentation can be found in chdkptp/lua/ptpevp.lua before con_methods:ptpevp_call. This should be usable with any cam that supports the API, regardless of whether CHDK is installed.
!require'extras/devutil'.init_cli()
After that, dptpdevinfo without options gives reasonable output. Adding -ptpevp calls InitiateEventProc0 before querying deviceinfo, which will add the related opcodes to the supported list. However, it won't work on cameras that expect a different handshake, or non-canon devices.Oddly, the CHDK PTP opcode isn't reported as supported on the initial connection, although it's available subsequently and works.The cause of this appears to be