CHDK PTP interface

reyalp

14114

Re: CHDK PTP interface

« Reply #1050 on: 10 / August / 2013, 23:32:26 »

Advertisements

Current debug patch. This does not completely fix the problem. It will only work on a540 as is, posting as a reference for camera log debugging.

On many vxworks cameras, CAM_CONSOLE_LOG_ENABLED sends the camera console to stdout.txt using fwrite. This is very slow and can cause crashes, so you don't want to use it for individual debug messages. The "Camera Log" is buffered and fast, and can be written to stdout with the ShowCameraLog eventproc.

So the debug procedure goes like this:

Code: [Select]

# truncate the existing stdout
=f=io.open('A/stdout.txt','wb');f:write('init\n');f:close()
# make the camera log buffer larger, so we get a reasonable number of lines
# should return 3 zeros if everything worked. ShowCameraLogInfo records the new camera log config in stdout
=return call_event_proc('StopCameraLog'),call_event_proc('StartCameraLog',0x20,0x4000),call_event_proc('ShowCameraLogInfo')
# shoot
rec
rs -cont=20
# download the ouput
d stdout.txt

An example of the problem. Lines beginning with > are my commentary.

Code: [Select]

> ... my log wasn't big enough to see the start of this shot
00150350: DSI_ControlShootInfo( 0x44, 0 )
00150360: LogicalEvent:0x311e:adr:0x0,Para:0
> filewrite hook signals image 107 is available
00150370: jpg av 107 A/DCIM/106CANON/IMG_0107.JPG ign 1
> raw hook signals raw 108 is available
00150590: raw av 108
> raw hook sees jpeg is still pending, so it stalls
00150590: raw av pend fmt 1 ign 1
> jpeg junk requested
00150630: rc chunk 1 ign 1
> completed with status "more chunks"
00150640: rc send comp 1 status 1 ign 1
00150650: rc chunk 1 ign 1
> second chunk completed, status last chunk, this clears the pending flag
00150660: rc send comp 1 status 2 ign 1
> raw hook sees pending flag clear after 6 sleep iterations,
00150670: raw av pend done wait 6 fmt 0 ign 1
> raw hook exits, this has called filewrite_set_discard_jpeg(1) for shot 108
00150670: raw av done no raw
> filewrite hook finally notices it is done with 107. After this, it will clear ignore_current_write!
00150720: jpg av done 108 A/DCIM/106CANON/IMG_0107.JPG ign 1
00150750: DSI_ControlShootInfo( 0x27, 0 )
> ... next shot (#109)
00150830: SS:ShootPicture
00150840: DSI_ControlShootInfo( 0x43, 0 )
00150850: ShootSeqToUI:0x2008:adr:0x272040,Para:2564160
00150850: _EntryPrepareRecreviewOff
00150850: ShtCon_StopReview
00150850: StopRecReviewController
00150860: SS:StopRecreview
00150860: LogicalEvent:0x3120:adr:0x0,Para:0
00150860: ShtCon_StartReview
00150860: SS:StartRecreview
00150870: ShootSeqToUI:0x2001:adr:0x211470,Para:2167920
00150880: ShootSeqToUI:0x2021:adr:0x0,Para:0
00150880: LogicalEvent:0x311f:adr:0x0,Para:0
00150880: Sht_CancelStrobeChargeTimer
00150880: DSI_ControlShootInfo( 0x2b, 0 )
00150880: DSI_ControlShootInfo( 0x44, 0 )
00150890: LogicalEvent:0x311e:adr:0x0,Para:0
> filewrite signals image 108 available, but notice the ignore write flag is zero. IMG_0108 was saved on the card
00150900: jpg av 108 A/DCIM/106CANON/IMG_0108.JPG ign 0
00151040: rc chunk 1 ign 0
00151050: rc send comp 1 status 1 ign 0
00151060: rc chunk 1 ign 0
00151080: rc send comp 1 status 2 ign 0
00151080: jpg av done 108 A/DCIM/106CANON/IMG_0108.JPG ign 0
> filewrite hook ended quickly this time, before raw hook
00151120: raw av 109
00151120: raw av done no raw

The solution to this seems to be to clear the jpeg pending flag from filewrite hook, after ignore_current_write is cleared.

remotecap-cont-debug-3.patch (8.43 kB - downloaded 21 times.)

Logged

Don't forget what the H stands for.

reyalp

14114

Re: CHDK PTP interface

« Reply #1051 on: 11 / August / 2013, 18:36:17 »

Test code with proposed fix attached.

Using this code, I have done successful runs of 1000 continuous mode remotshoots on both a540 and d10. I didn't observe any memory leaks. On d10, free memory was only 64 bytes less than at the start of the sequence.

The fix is generic for dryos cameras, but for cameras like A540, a minor change in the filewrite hook will be needed. Only the a540 is updated in the patch.

This is untested on CAM_FILEWRITETASK_SEEKS cameras.

The patch includes debug logging code, but it will only be used if you #define DEBUG_LOGGING in platform.h In this case, you will need to ensure that _LogPrintf is available. This shouldn't be needed unless you encounter problems.

The basic approach is to make the raw hook wait for filewrite to clear ignore_current_write before allowing the hook to continue for the next image.

To do this, I added pending_image_data which tracks the data types that have been requested, and only clears them when the relevant hook is completely done with them. For jpeg, this is clearly only after the jpeg hook is done with ignore_current_write for the current image.

Tracking the "pending" status of raw and DNG header isn't strictly needed right now, but I think it may be useful in the future.

Note that we could probably get slightly better performance if there was a way not to block waiting for filewrite, but it would mean raw N+1 potentially being transferred before jpeg N was complete. Even if we had a way to do this, the current protocol couldn't support it.

As an aside, I noticed that chdkptp continuous shoot doesn't handle the file counter wrapping... the chdkptp side will end, but the camera will keep shooting.

edit:
updated patch to include a410, which is the only other vxworks cam with filewrite.

Also, we may be able to get rid of current_write_ignored for dryos cameras with this logic.

remotecap-cont-debug-6b.patch (9.81 kB - downloaded 20 times.)

« Last Edit: 11 / August / 2013, 19:08:21 by reyalp »

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: CHDK PTP interface

« Reply #1052 on: 11 / August / 2013, 19:27:53 »

Quote from: reyalp on 11 / August / 2013, 18:36:17

Test code with proposed fix attached.

Thanks, will test when I get some time.

Logged

reyalp

14114

Re: CHDK PTP interface

« Reply #1053 on: 11 / August / 2013, 19:38:26 »

Slightly updated patch, to fix compile error on FILEWRITETASK_SEEKS cams and clear fwt_bytes_written for all dryos.

remotecap-cont-debug-7.patch (10.47 kB - downloaded 22 times.)

Logged

Don't forget what the H stands for.

reyalp

14114

Re: CHDK PTP interface

« Reply #1054 on: 11 / August / 2013, 20:47:13 »

Based on my testing and feedback from nafraf in irc, I've checked a version of this without the debug code (+ fixed vxworks compile error in the previous patch) into the trunk changeset 3026. I'd like to get a bit more testing before moving it over to the release branch.

I've attached a debug patch based on this in case anyone needs it.

Also, chdkptp r397 should handle the timeout errors better. This probably doesn't fix the underlying issue where -cont fails to start shooting, but after the timeout everything should return to a usable state.

remotecap-cont-debug-8.patch (4.52 kB - downloaded 23 times.)

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: CHDK PTP interface

« Reply #1055 on: 13 / August / 2013, 15:47:02 »

@reyalp

I have done a few test runs (without the additional debug code) on the a410 in continuous mode. It easily made 9800 photos. But there's some weirdness.
You have mentioned that counter wraparound is not handled. One of my test runs started with IMG_0106. I used rs -cont=9800, to avoid counter wrap. I got the 9800 images, but the last image numbers were 9900, 9901, 0002, 0003, 0004, 0005. The camera kept on shooting after this and saved the jpegs to the card (you mentioned this will happen in case of a wraparound), until I stopped the script manually.
At this point, USB connection was still available (!).

Could it be that the wrapping happens earlier than 9999?

Logged

reyalp

14114

Re: CHDK PTP interface

« Reply #1056 on: 13 / August / 2013, 16:18:55 »

Quote from: srsa_4c on 13 / August / 2013, 15:47:02

Could it be that the wrapping happens earlier than 9999?

From what I understand the canon firmware can create a new folder before 9999 (depending on number of images present, I think?)

I'm not sure if this resets numbering. Honestly I'm not totally clear how all this is supposed to work, and it's a bit annoying to find out by testing.

Anyway, I think 9800 continuous shots is a pretty good result

It's interesting (and nice) that this seems to avoid running into limit where having ~1000 images on the card kills Canon PTP...

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: CHDK PTP interface

« Reply #1057 on: 13 / August / 2013, 16:51:24 »

Quote from: reyalp on 13 / August / 2013, 16:18:55

From what I understand the canon firmware can create a new folder before 9999 (depending on number of images present, I think?)

I don't know. This amount of pictures could be unexpected by the fw, thus the weird numbering?
We could return the "shot count" param as-is, for debug use.

Quote

It's interesting (and nice) that this seems to avoid running into limit where having ~1000 images on the card kills Canon PTP...

I wonder how. Perhaps I should test non-continuous mode too.

Logged

tpont

81

Re: CHDK PTP interface

« Reply #1058 on: 14 / August / 2013, 17:37:16 »

nafraf sent me compiled test versions for A490 and A2300 that included reyalp's remotecap-cont-debug-8.patch However the problem with rs not working after a variable number of photos is still there in tests with both cameras.

Logged

reyalp

14114

Re: CHDK PTP interface

« Reply #1059 on: 14 / August / 2013, 22:19:56 »

Quote from: tpont on 14 / August / 2013, 17:37:16

nafraf sent me compiled test versions for A490 and A2300 that included reyalp's remotecap-cont-debug-8.patch However the problem with rs not working after a variable number of photos is still there in tests with both cameras.

The debug patch doesn't have any fixes that aren't in the trunk, it just has some debug logging calls, and to use those, you need DEBUG_LOGGING defined in platform.h have the _LogPrintf function found.

If you are referring to the error described in http://chdk.setepontos.com/index.php?topic=4338.msg103854#msg103854 this seems like it is just the shooting part of the script failing to take a shot and getting stuck (discussed earlier with the flash related issues). This shouldn't happen with the current chdkptp code in svn, though the shot would still fail. I'll try to get a new chkdptp snapshot build up soon.

Logged

Don't forget what the H stands for.