proposal - script shooting hooks - page 9 - General Discussion and Assistance

reyalp

14080

Re: proposal - script shooting hooks

« Reply #80 on: 12 / March / 2015, 02:02:16 »

Advertisements

Lapser suggested that flickering happens at high shutters speeds. The attached plot from my last run appears to support that.

I am also starting to suspect that the black frames I mentioned in that thread are not a hardware failure but are caused by some kind of timing/scheduling issue related to the script hooks. It does not seem to occur if I have the debug display enabled (with takes > 10ms, and so makes the kbd_task yield), where it occurs ~1% of the time if I don't. In this test, I added a 10ms sleep, and the problem didn't occur in ~4300 frames.

flickery video
https://youtu.be/HaxUQrLvXwE

test-rawmeter-2015-03-11-exp.png (68.76 kB, 1183x751 - viewed 56 times.)

Logged

Don't forget what the H stands for.

c_joerg

1248

Re: proposal - script shooting hooks

« Reply #81 on: 13 / March / 2015, 15:23:48 »

I’m still thing of the Display behavior.

I have a small test script, which holds the camera in half press, like in your script. When I run the script, I have live view between the shoots (S110 and G1X).

When I run your script
ui_display_mode_t=0, the script should not try to change the display state at all true?
On S110 I see the Review picture just flashing
On G1X I see the Review picture for the full interval time (in the Test 8s).

So where is the difference?

Test.lua (0.72 kB - downloaded 17 times.)

Logged

M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

reyalp

14080

Re: proposal - script shooting hooks

« Reply #82 on: 13 / March / 2015, 16:54:15 »

Quote from: c_joerg on 13 / March / 2015, 15:23:48

When I run your script
ui_display_mode_t=0, the script should not try to change the display state at all true?

Correct, though there could be bugs

Quote

On S110 I see the Review picture just flashing
On G1X I see the Review picture for the full interval time (in the Test 8s).

So where is the difference?

I'd guess this is probably because my script uses the "shutter hook" to keep the interval exact. It presses shoot_full for the next shot, and then makes the code wait somewhere very close to where the actual exposure would start. This gives a very precise interval, but where the hook waits is likely to be implemented differently on different cameras.

I said earlier that my cams show the review, but now that I look at D10 it seems like it behaves more like you describe the S110, the image just flashes and stays black for most of the "wait" time. I didn't really notice before because I usually use an interval close to the maximum shooting speed.

Logged

Don't forget what the H stands for.

lapser

1093

Re: proposal - script shooting hooks

« Reply #83 on: 14 / March / 2015, 22:20:36 »

Quote from: reyalp on 31 / December / 2014, 19:17:53

I think toward the end we see another issue which IIRC lapser noted before, that ISO changes only take effect in some increment greater than a single APEX96 unit (3?)

I've been away from coding for awhile, but I remember tracking the ISO problem down to the set_sv96(sv) function in CHDK. The glitches happen when it changes the "base" ISO.

I fixed it by using my own code for set_sv96. This is how I tested it in Lua:
===
props=require("propcase")
pSV=props.SV -- real sv
pDELTA_SV=props.DELTA_SV
. . .
(calculate desired sv)
set_prop(pDELTA_SV,sv-get_prop(pSV)+get_prop(pDELTA_SV))
set_prop(pSV,sv)
===

Or, you can use this code in C to replace set_sv96()
===
shooting_set_prop(PROPCASE_DELTA_SV,
sv96-shooting_get_prop(PROPCASE_SV)+shooting_get_prop(PROPCASE_DELTA_SV));
shooting_set_prop(PROPCASE_SV,sv96);
===

Maybe you could re-run your ISO test using the propcase method above instead of set_sv96() and see if that fixes it?

My theory on what's happening is that the "base" ISO is calibrated to match film cameras, so base ISO 200 isn't exactly twice as fast as ISO 100. But if you leave the base ISO unchanged and just use the "delta_sv" to set the ISO speed, then everything works as expected.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: proposal - script shooting hooks

« Reply #84 on: 14 / March / 2015, 22:27:03 »

Quote from: reyalp on 13 / March / 2015, 16:54:15

I said earlier that my cams show the review, but now that I look at D10 it seems like it behaves more like you describe the S110, the image just flashes and stays black for most of the "wait" time. I didn't really notice before because I usually use an interval close to the maximum shooting speed.

If you use continuous mode and just hold the shutter down, it displays the pictures on the screen in real time. That's one big advantage of continuous mode. You can see what you're doing. Review should be set to off.

The other advantage is that you get a faster maximum shot rate, about 2 shots per second, in continuous mode.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14080

Re: proposal - script shooting hooks

« Reply #85 on: 14 / March / 2015, 23:24:28 »

Quote from: lapser on 14 / March / 2015, 22:20:36

Quote from: reyalp on 31 / December / 2014, 19:17:53
I think toward the end we see another issue which IIRC lapser noted before, that ISO changes only take effect in some increment greater than a single APEX96 unit (3?)
I've been away from coding for awhile, but I remember tracking the ISO problem down to the set_sv96(sv) function in CHDK. The glitches happen when it changes the "base" ISO.

I verified later that this was not actually what I was seeing:
http://chdk.setepontos.com/index.php?topic=12165.0

Based on the those results, I believe the current CHDK ISO override code works very well. That code was reworked significantly a while back, so it's possible the version you worked on behaved differently.

Quote from: lapser on 14 / March / 2015, 22:27:03

The other advantage is that you get a faster maximum shot rate, about 2 shots per second, in continuous mode.

I thought that too, but in my testing on the cameras that I have, the "hold half press / click full press" approach gives very similar performance to continuous mode (http://chdk.setepontos.com/index.php?topic=11527.0).

That said, I do intend to add continuous support to my script at some point. It should be straightforward, I already have it in the contae script.

The continuous mode review is nicer.

Logged

Don't forget what the H stands for.

reyalp

14080

Re: proposal - script shooting hooks

« Reply #86 on: 14 / March / 2015, 23:43:12 »

I did some more testing on the D10 shutter issues over the last few days.

I have never seen the issue with the debug drawing on, > 12k shots since I first saw the issue.

Overlay off, sleep(10) before the raw hook continue: no failures in ~4300 shots

Overlay off, set_yield(-1,50). I figured if sleep made it go away, yielding less would make it happen more often. Nope. No failures in >6800 shots.

OK, maybe it's random glitchy hardware and it's not doing it any more? Go back to the original version that failed:
Overlay off, no set_yield or sleep - 21 failures in ~5400 shots.

The last run didn't use super short shutter speeds (all longer than 1/1000) and many of the failed shots were at the longer end.

Logged

Don't forget what the H stands for.

lapser

1093

Re: proposal - script shooting hooks

« Reply #87 on: 15 / March / 2015, 01:12:49 »

Quote from: reyalp on 14 / March / 2015, 23:24:28

Quote from: lapser on 14 / March / 2015, 22:27:03
The other advantage is that you get a faster maximum shot rate, about 2 shots per second, in continuous mode.
I thought that too, but in my testing on the cameras that I have, the "hold half press / click full press" approach gives very similar performance to continuous mode (http://chdk.setepontos.com/index.php?topic=11527.0).
inuous mode review is nicer.

If you're saving DNG files or writing log files in continuous mode, that can slow things up. Try holding the shutter down without CHDK active and compare that to the rate your script gets. You could also try running the script without writing a log and see if that's faster.

I get a maximum of about 1 shot per second in single shot mode, and 2 shots per second in continuous. This is an example done at 2 shots per second:

https://www.youtube.com/watch?v=b_nVZbV-MR4

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14080

Re: proposal - script shooting hooks

« Reply #88 on: 15 / March / 2015, 02:16:21 »

Quote from: lapser on 15 / March / 2015, 01:12:49

I get a maximum of about 1 shot per second in single shot mode, and 2 shots per second in continuous. This is an example done at 2 shots per second:

Canon continuous mode spec for D10 is 1.1 FPS. The script does around 0.9 FPS. Other cameras may well behave differently.

Logged

Don't forget what the H stands for.

reyalp

14080

Re: proposal - script shooting hooks

« Reply #89 on: 16 / March / 2015, 23:17:05 »

Quote from: reyalp on 14 / March / 2015, 23:43:12

Overlay off, set_yield(-1,50). I figured if sleep made it go away, yielding less would make it happen more often. Nope. No failures in >6800 shots.

I did a further test with set_yield(-1,-1): >7100 shots with no failures.

At this point I'm fairly confident that it's a timing issue related to the script rather than a hardware failure. The exact scenario isn't very clear though. The normal shooting cycle goes like

wait_click 10 to check for key presses
press shoot full
sleep until remote hook is hit
sleep for as long as needed to achieve desired interval, typically a few hundred ms in my tests
continue from remote hook
release shoot full
sleep until raw hook is reached
do some intensive processing: meter, exposure calculation, debug drawing
* meter and drawing can take 10s of ms, so will likely result in a yield under default settings.
* there is also a lot of code, so the count hook might also trigger one or more yields
continue from raw hook
write log line
do lua garbage collection step
goto 1

The above works without issue if debug drawing is enabled. This typically adds 10-20 ms on d10, so would normally cause one extra yield inside the raw hook.

Metering takes a similar amount of time, so with metering alone, there is probably at least one in the raw hook. In the cases that failed, the meter time is shown as 10 on both failed and successful shot.

Both meter and drawing will tie up the CPU pretty solidly when they are actually running, although the dryos scheduler might preempt them.

capt_seq is blocked for the duration of the hook.

The two "fixes" I have found are
1) adding an extra yield inside the raw hook, with sleep(10)
2) preventing any yield with set_yield.

The script yields almost immediately after the raw hook as been released, so it's not clear why 10ms inside the hook should make a difference. The obvious difference is that capt_seq has been allowed to continue, which will keep the cpu busying for a while creating the jpeg and may also interact with other tasks.

If there was a way to figure out when the shutter normally opens, that could provide some clues.

Logged

Don't forget what the H stands for.