Timing problems with "shoot_full_only"

lapser

1093

« on: 04 / December / 2012, 22:53:22 »

Advertisements

I've re-written shot_histogram.c so it now measures exposure from the raw buffer. I tracked down some strange problems to a timing issue. I was analyzing the histogram data while build_shot_histogram was still reading the raw buffer and calculating it.

In Lua, I call press("shoot_full_only") and wait until get_exposure_count() changes. But my tests show that this happens BEFORE build_shot_histogram() is called. I need to be able to detect when the camera has finished processing the shot and is ready for another shoot_full_only.

I've seen some discussion of this problem, but I haven't seen an answer. Does someone know how to solve this problem.

I need a function like get_shooting(), that works with shoot_full_only, when you don't release half_shoot. I.E. something like this:

get_shot_ready()

In shot_histogram.c , I can set a volatile variable when build_histogram finishes, but I'm also interested in when the entire shot is finished, and ready for another shoot_full_only. Any ideas or other help would be greatly appreciated.

Thanks

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

waterwingz

12138

Re: Timing problems with "shoot_full_only"

« Reply #1 on: 04 / December / 2012, 23:44:26 »

umm .. I almost hate to ask this .. but you've been here a while ... so .. umm .. why are you starting new threads all the time .. for stuff that belongs in threads you are already posting related stuff in ? http://chdk.setepontos.com/index.php?topic=9056 It makes it harder for the older members of this form (like me) to keep track of ...

never mind .. not worth the flame war .. sorry

« Last Edit: 05 / December / 2012, 00:02:55 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Timing problems with "shoot_full_only"

« Reply #2 on: 05 / December / 2012, 00:26:57 »

Quote from: waterwingz on 04 / December / 2012, 23:44:26

never mind .. not worth the flame war .. sorry

I have nothing but respect for you and all you do for CHDK, so I only have apologies if I'm making things hard for you and others. I have a lot to learn, including how things are done here. Please feel free to criticize or make suggestions, and I'll do my best to follow you're instructions.

So I promise I'll only start one more thread... when I release the shot_histogram modification source. I'm hoping you'll be the main one to try it out, and help make improvements.

I think this timing problem is at the root of all the strange bugs I've been uncovering lately, including the one in print_screen that I thought was related to your modifications. I'm hoping that someone who understands the inner workings of CHDK and shed some light on what's happening between different tasks. Is there a list of all the tasks that are running, and what they do?

What's the best way to communicate between tasks. I tried to set a variable in build_histogram and read it in get_histo functions called by Lua. It was unreliable. I just changed the flag variable to "volatile" and that seems to work, but I'm not sure. Does anyone know more about this?

And don't worry. Once I get this shot_histo thing working, I'm planning to spend a lot of time out in the snow taking pictures, so you'll have a break!

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Timing problems with "shoot_full_only"

« Reply #3 on: 05 / December / 2012, 02:37:00 »

The control flow goes like this:
- Pressing the shutter button activates the code in 'capt_seq_task' (which runs as a task in the camera)
- The CHDK modified version of 'capt_seq_task' calls 'capt_seq_hook_raw_here' when it thinks the image has been captured and is ready in the raw buffer memory.
- 'capt_seq_hook_raw_here' signals the CHDK 'core_spytask' task that a raw image is ready, then waits until 'core_spytask' tells it that the raw processing is finished
- 'core_spytask' calls 'raw_savefile' to process the raw data, this includes calling 'build_shot_histogram' if needed
- after 'raw_savefile' is done, 'core_spytask' signals 'capt_seq_hook_raw_here' to tell it raw processing is finished
- 'capt_seq_hook_raw_here' then returns to the normal firmware code to create and save the JPEG image.

You can add a flag in 'build_shot_histogram' to tell you when it is finished; but if 'get_shooting' does not work when using 'shoot_full_only' then I'm not sure if there is any easy way to know when the camera is ready to take another shot.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Timing problems with "shoot_full_only"

« Reply #4 on: 05 / December / 2012, 03:55:27 »

Thanks a million. I was tracking back through those routines trying to figure it out, but that explains it completely.

Quote from: philmoz on 05 / December / 2012, 02:37:00

- 'capt_seq_hook_raw_here' then returns to the normal firmware code to create and save the JPEG image.

You can add a flag in 'build_shot_histogram' to tell you when it is finished; but if 'get_shooting' does not work when using 'shoot_full_only' then I'm not sure if there is any easy way to know when the camera is ready to take another shot.

I think I figured out how to get the correct results in shot_histogram at least. That was a tough bug to figure out. Thanks for your help with my log2 routines. I threw them all out, but that's how I figured out the timing bug.

I did some experimenting with watching the card memory available to see when it changed from saving the jpg. It didn't work all the time, so I gave up. But knowing that's the last thing in the camera sequence, I'll try again. I think a lot of cameras buffer the saving of pictures, especially in continuous mode.Maybe I can check for when the jpg appears, or the card free memory stops changing for a certain length of time. All that will be in Lua. It should be fun.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Timing problems with "shoot_full_only"

« Reply #5 on: 05 / December / 2012, 12:03:00 »

Here's another idea, and question. Is there any way to know that a shot is in progress after presssing shoot_full_only, particularly with long shutter times? It seems like a long exposure completely blocks CHDK while the shutter is open? Is this correct?

I'm thinking I could click shoot_full_only, loop with a short delay checking the difference between each tick time and the previous one. If that difference is a little greater than the shutter time, I know something, hopefully the camera taking the picture, blocked me and it worked. If I time out without a big difference, I could try clicking shoot full only again and repeat the process. It wouldn't be necessary if CDHK knows the camera started taking the picture, even if it doesn't know it finished.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14079

Re: Timing problems with "shoot_full_only"

« Reply #6 on: 06 / December / 2012, 01:12:15 »

Quote from: lapser on 04 / December / 2012, 22:53:22

In Lua, I call press("shoot_full_only") and wait until get_exposure_count() changes. But my tests show that this happens BEFORE build_shot_histogram() is called.

Since no one else spelled this out:
shot_histogram is done by the same code that does raw
raw is not done until after the counter is updated

There isn't currently a way to know when the shooting process is completely finished.

Quote

It seems like a long exposure completely blocks CHDK while the shutter is open? Is this correct?

This cannot be correct, or this wouldn't work http://chdk.setepontos.com/index.php?topic=4337.msg61335#msg61335

Logged

Don't forget what the H stands for.

lapser

1093

Re: Timing problems with "shoot_full_only"

« Reply #7 on: 06 / December / 2012, 02:11:09 »

Quote from: reyalp on 06 / December / 2012, 01:12:15

shot_histogram is done by the same code that does raw
raw is not done until after the counter is updated

Thanks. I figured out that this was the cause of my timing problems. By using the counter as my trigger to check the histogram data, I was checking it at exactly the time when it was being built. The result from my function depended on the timing, so my slower log2_96 computation happened a little later than the faster log2.

I added an instruction that stores a 0 in a volatile variable at the end of build_shot_histogram, that I can poll from Lua. Then I wait for that to change, rather than the shot counter. It works!

Quote

Quote
It seems like a long exposure completely blocks CHDK while the shutter is open? Is this correct?
This cannot be correct, or this wouldn't work http://chdk.setepontos.com/index.php?topic=4337.msg61335#msg61335

Pretty nice script! Thanks for the link.

Yes, of course it can't be true. I've done lots of tests where I print something in my polling loop waiting for get_shooting() or the counter to change, and there's no time gap. CHDK is fully awake during long shutter times. The camera stops updating the display with the live image, so I thought everything had stopped.

I'll go through the get_shooting() function and see if I can get it to work with shoot_full_only, or maybe add a get_shot_ready() function that does. Thanks for your help in figuring this out.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Throwing away the shot after build_histogram?

« Reply #8 on: 06 / December / 2012, 11:13:12 »

So build_histogram is called after the picture is taken, but before it is converted and saved as a jpg, as evidenced by the changes I make with set_pixel appear in the jpg.

So my question is, after calling build_histogram, is it possible to abort the saving of the jpg? I suspect it isn't possible, since saving CHDK raw files always saves the jpg.

I've got my shot_histo functions working so you could use it as a camera meter, without the meter. The first shot is used for metering, and a second shot uses the exposure change calculated from the first shot.

An alternative would be to rename the first shot, with Lua, from IMG_xxxx to MTR_xxxx. That way, you could sort by name to separate them.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

srsa_4c

4451

Re: Timing problems with "shoot_full_only"

« Reply #9 on: 06 / December / 2012, 20:51:58 »

Quote from: lapser on 06 / December / 2012, 11:13:12

is it possible to abort the saving of the jpg?

You can try to hack in this kind of functionality by using this branch: https://trac.assembla.com/chdk/browser/branches/ptp-remote-capture-test
You'll need to implement the FileWriteTask hook for your cameras. Reference implementation is the A810 for DryOS>=r50 (your SX260), Ixus870 or A470 for earlier DryOS cameras. If you're interested and have questions, just ask.

Quote

An alternative would be to rename the first shot

Also possible to implement with the FileWriteTask hook (rename before saving, downside: the camera probably won't recognise the files).

The above testing branch also has a change related to file save detection, see core/main.c. This one should work (if it works the way it's intended) with your cameras as is.

Logged