Shot Histogram Request

lapser

1093

Timing Problems Solved

« Reply #70 on: 19 / December / 2012, 14:18:01 »

Advertisements

The fastest way to take multiple shots is in continuous mode (at 2 or 3 shots per second). But there is no way to control the shot rate, or meter exposure. You can change the exposure between shots (i.e. bracketing in continuous mode), but you don't know how much to change it.

The new get_histo_meter() function returns an exposure value based on the previous shot that works well for measuring exposure. I'm also working on turning the histogram of the last shot into a meaningful exposure correction value.

Now, I've also solved the timing problem in continuous mode. It depends on the new function:
set_shot_interval(interval, delay)

"interval" is the time, in msec from the last shot, to wait BEFORE taking a shot. The delay occurs right before the shutter opens, and sets the interval between shots very precisely (within 10 or 20 msec).

"delay" is the time to wait AFTER the shot. The delay occurs right after the shot meters and shot histogram are ready, and "shot_number" has incremented. The lua script is waiting for "shot_number" to increment, and then it reads the shot meters and sets the exposure for the next shot. But I found that shots are going by so fast in continuous mode that there wasn't enough time for the script to finish. The "delay" after the shot gives the script time.

The last piece of the puzzle is for the script to be able to abort the delay when it is finished with the histogram and meters. This is done with:
set_shot_interval(-1)

Here's how to do a time lapse as fast as possible, with exposure correction between shots:

Code: (lua) [Select]

set_shot_meter(1) -- enable meter #1 with full sensor measurement area
set_shot_interval(0,500) -- no pre-shot delay, 1/2 second (max) post shot delay
-- if you want your shots at 1 per second: set_shot_interval(1000,500)
ishot=0 -- get_shot_number() always starts at 0 now
press("shoot_full")
repeat
  nshot=ishot
  repeat
    sleep(10)
    ishot=get_shot_number()
  until ishot~=nshot --signals new shot is ready
  -- post shot delay starts here
  if(nshot==0)then tv=get_tv96_direct() end-- metered by camera on first shot
  tv=tv+get_shot_meter(1) -- adjust exposure time
  -- more processing here if needed, i.e. write log file
  set_tv96_direct(tv) -- happens immediately now instead of next half shoot
  set_shot_interval(-1) -- abort post shot delay. Camera takes next shot as fast as possible now
until false

An unexpected bonus from using this method is that the pictures are displayed on the screen (even with review off). In other shooting modes, the live view is displayed, and the screen goes blank during the exposure. At night, with long exposures, all you see is the screen flash between shots. But with this method, the screen shows the last shot for the entire duration of the long exposure. This is a big improvement.

« Last Edit: 19 / December / 2012, 14:36:14 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #71 on: 20 / December / 2012, 21:12:55 »

I tried a time lapse at 1 shot per second with the new functions. Lua stopped working towards the end, while I was stopped in the shopping center parking lot. The message was:
1219010
Press Shutter To Continue

But the interesting thing was that the pictures kept right on coming. The log and shot metering stopped, and the frame rate must have dropped, since the post shot delay was timing out at 1 second. But you can't even tell where Lua stopped working when looking at the pictures and video. Note that the video continues past the stop at the shopping center where Lua stopped. I'm pretty sure the cause was closing and reopening the log file in case of a power loss. This sometimes took longer than 1 second, so the camera went on to the next shot before the lua script was ready. Whatever happened, it wasn't good. A longer, or no time out may be better. All delays are aborted when a script ends, so if the camera hung from a script error, it would be cleared when the script was ended with a shutter press.

CHDK Time Lapse Test

The attached log file shows how accurate the pre-shot interval delay was. It was always exactly 1000 msec, except where I closed and reopened the log file. The second timing number measures the get_shot_ready() to get_shot_ready() time, i.e. when the shot meters are ready.

I got rid of the shot counter and added these new functions, which worked and are much easier to use and understand than the shot counter:

get_shot_ready() -- true when shot_meters and histogram are ready
reset_shot_ready() abort post shot delay after processing shot data and reset shot_ready flag for next shot

H_1212201251.LOG (91.89 kB - downloaded 46 times.)

« Last Edit: 20 / December / 2012, 23:46:38 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

New Time Lapse - Timing Progress

« Reply #72 on: 28 / December / 2012, 14:39:46 »

I've made a lot of progress with the timing of shots in the last few days. I set up a handshake from the camera shooting routines in continuous mode, to the Lua script so they wait for each other. This was my first test of the new routines, on the SX260 and also the G1X. Here's the G1X video (watch in HD):

http://www.youtube.com/watch?v=1-Dez4slXU4#ws
It's instructive to look at the first two frames, and the last frame of the video. Here they are, in full resolution:

This shot was metered by the camera with the standard center metering area. I set 4 shot metering areas, one in each quadrant, which are drawn over the first picture. If you click on the picture and zoom in, you can see that the areas are slightly separated from each other.

This shot was taken about 1/2 second after the first shot, and is metered by the new shot_meter functions using the first shot. Each metering area returns an exposure correction to the script. The script picks the darkest correction, and sets the exposure for the next (this) shot. This results in optimum exposure of the sky, and underexposure of the darker part below, which is what I wanted.

This is the last shot of the video, when the SDHC card filled up (with RAW images). The camera kept on shooting correctly exposed images that were never saved, but were logged and timed. I knew something wasn't correct, but didn't realize it was a full card. I'll have to check for this in the future. The log says that the script was ready for this shot 730 msec early (out of a 1000 msec total interval). That's pretty good. However, the next shot was 1200 msec LATE! That's horrible. The script spent about 2 full seconds hung up by the file save problem, it would appear, which delayed the next shot because of the hand shake.

The result of this test is that I realized that the Lua script can't keep up with the speed of continuous mode shooting. I'll move the exposure adjustments into the C shoot loop, and the script will just monitor what's going on, enter new exposure compensation values occasionally, and start/stop the whole thing. I know this will work because I've done it by setting my interval to 0, which disables the hand shake. The script misses shots, but still sets exposure whenever it can, so it looks ok for a time lapse. By moving the set exposure part into the C shoot loop, each shot will be correctly exposed.

I'm not sure I've gotten across how useful this all is. Time lapses come out crystal clear, and slow enough to appreciate, when each shot can be a single frame. This requires high speed shooting. And with a picture every 1/2 second, you're pretty close to having a video, but with 4000x3000 resolution, 14 bit dynamic range, and accurate exposure using shot metering. The pictures in this time lapse have enough resolution to zoom in on the post office and watch the trucks come and go. The first picture shows 4 trucks at the post office loading dock. The last picture shows 5 trucks. You could look through the pictures and find exactly when the 5th truck arrived, or make a video zoomed in at the post office lot. Right click on the picture, and save or open it in a new tab to see the full resolution image. The place I'm talking about is in a notch in the trees towards the lower, right center of the image.

Also, I was showing some friends how fast the camera took pictures in continuous mode, and how it still adjusted the exposure between shots. They hammed it up in front of the camera for awhile, but when they stopped, there was one image where their phony smiles became real smiles for just a moment. Those moments don't happen very often, and you don't capture them with correct exposure unless you're really lucky. High speed multi-shot metering could be an extremely useful feature for point and shoot cameras. Stay tuned.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

waterwingz

12122

Re: Shot Histogram Request

« Reply #73 on: 31 / December / 2012, 14:46:19 »

Quote from: lapser on 18 / December / 2012, 23:06:49

Rather than releasing new diff files, it might be easier to send them to philmoz to put in his branch?

@lapser : I've scanned as many of your threads as I could find looking for patch files. I have a few moments and I thought I'd like to try out some of the tricks you've learned. But the only one I could find is the recent one in the "Improvements to press, click, release for shooting keys" tread and that's not what I'm looking for.

Did you ever pull together anything on your histogram metering & continuous shooting sync / timing stuff ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Shot Histogram Request

« Reply #74 on: 31 / December / 2012, 22:19:24 »

Quote from: waterwingz on 31 / December / 2012, 14:46:19

Did you ever pull together anything on your histogram metering & continuous shooting sync / timing stuff ?

I did get a little side tracked on the key function changes. I finally found that all day bug, and have it working.

I'll make it the top priority to get a new diff file out, and an understandable demo Lua script. I'll post it here as soon as possible. Thanks for your interest, and your help.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

Lightning_Lee

Re: Shot Histogram Request

« Reply #75 on: 02 / January / 2013, 01:51:26 »

lapser, I absolutely love your work mate, I would like to go thank everyone in this forum/community but not to be annoying...I won't.
I have followed your work now for a while at the same time as trying to get my head around using and working with CHDK. The things you are working on will come in very handy for me one day, that is when I get a few P&S's(already ordered) and see what I can't blow up...I mean figure out!

Going to be a steep learning curve for me but one day I hope I can help out some and give back something!

Cheers again!

Logged

Just want to say thanks too to all the people around here,without you sad little leeches like me get nothing,I know some of us give leeches a bad name but while skinny on programming ability/understanding/knowledge if there is any other way I can help,let me know or PM me,I am more than happy to.

lapser

1093

Re: Shot Histogram Request

« Reply #76 on: 02 / January / 2013, 04:32:49 »

Quote from: Lightning_Lee on 02 / January / 2013, 01:51:26

lapser, I absolutely love your work mate, I would like to go thank everyone in this forum/community but not to be annoying...I won't.
I have followed your work now for a while at the same time as trying to get my head around using and working with CHDK. The things you are working on will come in very handy for me one day, that is when I get a few P&S's(already ordered) and see what I can't blow up...I mean figure out! Going to be a steep learning curve for me but one day I hope I can help out some and give back something!

Thanks for the comments. I'm looking forward to your input. Let me know if there's anything I can do to help, and don't be afraid to ask stupid questions. Hopefully, I'll have something ready for you to try out when you get your CHDK camera going. Good luck.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

New Sunset Time Lapse Tests

« Reply #77 on: 04 / January / 2013, 00:21:36 »

I finally got a real sunset to test the new shot metering routines. It was too cold to stay very long after dark, but I got a nice smooth fade with both cameras, with no flashes.

The G1X was pointed west, with the sun in the initial frame. I don't have the script set to change anything but Tv, so I set ISO to 800 for the dark part. But the sun was way too bright. I set the aperture as small as possible, which helped a lot. The shot interval was 1.5 seconds per shot. Here's the video from the unedited pictures at 24fps:
http://www.youtube.com/watch?v=Qi8lkk6SlWQ#ws

I also set up the SX260 at 2 shots per second pointed East, starting about 5 minutes before official sundown. It was clear enough to see the snow covered 3 Sisters volcanoes on the horizon. The sx260 hit the maximum 6 second shutter speed I set for it, and got dark. But the stars and cars were visible. At 2 seconds apart, and 1 frame per shot, the car motion seemed smooth at 48X normal speed. Here's the video:
http://www.youtube.com/watch?v=Ymq2E5hMH_E#ws

I've attached the 2 log files, which show the timing variables, and the shutter speed for each shot. It's interesting that all the times for the SX260 are slightly faster than for the G1X. The G1X uses the Digic 4, while the SX260 has the Digic 5. This might be the reason for the difference.

G1X_Spencer_130102.LOG (157.63 kB - downloaded 32 times.)

SX260_Spencer_130102.LOG (79.89 kB - downloaded 36 times.)

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

One shot per second test

« Reply #78 on: 10 / January / 2013, 01:39:34 »

I had to hurry off the hill before the rain/snow shower arrived, but this time lapse was interesting while it lasted. And it lasted for over 2,000 shot! One shot per second is really fast!
http://www.youtube.com/watch?v=WFdCfd5VkjE#ws

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Two New Time Lapse Videos

« Reply #79 on: 17 / January / 2013, 23:48:43 »

I used both cameras last night at 1 shot per second for over 2,000 shots each. Both cameras stopped with a lua error number (large number). I've seen this happen before. I think it's related to writing a lot of data to the log file at the same time that the camera is writing pictures in continuous mode as fast as it can. I've modified the script to print only to the screen and not to a log file. I'll try to test it again soon.

I used 4 shot meters, one in each quadrant, on both cameras. The script is set up to pick the brightest meter, which gives the darkest picture. Because of the moving fog below the top of the mountain, where I was, the brightest meter was on the moving fog, and there was some flickering of exposure when looking at the sky above the fog. It would have been best to move the meter to the sky. The meter should be on something that isn't changing brightness, like empty sky, or even a card in front of the camera.

I also modified my exposure algorithm so it adjusts shutter speed until it reaches 1 second, then adjust ISO until it gets to 800, then goes back to shutter speed. The transition isn't noticeable.

http://www.youtube.com/watch?v=OzZ3giiQcNg#ws

http://www.youtube.com/watch?v=sUAnTJ_LFGI#ws

In addition to the log file write conflict, I discovered a bug in the depth of field function in Lua that philmoz has already corrected. While investigating the bug, I improved the focus part of the script so it focuses faster, and restores auto-focus when the script exits, instead of leaving the camera in manual focus.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos