Shot Histogram Request

straessi

14

Re: Shot Histogram Request

« Reply #420 on: 17 / September / 2013, 16:00:38 »

Advertisements

Quote from: lapser on 13 / September / 2013, 14:03:28

There are no real advantages of single shot mode with my time lapse script, except the possibility of losing pictures with an external battery. The solution to that problem isn't to use single shot mode though. It's to stop the script when the external battery gets near its low voltage cutoff.

That's nice.

I now always use the continuous mode and it works quite smoothly.
I didn't do much testing recently, because it's raining a lot here in Switzerland... But we hope to get a couple of nice Autumn days soon.

@lapser: Thanks again. I really appreciate the awesome work you did!

Logged

lapser

1093

Re: Shot Histogram Request

« Reply #421 on: 17 / September / 2013, 19:37:35 »

Quote from: straessi on 17 / September / 2013, 16:00:38

I now always use the continuous mode and it works quite smoothly.
I didn't do much testing recently, because it's raining a lot here in Switzerland... But we hope to get a couple of nice Autumn days soon.

That's good news. I'm excited to see what you come up with. Switzerland is such a beautiful country.

The extra print statement I added to print more log data causes an extra delay about every 10th shot. I think the camera is filling up a memory buffer with pictures in continuous mode, and then writing them out all at once. The camera keeps on taking pictures while it's writing, but when the script tries to write a log at the same time, it seems to block until the pictures are written first. I like the extra log data, but I need to re-do the script so it writes all the data in one print statement instead of two.

I think I figured out a way to vary the white balance and brightness, and everything else, smoothly during the video without going through LRTimelapse and Lightroom (and the 24 hours or more of computer time that takes). I first turn all the pictures into a video clip with VideoMach. Then I use the TMGENC encoding program to turn the clip into a finished mp4 video file.

TMPGENC can adjust colors and brightness of a clip, but the adjustment is the same for the entire clip. But you can put the same clip on 2 different tracks, and fade between them. So I just need to create a duplicate clip and adjust the white balance for it based on the end of the video. Then I can gradually fade from the original clip to the duplicate (white balance adjusted) clip. I'll try it for the next video.

It rains all winter here, so I've had to figure out a way to protect the G1X in the rain. The underwater housing is way too expensive, and I'm not planning to take it diving. I got an underwater bag, but it can't be mounted to a tripod. So here's what I came up with.

Unroll a long piece of plastic wrap and place it flat on a table. Turn the display facing out. Place the camera, display side down, on the plastic wrap. Then fold the wrap around the camera, and put a rubber band around the base of the lens, holding the plastic in place. Then cut the excess plastic wrap off so it doesn't cover the lens.

If you're using an external battery, Scotch tape the wrap around the cable. Now screw the tripod attachment through the wrap into the bottom of the camera. Place the camera on the tripod, turn it on, and line up the shot. Use a lens hood if you have one, since it helps keep sprinkles off the lens. Now tear off a 2nd, narrow piece of plastic and wrap it around the lens barrel. Tape the wrap to the lens hood.

Now, the entire camera, except the front of the lens, is covered in plastic, and water resistant. It also holds in the heat of the camera enough to keep the lens from fogging up, at least under the conditions I've used it so far. I left the G1X out all night using this set up to get this video:

http://www.youtube.com/watch?v=luIXZWJUFvw#ws

Here are two more recent videos that came out pretty interesting:

http://www.youtube.com/watch?v=u1yy1qXU-WU#

http://www.youtube.com/watch?v=-2JqVZzniz8#

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #422 on: 18 / September / 2013, 18:59:23 »

I changed the script to use "write" instead of print_screen and print to see if I could write 2 lines of log data at a time faster. It worked OK on the SX50, but caused lots of >1000 msec delays on the G1X for some reason. The Lua system handles a "write" to SD, but CHDK does the file writing with print, so CHDK must be a little faster. Back to the drawing board, I guess.

The title of this topic is Shot Histogram Request, but I still haven't finished the shot histogram functions, other than making the existing function more accurate. I want to use the shot histogram to adjust exposure, like the shot meters, so I need to return Apex96 values, like the meters. I also want to get the full accuracy of the 14-bit sensor on the G1X, but that takes 16 times the memory, or 32K for the full histogram, instead of 2K for the current, bit shifted histogram.

My idea today is to compute the histogram in logarithmic, Apex96 units instead of linear. That way, I'll only need 500 or 600 values to store the histogram, without losing any accuracy with 12 or 14 bit sensors. It may take a little longer to calculate what Apex96 value corresponds to each pixel value, but it should be doable with integers using bit shifts and table lookups. I'll experiment with it in the near future.

For the time lapses last night, I set the underexposure at -500 Bv to 0, instead of the usual 192. Then I used my video rendering program to fade the video using set points to try to get it to look right. It came out ok, but I think the slight "fluttering" is still there, although maybe not as noticeable as with 192. The disadvantage is that the shutter times get longer quicker, speeding up the end of the video. Plus, part of the picture where the sun set ended up overexposed and blown out, so it couldn't be corrected by lowering the brightness. So I'll probably go back to using 192 for now. When I get the histogram feature working, maybe I can try again without blowing out the pictures.

Here's the two videos from last night:

http://www.youtube.com/watch?v=2SPOG_qbgtA#

http://www.youtube.com/watch?v=2SPOG_qbgtA#

Before turning off the SX50, I turned it at the moon and did a time lapse of the moon going through a shot meter area box. As the moon moved out of the metering area, it got brighter slowly until it was blown out. But on the way, I caught this perfect exposure (attached)

Moon130917_5450crop2.jpg (28.36 kB, 593x535 - viewed 39 times.)

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #423 on: 19 / September / 2013, 15:15:38 »

I did another time lapse with the sun in the frame, causing the shutter time to hit the minimum 1/3200 seconds. That pesky fluttering at high shutter speed is still visible. My idea of overexposing a little and turning down the brightness in post wouldn't work here. It doesn't really work for dark scenes either, since after you turn down the brightness, the variation in exposure is still there.

I recently added the 4 meter readings to the log when I was trying to track down the G1X major flickering problem. That turned out to be a problem with the G1X ports we haven't tracked down, but it doesn't happen in continuous mode, so it's on the back burner for now. But the extra log data has proven interesting.

While the D20 was at 1/3200, you can see the fluttering in the video, and the meter data shows it. All 4 meters change by the same amount, about 7 ev96 units. This shows that it's a problem with the actual exposure. The camera isn't quite accurate enough to time a 1/3200 second exposure perfectly every shot. So this kind of exposure variation could be corrected in post with Photoshop Lightroom, and a program like LRTimelapse.

I tried my moon going out of the meter box method of exposure bracketing (kind of) last night, and it worked pretty well. The moon gets a lot brighter when it's full, and loses some contrast, so the detail isn't quite as good as last night, but it was still pretty good. I was hoping to get a time lapse of the full moon rising, but the sun hadn't quite set when the moon rose, so it was too light out to get a clear moon shot (maybe next month).

http://www.youtube.com/watch?v=b1RL49rgfY0#

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #424 on: 28 / September / 2013, 00:03:19 »

The rainy season in Oregon is starting again, so I'm planning to get some good use out of the waterproof D20. The problem isn't the camera getting wet, it's the droplets on the lens. The wind, and rain, usually come out of the West here, so the lens is usually pointed into the wind for a sunset.

When it's raining, I need to wipe the lens off every once in awhile. But if I take a picture while wiping, the script thinks it got suddenly darker, and overexposes the next picture, causing a flash. I have a "glitch" detector that ignores one picture with a large exposure change. But if you're wiping for 2 pictures, look out. I need to modify my glitch detector to ignore more than 1 photo.

In this time lapse, I caused a flash in the D20 when wiping the lens. I removed a few frames so the flash isn't visible, but the clouds jump suddenly. I wiped faster in the 2nd part of the video (SX260), and didn't trigger a flash.

http://www.youtube.com/watch?v=Sx1exJJHepw#

I took the SX50 and G1X to a pretty new place for the next video. I set the shutter minimum to 1/2000. This worked great on the G1X, but the SX50 showed flickering until the shutter time got to 1/500. I've been measuring the ND filter density, and trying to figure out how to automatically shift it in and out without a visible change. This way, I can keep the shutter times slower, and hopefully prevent the flickering. Here's the video. The flickering SX50 starts half way through.

http://www.youtube.com/watch?v=-3osLQs6uHE#

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

Microfunguy

4037

Re: Shot Histogram Request

« Reply #425 on: 09 / October / 2013, 11:26:51 »

Well Mr.Lapser,

I am exhausted after reading this entire thread.

Bottom line is, do we have to request a custom build from yourself or can we compile it ourself ?

Please tell me which of the 43 pages is the most relevant to this.

Thanks.

David

Logged

waterwingz

12139

Re: Shot Histogram Request

« Reply #426 on: 09 / October / 2013, 12:01:34 »

Quote from: Microfunguy on 09 / October / 2013, 11:26:51

Bottom line is, do we have to request a custom build from yourself or can we compile it ourself ?

He posted a patch file here :
http://chdk.setepontos.com/index.php?topic=8997.msg103871#msg103871
I cleaned it up a bit and posted that immediately after in the same thread.

Logged

Ported : A1200 SD940 G10 Powershot N G16

Microfunguy

4037

Re: Shot Histogram Request

« Reply #427 on: 09 / October / 2013, 12:17:50 »

Quote from: waterwingz on 09 / October / 2013, 12:01:34

He posted a patch file here :

Thanks, I will check it out.

The main question is does averaging the values of an equal number of R,G,B pixels accurately relate to scene brightness ?

I think early on in the thread Reyalp (or Phil) suggested they do not, the values need de-bayering and 'gamma correction' curves applied.

Logged

lapser

1093

Re: Shot Histogram Request

« Reply #428 on: 09 / October / 2013, 18:22:42 »

Quote from: Microfunguy on 09 / October / 2013, 12:17:50

The main question is does averaging the values of an equal number of R,G,B pixels accurately relate to scene brightness ?

I think early on in the thread Reyalp (or Phil) suggested they do not, the values need de-bayering and 'gamma correction' curves applied.

In practice, it's proven extremely accurate. It's much more accurate at low light levels that the built in camera meter. The get_shot_meter function returns an apex96 value that is used to compute the exposure for the next picture. As long as there are no blown pixels in the metering area, and the shot isn't severely underexposed, the next picture is usually right on the money.

Remember, the goal isn't to measure the absolute brightness of the image. It's to measure the exposure change needed, which is the difference in brightness between two images. This is best done on the raw pixels, since doubling the shutter time, for example, doubles the linear value of all the raw pixels.

The first shot in my time lapses is, by necessity, metered by the camera. Then the second shot is adjusted based on the get_shot_meter() reading of the first shot. In this video, I held the first two shots for 5 seconds. Note that the exposure changes quite a bit on the second shot, but then stays steady after that. This means that the second shot was exposed correctly based on the first shot.
http://www.youtube.com/watch?v=MQah7sksQaw#ws

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

Microfunguy

4037

Re: Shot Histogram Request

« Reply #429 on: 10 / October / 2013, 05:23:07 »

Thanks.

Whether the exposures are technically correct I am not sure.

However, the result is effective.

It is a very difficult problem.

I have a similar location a short walk away (in Wales) though overlooking a town rather than a city and with hills in the background.
I will ask the farmer for permission to leave my equipment setup for a few hours.

Wonder if taking the exposure reading from a grey card at a corner of the view would be better ?

David

« Last Edit: 10 / October / 2013, 07:41:10 by Microfunguy »

Logged