Shot Histogram Request

lapser

1093

Finished Time Lapse Video

« Reply #80 on: 23 / January / 2013, 01:26:43 »

Advertisements

http://chdk.setepontos.com/index.php?topic=9270.0
The original request I started with was to improve get_histo_range so it had more precision. Take a look at the above link, where the new functions are actually used to successful set exposure for ETTR (exposure to the right), and maximum dynamic range.

I've been a little side tracked, but I did find the time to make this finished time lapse video from my pictures last Wednesday above the fog at sunset:

http://www.youtube.com/watch?v=4pAmnpii3HU#ws

I ran the pictures through lightroom for leveling and color tweaking, and added some background music. The effect is kind of surreal.

I did a similar time lapse during the summer from Mary's Peak, with clouds below that looked like the ocean. The shot rate was much lower back then, and I processed the video by cross-fading between frames. It works, but takes all day to do a single video. It's much easier now with 1 shot per frame, and no cross-fading. Here's the Mary's Peak video:

http://www.youtube.com/watch?v=Ed_HXjPew2A#ws

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

pigeonhill

1155

Re: Shot Histogram Request

« Reply #81 on: 23 / January / 2013, 09:03:32 »

Lapser

Great timelapse work. I see you use LR. I also use LR for my timelapse work, but with LRTimelase. I find LRT to be the ideal adjunct to LR. Have you looked at LRT?

Also how do you use your ETTR calls?

Cheers

Garry

Logged

lapser

1093

Re: Shot Histogram Request

« Reply #82 on: 23 / January / 2013, 14:54:49 »

Quote from: pigeonhill on 23 / January / 2013, 09:03:32

Great timelapse work. I see you use LR. I also use LR for my timelapse work, but with LRTimelase. I find LRT to be the ideal adjunct to LR. Have you looked at LRT?

Also how do you use your ETTR calls?

Thanks! The higher speed shot interval really helps.

I haven't written any ETTR calls yet. The idea is to return an Apex96 expsoure value that you can add to Tv96 to reach ETTR on the next shot. I'm thinking it could be something like this:

tv96=get_ettr(5)+tv96-32

This would mean 5/1000 of the pixels can be above white leve -1/3 fstop. You could have a similar function for ETTL. Does that sound like it would be useful?

====
I've used LRTimelapse, but I don't think it's worth the $100. I also don't like the way he writes warnings that you haven't paid him in all the pictures.

My get_shot_meter() function allows smooth exposure changes without the need for post processing in LRTimelape. It's much better to get the exposure right the first time (sounds like a Billy Joel song).

It would be nice to be able to adjust the white balance in CHDK. They always say not to use auto white balance with a time lapse, but it might work, depending on how course the adjustment is. I'll experiment around a little.

The only useful thing LRTimelapse does for me at the moment is smoothly change Lightroom settings between two key pictures. The implementation is pretty clunky, though. I noticed the smoothing curves are wrong too. They overshoot the key points.

Eventually, I'll write a LRTimelapse replacement that just modifies the XMP data of pictures between Lightroom key points. It's just text manipulation, so I may even do it in Lua for Windows, which I already have.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

pigeonhill

1155

Re: Shot Histogram Request

« Reply #83 on: 23 / January / 2013, 15:58:08 »

Lapser

The timelapse community talk of the holy grail technique, that lets you address jumps in the exposure, because of ISO jumping etc. I believe the other 'holy grail' is how to achieve in-camera ETTR. I believe CHDK, and your histo functions, could lead the way here, at least until the next generation of camera come out with this feature built in.

I say this, as your histogram calls get us further along the line, as you are moving us away from being restricted to using the low data content of the 8-bit LCD-like histograms that the camera gives us. Thus the current ETTR strategy has to go something like this: take your first guesstimate shot, chimp at the histogram and or the 'blinkies', adjust as far as you think you can go, based on a JPEG histogram created from a very downsampled in-camera image; hope you haven't overshot and rely on a bit of RAW headroom in post.

Your calls give the in-camera ETTRer some tools, however, I think the real challenge, as you imply, is to come up with a suitable algorithm to maximize the probability of getting the exposure as far to the right as possible, without blowing out too much of the image. Hence, the first algorithmic problem, how do you differential between specular and blown out data that you can live with, and bits of the image that you don't want to be blown out?

In other words, ETTR to me implies someone wishes to take a single image and ensure the maximum RAW data content. But what about the bracketers!

When I do auto-bracketing (for HDR), I believe, at one level, I am addressing ETTR and ETTL, at the same time. The difference between single-image-capture ETTR and auto-bracketing-based ETTR is that one creeps up on ETTR (and ETTL) over several brackets with one, whilst the 'holy grail' ETTR script would attempt to 'jump' to the ETTR 'bracket' in the minimum number of goes, ideally 2 or 1.

When I use my auto-bracketing script, there are several strategies I could adopt:
1. Meter with an evaluative (image wide) base exposure (0Ev) and let the script take as many brackets +ve or -ve as it needs to meet my criteria;
2. Spot meter for the darkest shadow were I wish to see 'image data' and, once again, let the script run;
3. Spot meter for the highlights were I wish to see 'image data' and, once again, let the script run.

I must say, I tend to use strategy 2, and then my script creates one exposure brighter, as I call this function first and it must take at least one image to assess the histogram. I then move to the function that addresses highlights and it takes as many of these ETTR-like images as it needs, ie it takes images until I have an ETTR image that meets my criterion, ie is the lapser-count in my sub-zone (usually a 1/3 of a stop from the top) less than my count criterion.

I believe the auto-bracketing strategy gives us ETTR and it is scene-independent. The 'over head' being I will take more images than I need, if I had a holy grail, single-image ETTR shooting mode. But, as I say, with auto bracketing and cheap SD cards, is this a problem. Well it could be if I wanted to shoot fast and freeze a scene, ie one image. But, for me, bracketing serves me well, and motion blur can be artistic!

BTW my thoughts o this subject will likely take a few posts, so I wont complete my thinking here. Also I always reserve the right to change my mind if a better idea comes up!

However, one thought I have had is this. Is there a way to get CHDK to take an image without recording it to the SD card, look at the histogram and based on this take a decision to record the image or adjust the image exposure and take another non-recorded image to evaluate that histogram? Just a thought?

Bottom line: I believe any reasonable auto-bracketing script, IMHO like mine, is a form of ETTR; it just comes with an overhead of needing more memory and shooting time. But it has the 'advantage' in that you don't need to worry about the scene-specific algorithmics. In other words, I worry about these in post-processing. BTW my post-processing for brackets these days is to create a 32-bit file in LR, using Merge to 32bit HDR, and process the image in a 32-bit LR space.

Cheers

Garry

Logged

lapser

1093

Re: Shot Histogram Request

« Reply #84 on: 23 / January / 2013, 16:44:48 »

Actually, I've already gotten to the time lapse holy grail with my get_shot_meter() function. That's how I made all these time lapses with the sun going down. Instead of the histogram, it uses the shot meter, which returns an exposure compensation change so the average of the pixels values in the next shot will be 2 f/stops below the 50% value of 2048. By returning the change in Apex96 units instead of linear, it's useful for setting exposure on the next shot.

With 4 shot meters available, I can set each one at a different spot in the picture, and use an algorithm to choose the meter. For testing, I've been setting a meter for each quadrant and picking the brightest meter. But the lua script can do whatever it wants. It probably will give better results for time lapse to use the average meter value, rather than go for ettr, especially with the sun in the shot at the beginning. But the script will have all the meters and ettr, ettl values available to it. It should give a lot of flexibility for setting exposure.

get_ettr(x) would do the same thing as get_shot_meter(), but instead of using the average of the pixel values, it would find the exposure change to get to the point where x/1000 of the pixels would be above white level. Then, you would subtract Apex96 units from there. By starting at that point, and subtracting 32 for each subsequent picture, you would bracket the ETTR value. It's just a faster way of doing what you're doing by starting closer to the target ETTR value. The magic of working with Apex96 log values is that it's just simple addition to change exposure however you want.

One feature I've included is the ability to take more than one shot per interval. You specifiy that with:

set_shot_interval(10000,3), for example. Every 10 seconds, this would take 3 quick shots (in continuous mode). The idea is to get the correct exposure after a long delay.

I've asked several times if one of the hackers could figure out a way to throw away the shot without creating and saving the jpg. It takes about 250 msec to get to the raw buffer, so throwing it away give you a really quick meter, probably quicker and more accurate than the pre-shot meter in the camera. I can see future cameras doing all their final metering this way, maybe including an HDR bracketing option using the ettr,ettl values.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

waterwingz

12138

Re: Shot Histogram Request

« Reply #85 on: 26 / January / 2013, 12:30:23 »

Quote from: lapser on 31 / December / 2012, 22:19:24

Quote from: waterwingz on 31 / December / 2012, 14:46:19
Did you ever pull together anything on your histogram metering & continuous shooting sync / timing stuff ?
I did get a little side tracked on the key function changes. I finally found that all day bug, and have it working. I'll make it the top priority to get a new diff file out, and an understandable demo Lua script. I'll post it here as soon as possible. Thanks for your interest, and your help.

@lapser : Its been a couple of week, time for a "bump" on this one? Unless I missed it in one of your (many) other threads

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Shot Histogram Request

« Reply #86 on: 26 / January / 2013, 13:51:43 »

Quote from: waterwingz on 26 / January / 2013, 12:30:23

@lapser : Its been a couple of week, time for a "bump" on this one? Unless I missed it in one of your (many) other threads

Point taken! It would be easier for me to compile a version for your camera and tidy up the documentation for the functions that are working so far so you can start playing with them. Would that be acceptable?

I've been really stuck on the timing thing at high shot rates in continuous mode. I figured out that writing a log file at shot rates faster than about 1.5 per second creates long delays. I can see the print(..) statements falling behind and then catching up (with print_screen), but there are no delays without print_screen.

I thought I was going to have to add auto exposure adjust functions to get the timing right, which turns out to be pretty complicated. But now I know that Lua can keep up as long as there aren't any writes to the SD card.

For logging, I think I can write a buffered Lua bwrite(string) function that saves everything in a string array and writes it out at the end.

Anyway, let me know your camera (or cameras) model and firmware revision, and I'll post something for you today.

====== EDIT =========
I've updated and attached the documentation for the time lapse modifications as they exist at the current moment. I'm ready to post a build for you as soon as I know your camera model/firmware.

The build also includes the Apex96 exposure functions discussed here:

http://chdk.setepontos.com/index.php?topic=9335.msg96139#msg96139

I think I'll need to add a scale factor for ISO conversions for accuracy. I need to test that.

1000 is the default scale for shutter speed, which means msec accuracy. reyalp likes usec (microseconds) with no scale option. The shooting_set_shutter_speed_ubasic function uses 100,000 as the scale.

1000 is also the default scale for av96 conversions.

« Last Edit: 26 / January / 2013, 17:49:51 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

waterwingz

12138

Re: Shot Histogram Request

« Reply #87 on: 26 / January / 2013, 15:19:36 »

Quote from: lapser on 26 / January / 2013, 13:51:43

Anyway, let me know your camera (or cameras) model and firmware revision, and I'll post something for you today.

Thanks, but I tend to rebuild daily (to test my changes and those posted by others) so having a custom build is not a lot of use. A patch file would help - even if it won't apply immediately I can probaly tweak the parts I need for the current build and then just carry those changes forward myself.

Quote

I've updated and attached the documentation for the time lapse modifications as they exist at the current moment.

Thanks again but without a patch file to work with having documentation for functions but no source code doesn't do me a lot of good.

Rather than ask you to go to a lot of additional work, I'll go back to my other projects. When you get ready to release something I'll see it in the forum and give it a try.

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Shot Histogram Request

« Reply #88 on: 26 / January / 2013, 15:53:45 »

Quote from: waterwingz on 26 / January / 2013, 15:19:36

Thanks, but I tend to rebuild daily (to test my changes and those posted by others) so having a custom build is not a lot of use. A patch file would help - even if it won't apply immediately I can probaly tweak the parts I need for the current build and then just carry those changes forward myself.

OK, when I have a more stable patch file ready, I'll post it and notify you. Thanks for offering to test it.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Time Lapse Mods Diff File

« Reply #89 on: 26 / January / 2013, 18:05:27 »

OK, here's the current diff file. I'm experimenting with the shoot() bug, so you can probably just delete the action stack portion if you want to.

It's still preliminary, but it seems to work on my cameras.

I also updated the directions a little, and attached the latest documentation file.

Thanks to everyone who made CHDK possible, and especially to reyalp, philimoz and waterwingz for all the help that got me this far.

Lapser_TimeLapseMods.txt (7.02 kB - downloaded 34 times.)

LapserMods_1.2_T2490.diff (51.75 kB - downloaded 31 times.)

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos