checking exposure - best method? - Script Writing

mattkime

299

checking exposure - best method?

« on: 16 / April / 2009, 11:23:04 »

Advertisements

i'm creating a function which checks the brightness value (bv) to be used in calculating the next exposure. i can press and release shoot_half and then do get_bv() but i'm sure a number of other values are affected by it. does anyone know which camera values are altered? or a more efficient way of getting the current bv?

thanks!

Logged

fudgey

1705
a570is

Re: checking exposure - best method?

« Reply #1 on: 26 / April / 2009, 16:20:13 »

Umm...

If you intend to shoot immediately after determining Bv by half_shooting, there's no point in releasing shoot_half after autoexposure since it takes time and stresses the mechanics. You really should just override the way you wish and then shoot_full. If you don't want to do that for some reason, you probably will have to do with Bv from a previous frame after a shot or use CHDK's RAW shot histogram (which also slows things down and is also from the previous shot, but has the added bonus of being usable in dark whereas normal bv from Canon's autoexposure can only work well when the live view image during half press is somewhat usable).

AFAIK, half shooting (likely need to be in a mode other than truly manual M mode, but I haven't checked) is the only way to make the cam determine Bv. During the half shoot process, various of things happen with propcases, and they depend on camera modes (automatic/manual everything/something), but that's just the camera's Tv/Av/Sv decision based on the Bv it just metered and something you are going to override. In theory you could probably calculate Bv yourself from the live view image or maybe even the live histogram (requires new C code of course), but then you must take care to put the camera in a mode where live view image actually represents the coming image). Basically that means full manual mode, which is not available in all/most cameras.

Btw, I've been playing around with an idea to implement a fast + accurate continuous mode intervalometer with autoexposure by speeding up the shot histogram code (maybe by making the number of points user selectable), using it to adjust exposure during continuous mode burst and minimizing phase noise by adding a user configurable delay to somewhere around where RAW saving happens.

This would probably result in reliable intervals even under 2 seconds (assuming short enough Tv) and will put less strain on camera mechanics than a normal intervalometer that uses Canon's autoexposure for each and every shot. Anyway, the algorithm to derive bv from the shot histogram (or the raw image itself) is kind of missing. The sunset timelapse script has something that clearly works for many uses, though.

Logged

mattkime

299

Re: checking exposure - best method?

« Reply #2 on: 26 / April / 2009, 18:12:09 »

I previously thought that the shoot_half -> shoot_full method of shooting produced stability problems but i'll give it another test. it certainly would make more sense to calculate the exposure between shoot_half and shoot_full

>>Anyway, the algorithm to derive bv from the shot histogram (or the raw image itself) is kind of missing. The sunset timelapse script has something that clearly works for many uses, though.

I've found from my camera setup (a canon a650) that exposures are fairly reliable into the dead of night. i currently can't see a reason to use the histogram. i'm going to do an evening of exposures to judge how reliable the light meter is. i'll post a graph shortly.

Logged

reyalp

14080

Re: checking exposure - best method?

« Reply #3 on: 26 / April / 2009, 18:38:08 »

Quote from: fudgey on 26 / April / 2009, 16:20:13

Umm...
If you intend to shoot immediately after determining Bv by half_shooting, there's no point in releasing shoot_half after autoexposure since it takes time and stresses the mechanics. You really should just override the way you wish and then shoot_full.

FWIW, I had trouble overriding Sv during the half shoot on the SD990, no matter which combination of propcases I tried. Some of this may have been confusion due to the bug of not actually going over 800.

You also can't (AFAIK) override the ND filter once Bv is available (from script... if you could call the actual PutInNDFilter function you'd be fine).

There really should be some better documentation of what you can use when. Most of the shooting related set_* functions only work before a halfshoot, because they just set the put_off value in shooting.c, and then the override is applied once some time after half press in shoot_seq_task. Some of the propcases can be set any time, while others only take effect if you update them at some point in shoot_half.

Quote

AFAIK, half shooting (likely need to be in a mode other than truly manual M mode, but I haven't checked) is the only way to make the cam determine Bv.

Bv appears to be accurate in manual mode to me. The cameras I've used still calculate under/over exposure even if you are shooting manual.

Quote

In theory you could probably calculate Bv yourself from the live view image or maybe even the live histogram (requires new C code of course), but then you must take care to put the camera in a mode where live view image actually represents the coming image). Basically that means full manual mode, which is not available in all/most cameras.

This would be very difficult.

I would like to add code get the live histogram data from the last (or current, if you are half shooting) live histogram in script. This should be pretty straightforward, and would give you a lot more to work with than just a single Bv value.

Quote

This would probably result in reliable intervals even under 2 seconds (assuming short enough Tv) and will put less strain on camera mechanics than a normal intervalometer that uses Canon's autoexposure for each and every shot. Anyway, the algorithm to derive bv from the shot histogram (or the raw image itself) is kind of missing.

This should be something you can get a reasonable approximation of from the APEX exposure equation and experiment.
Bv = Sv - (Tv + Av) for a "correct" exposure. You know all the stuff on the right. So you need some function F(pixel value) that maps the pixel value into stops of under/over exposure. Then F(average pixel value) + Sv - Tv - Av should be your scene Bv. You should be able approximate F, either as a formula or a lookup table, by shooting scenes of a known Bv at various exposures.

edit:
When getting Bv in a half press, you need to wait until get_shooting returns true. (actually it updates slightly before, but the only way to detect that is to see if it changed from the previous value)

« Last Edit: 26 / April / 2009, 18:40:14 by reyalp »

Logged

Don't forget what the H stands for.

fudgey

1705
a570is

Re: checking exposure - best method?

« Reply #4 on: 26 / April / 2009, 19:10:17 »

Quote from: reyalp on 26 / April / 2009, 18:38:08

FWIW, I had trouble overriding Sv during the half shoot on the SD990, no matter which combination of propcases I tried. Some of this may have been confusion due to the bug of not actually going over 800.

You also can't (AFAIK) override the ND filter once Bv is available (from script... if you could call the actual PutInNDFilter function you'd be fine).

Bug? I don't know about the new ones that may have more than ISO1600 pre-set, but for AFAIK for all others ISO800 is the maximum amplification, above that it's all software for JPEG only. So it's not surprising that 800 would be a point where behavior changes...

I don't have a cam with ND filter so can't really investigate that one.

Quote from: reyalp on 26 / April / 2009, 18:38:08

There really should be some better documentation of what you can use when. Most of the shooting related set_* functions only work before a halfshoot, because they just set the put_off value in shooting.c, and then the override is applied once some time after half press in shoot_seq_task. Some of the propcases can be set any time, while others only take effect if you update them at some point in shoot_half.

I wrote something a while back here: http://chdk.setepontos.com/index.php/topic,3455.msg31907.html#msg31907. Basically any propcase can be set at any time, but in some shooting modes half shoot overwrites some of them so writing to them is sometimes pointless.

When Tv or Av or both are automatically set by the camera, the "user" values have no function (they are not updated by the camera and writing to them is no use).

When in a manual mode or semi-manual mode, the "user" Av and Tv propcases change immediately when you set a new Av (in Av or M mode) or Tv (in Tv or M mode) by pressing the camera's buttons. After you half shoot and wait for exposure to finish (even in M mode the camera does some tricks here), the "real" or "coming" values change.

If you half-press the shutter and wait for exposure to finish and then write to the "coming" Av or Tv propcases, your changes will be effective for the next shot unless you release the shutter.

If you write to the "user" Av, Tv or Sv propcases, nothing will happen until you half shoot and wait for exposure. When you do that, your setting will be transferred to the "coming" Av/Tv/Sv propcases. Tv seems to be copied without a change on my a570is. In case of Av, the value is not copied, because most of the camera's "market" values shown on the LCD are not the actual exact aperture values it uses. For ISO, the "user" ISO mode is transformed to the "market" and "real" Sv values. As with Av, the "real" value corresponds to the actual ISO sensitivity the photo is taken with, while the "market" value corresponds to what was shown on the LCD and/or selected from the menu.

In AUTO ISO mode the "market" ISO propcase is always 480 (ISO 100) on my a570is.

Propcase 24 and 25 seem to be identical copies of each other. They contain the widest available aperture for the current camera settings (zoom). It updates after you half shutter and wait for exposure.

Quote from: reyalp on 26 / April / 2009, 18:38:08

I would like to add code get the live histogram data from the last (or current, if you are half shooting) live histogram in script. This should be pretty straightforward, and would give you a lot more to work with than just a single Bv value.

In a script a single value is usually (until it fails to work well) what I'd want, for simplicity. But access to the live histogram could be useful sometimes, of course. Especially if it could be determined during continuous mode shooting.

Quote from: reyalp on 26 / April / 2009, 18:38:08

This should be something you can get a reasonable approximation of from the APEX exposure equation and experiment.
Bv = Sv - (Tv + Av) for a "correct" exposure. You know all the stuff on the right. So you need some function F(pixel value) that maps the pixel value into stops of under/over exposure. Then F(average pixel value) + Sv - Tv - Av should be your scene Bv. You should be able approximate F, either as a formula or a lookup table, by shooting scenes of a known Bv at various exposures.

Indeed... but this smells like work, well worth avoiding

Quote from: reyalp on 26 / April / 2009, 18:38:08

When getting Bv in a half press, you need to wait until get_shooting returns true. (actually it updates slightly before, but the only way to detect that is to see if it changed from the previous value)

Yes, of course. And with waiting for get_shooting comes great responsibility of not using anything that's incompatible with it. Well, AE lock doesn't apply here I suppose, just can't remember what else conflicted with it.

Logged

fbonomi

469
A570IS SD1100/Ixus80

Re: checking exposure - best method?

« Reply #5 on: 27 / April / 2009, 02:51:21 »

Determining an exposure from an histogram is tricky indeed. I have tried to find online some explanation of such an aglorithm, with no success.

In order to have a significant sample of data to check while developing the sunset script I did the following:

- shoot one shot letting the camera decide the exposure
- shoot a series of shots of the same scene with a wide variation of exposures (say, from -6 to +6 EV)
- compare the histograms of the resulting JPEG files and try to interpret how the camera would select the "correct" exposure
- repeat for a number of different situations

My conclusions:

1) (WILD GUESS) There must be several assumptions made by the camera about the scene. These assumptions change the way the camera exposes, i.e. change the way the camera selects values from the histogram to determine "correct" exposure (for example, the relative weight of median, average, over-exposed area and under-exposed area). So, what I thought would be the same process repeated over different scenes was in effects a series of different processes.

If you think about it, that's quite obvious... The camera after all has different modes for beach / landscape / snow / fireworks etc.... In "normal" mode, tha camera would probably try and guess what "scene" is being shot at...

2) I probably did a basic mistake working on the JPEG files: this puts you "downstream" after too much processing to have a really significant data. Changing this would have required quite a rewrite of the PHP scripts I had written to analize the histograms, but due to point 1) above I didn't think it would be worth while.

3) again, wild guess... there is no reliable (easy) way to get a histogram, calculate its values, derive its "delta" (i.e., how much it's under/over exposed), apply this delta and shoot correctly, FOR LARGE VALUES OF DELTA. Maybe it's just a matter of linearization / gamma etc, but the data become very erratical. After all, the little indicator in the camera display that tells you if you are under/over exposing won't give you any value outside the +/- 2EV range: it just gets red, and hides the computed value.

4) Especially talking of night shots, thinking about a "correct" exposure is misleading... If really I could emulate perfectly the camera's exposure algorithm, then I would take a photo in moonlight and have a photo that is illuminated in a way that is very similar to a day photo. This might be "correct", but is it "desirable"? I want the scene to look different, after all.

All this considered, that's why I came out with my kludgy system for determining exposure..

Logged

fbonomi

469
A570IS SD1100/Ixus80

Re: checking exposure - best method?

« Reply #6 on: 27 / April / 2009, 02:56:59 »

@fudgey:
yuor analysis of the properties' working is quite convincing. Reliably setting the properties has always been a pain for me. Often, a delay in the script helps too.

As for the ND filter, (which I studied for the SD1100 used in the balloon project), I haven't been able at all to set it via script via propcases or functions. I resorted to over-riding it via the CHDK menu, and calculate exposures consequently.

Logged

fudgey

1705
a570is

Re: checking exposure - best method?

« Reply #7 on: 27 / April / 2009, 14:05:14 »

Yes, determining exposure is clearly a complex task... and of course a matter of opinion in any scene whose dynamic range doesn't completely fits the sensor dynamic range (or jpeg dynamic range or LCD live view image dynamic range if that is to be used as a histogram source) or one that is in significant motion compared to exposure time.

For continuous mode autoexposure there's one clue that can be used, i.e. Canon's chosen exposure for the first shot. That shot can be assumed to be well exposed, after all, unless the user screwed up. If the scene changes a lot after the beginning of the burst (like when driving around in town with the camera), relying on that initial value a lot could lead to poor results, though.

Logged

mattkime

299

Re: checking exposure - best method?

« Reply #8 on: 27 / April / 2009, 15:58:14 »

results from mapping time and exposure (bv96 values) from my a650. ran overnight

below -600 it looks like some noise creeps in but looks like its only half a stop.

full res - http://img82.imageshack.us/img82/9451/picture2eur.png

Logged

reyalp

14080

Re: checking exposure - best method?

« Reply #9 on: 27 / April / 2009, 23:43:14 »

Quote from: fbonomi on 27 / April / 2009, 02:56:59

As for the ND filter, (which I studied for the SD1100 used in the balloon project), I haven't been able at all to set it via script via propcases or functions. I resorted to over-riding it via the CHDK menu, and calculate exposures consequently.

It should work if you call set_nd_filter(1) before pressing halfshoot. It will only be in effect for one halfshoot or full shoot.

On the SD990 (and probably other ixus with manual ND setting), it can be set via the user Av propcase as well (I think, I only discovered this toward the end of my testing before sending the camera back).

Logged

Don't forget what the H stands for.