EV (exposure) and RAWs experiment

jmac698

138

« on: 13 / October / 2023, 12:29:04 »

Advertisements

Hello,
I'm experimenting with shooting RAWs of a known light-source and predicting the resulting RAW pixel values.
In theory, there is the traditional photographic exposure system where ISO, Aperture, and shutter speed are interrelated. At ISO 100, Tv (shutter time)=1s and Av (aperture) of f/1, you get 0 EV. You can then exchange units called "stops" in each of the settings to balance or change the EV. A change of 1 stop in Av is in a ratio of sqrt(2), so 4, 5.6, and 8 would be 3 stops (note, it should be 5.7 but the values are rounded in some arbitrary way).
For ISO, it's every doubling; ISO 100, ISO 200, ISO 400 etc. Same with Tv. The reason that Av is different is because it's 2d; if the area of the stop changes by 2, that means the radius must be root 2, since area is r^2.
To test this theory, I took pictures of an even white light in all the apertures, then averaged a 256x256 square in the middle of the RAW. The light source had a luminance of 260.5 cd/m2 or nits and a colour temperature of ~65k (CCT=6529k). I set a custom auto white balance, then took multiple frames at each Av, eliminated outliers and averaged the rest. The following values should be accurate +-1, 99% (3 standard deviations) of the time:
Av DN
3.4 2174.7
4 1739.4
4.5 1412.5
5 1159.7
5.6 919.9
6.3 755.2
7.1 699.5
8 636.3
Av is in units of f stop, which is the ratio of FL (focal length) to aperture, and DN means Data Number, or just the RAW pixel value.So here's the problem. The pixel values don't relate in any way to the predicted ratio of the apertures. For example, from f/4 to f/8 should be exactly 4, but it's only 2.7. Does anyone know where I'm going wrong in my calculations?I should note, you need to subtract 128 from each value before finding the ratio, as this is a constant offset known as black level. It's not exactly 128, but for this ISO and ambient temperature, it's pretty close.

« Last Edit: 13 / October / 2023, 12:37:02 by jmac698 »

Logged

reyalp

14082

Re: EV (exposure) and RAWs experiment

« Reply #1 on: 13 / October / 2023, 22:49:05 »

Quote from: jmac698 on 13 / October / 2023, 12:29:04

Hello,
I'm experimenting with shooting RAWs of a known light-source and predicting the resulting RAW pixel values.
In theory, there is the traditional photographic exposure system where ISO, Aperture, and shutter speed are interrelated. At ISO 100, Tv (shutter time)=1s and Av (aperture) of f/1, you get 0 EV. You can then exchange units called "stops" in each of the settings to balance or change the EV. A change of 1 stop in Av is in a ratio of sqrt(2), so 4, 5.6, and 8 would be 3 stops (note, it should be 5.7 but the values are rounded in some arbitrary way).

Or more formally, APEX http://dougkerr.net/Pumpkin/#APEX
Note the Canon firmware (and CHDK) generally works in APEX*96 units, which allows many convenient fractions of a stop to be represented as integers. Also, as noted below, values displayed in the Canon UI may differ from the values used in exposure calculation significantly beyond the typical rounding.

Quote

For example, from f/4 to f/8 should be exactly 4, but it's only 2.7. Does anyone know where I'm going wrong in my
calculations?

One thing I notice is the "stop" from f/8 to f/5.6 value is really far off, but others are much closer.
That is (after subtracting 128 black level), f/5.6 = 791.9, f/4 = 1611.4 / 2 => 805.7, which is within 2% of the expected value. (edit: black level should be 127 above)
It's possible that what's shown as f/8 in the UI on your camera (which model are you testing?) is not actually very close to f/8. For example, on my SX730, setting "f/8" in the UI results in PROPCASE_AV = APEX*96 561 => APEX 5.84 => f/7.57.

If you use the APEX*96 values from the procases to identify the "stops", rather than the UI values, that should be a better reflection of the actual values used.

Note that rawop https://chdk.fandom.com/wiki/Lua/Raw_Hook_Operations allows you do this kind of thing with script on the camera.

In my experience, values from the rawop.raw_to_ev function (source here: https://app.assembla.com/spaces/chdk/subversion/source/HEAD/trunk/modules/rawhookops.c) reflect exposure propcase changes very well (that is, changing an exposure propcase by one "stop" results in a one stop change in the "ev" value). Note that the "ev" values are relative to an arbitrary, experimentally derived "neutral" value that very roughly corresponds to what Canon autoexposure would use on a uniform grey scene.

The script CHDK/SCRIPTS/TESTS/drtest.lua is an example.

A script I wrote to test aperture can be found in this thread https://chdk.setepontos.com/index.php?topic=13532.0

« Last Edit: 29 / October / 2023, 00:39:28 by reyalp »

Logged

Don't forget what the H stands for.

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #2 on: 14 / October / 2023, 02:06:13 »

Very helpful, thanks!My camera is sx50hs fw:100b with propset 5.I used the debug display on page 3 and the location 23 didn't change as I changed Av in manual mode. I found that location 26 changed in an expected way. They showed me a perfect set of 1/3 stops. I checked another run of my pictures and got slightly lower values, and the ratio of f/5.6 to f/8 was 2.02 in that case, which is great. I'll have to try your script later. Here are my results so far:

Av Disp	Av Ideal	APEX96	Ev derived
8.0	8.00	576	6.00
7.1	7.13	544	5.67
6.3	6.35	512	5.33
5.6	5.66	480	5.00
5.0	5.04	448	4.67
4.5	4.49	416	4.33
4.0	4.00	384	4
3.4	3.67	352	3.67

Logged

reyalp

14082

Re: EV (exposure) and RAWs experiment

« Reply #3 on: 14 / October / 2023, 12:26:08 »

Quote from: jmac698 on 14 / October / 2023, 02:06:13

Very helpful, thanks!My camera is sx50hs fw:100b with propset 5.I used the debug display on page 3 and the location 23 didn't change as I changed Av in manual mode.

23 should be the actual value, which only updates on half press
26 is USER_AV, which generally reflects the value entered in the UI.

Logged

Don't forget what the H stands for.

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #4 on: 14 / October / 2023, 13:06:44 »

Awesome, I will try that. I did think to try that at first, but I will try again. Also I'm trying your script but, error in 323, attempt to compare NIL values

Code: [Select]

if av96_end <= av96_start thenI'll try to work around it.

Logged

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #5 on: 14 / October / 2023, 13:16:04 »

ok, according to the documentation, https://chdk.fandom.com/wiki/Script_commands#get_min_av96 "nil (-1 in ubasic) if in playback" so that means I need to be in REC mode. Now it's working and collecting data.

Logged

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #6 on: 14 / October / 2023, 13:48:35 »

Your methods are working. I'll have to try my way as well. The sensor is so hot, that black level is significantly affected though. How does this log compare to your experiments?

avtest.csv (13.87 kB - downloaded 4 times.)

Logged

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #7 on: 14 / October / 2023, 14:54:29 »

Here's my results, based on your script and my analysis:

Av	Meter	Ratio	Predicted	Rel. Error %
3.56	1601.7		1.12
4.00	1360.6	1.18	1.12	4.9
4.49	1020.8	1.33	1.12	18.7
5.04	853.9	1.20	1.12	6.5
5.65	720.6	1.18	1.12	5.6
6.35	610.5	1.18	1.12	5.2
7.12	490.8	1.24	1.12	10.8
8.00	419.8	1.17	1.12	4.2
8.98	355.4	1.18	1.12	5.2
10.07	309.7	1.15	1.12	2.2
11.31	274.2	1.13	1.12	0.6

I averaged all the meter values for each Av, and relative error is (actual-expected)/expected * 100.I assume your script subtracts black level, but is that a defined constant in each build? The predicted ratio is 2^(1/(2*96/32)), so 96/32 means 1/3 stop, and 2 is the ratio of 1 stop, so this is 2^1/6.
But meter(f/4)/meter(f/8) is still way off by my way of looking at it, 3.24 vs ideal 4.0

« Last Edit: 14 / October / 2023, 15:05:36 by jmac698 »

Logged

jmac698

138

Re: EV (exposure) and RAWs experiment

« Reply #8 on: 14 / October / 2023, 16:49:25 »

I checked location 23, I had to take a picture before it would update. All the values are APEX96e 16 below what the GUI specifies, except the widest aperture which the GUI calls 3.4, but should be 3.56, and is actually narrow and darker at 3.50. Other than that, all the rest of the ratios remain the same, and still follow the ratio of 1.12.

So, the actual Av differences can't explain the poor ratio tracking I'm seeing. I also turned off the light for some pictures, and the meter gets 127 for those. If I subtract that from the other meter values, the error is just worse. It doesn't make sense. I need this for a special auto exposure script, so I can just accept the data as it is and use a table.

Logged

reyalp

14082

Re: EV (exposure) and RAWs experiment

« Reply #9 on: 29 / October / 2023, 00:37:44 »

Quote from: jmac698 on 14 / October / 2023, 13:48:35

Your methods are working. I'll have to try my way as well. The sensor is so hot, that black level is significantly affected though. How does this log compare to your experiments?

@jmac698
Sorry for the slow response. I haven't actually looked at that script much since that thread in 2018, and most of the tests I did weren't directly comparable.

Anyway, I did some more tests and while I'm not entirely sure it will answer your questions, it maybe you'll find something useful.

Quote

I assume your script subtracts black level

Not for the "meter" value. This likely is the biggest confounding factor in your numbers above. It is simply the raw sensor value, so would need 127 subtracted (I mistakenly said 128 in an earlier post, but most 12 bit cams including yours and the ones I tested are hard coded to 127). The blacklevel used is recorded in the "desc" column.

The meter96 and m_96* values do have the black level removed in the conversion to APEX*96. I almost never look at the raw values with these kinds of scripts, because the meter96 is so much more convenient to relate to exposure. Of course, this assumes the conversion is valid, but experience suggests it's good enough.

With blacklevel subtracted, I get 4.2 for meter(f/4)/meter(f/8) in your data, or about 5% error, which doesn't seem unreasonable.

Quote

The predicted ratio is 2^(1/(2*96/32)), so 96/32 means 1/3 stop, and 2 is the ratio of 1 stop, so this is 2^1/6.

Maybe I'm misunderstanding or missing something, but if you mean the ratio between (raw - blacklevel) counts at 1/3 stop increments, shouldn't it just be 2^1/3 (= ~1.25992)?

Looking at your log, there's quite a lot of shot to shot variation. The "WARN: high error" message is triggered when the measured value varies more than 6 APEX*96 units (1/16th of a stop) from the predicted value. The threshold is arbitrary, but I believed I picked it because it wasn't common in the runs I did while developing the script.

I've attached a zip with .ods spreadsheets of a few of my runs, as well as yours. You should be able to load them in google docs if you don't use LibreOffice or similar.

The rows in the 'calcs' tab average the values for each ten shot step, as well as various ways I tried comparing them. IMO the most relevant are:
meter - bl and m_pred - bl are the average raw meter value and predicted meter value with black level subtracted
prev_frac is the ratio between the meter value, and the meter value of the preceding step
"1 stop frac" is similar to prev_frac, but for next full stop
m96_err is the average error (difference between the expected and actual meter96 value). Note for the first step, this just reflects variation in the baseline shots

In the three runs form my cameras, 2 are from sx710 and one is sx730. I think the sx710-avtest-20231022-3-calcs.ods run had some change in lighting after the baseline shot, but I didn't really run enough test to confirm that
These were done under artificial (compact florescent) lighting in a room at night with blinds drawn, facing a white wall.

Yours has a bit more variation, but it seems broadly similar.

It's not obvious to me how much of the error is lighting variation, variation in how close the hardware gets to ideal values, or other factors.

NOTE, my runs were done with a newer version of the script (also included), and 10 shots for the baseline. I fudged the one from your CSV (avtest-jmac698-20231014-calcs.ods) by adding the missing columns (av96_cur2 is simply duplicated from av96_cur, m_pred reverses the conversion of m96_pred)

Some background on how the script works:
It takes the number of shots specified by "Baseline shots" and averages those to get the expected raw value for the current exposure setting. This also acts as the first step in the test.

The exposure is measured by averaging a 1200 pixel square in the center of the sensor, with 5 pixel steps in x and y. The step being odd means all the colors in the CFA are sampled.

For each step after the baseline, it changes the aperture and takes "Shots per step" shots. The m96_err column represents the difference (in APEX*96 units) between the actual and expected exposure.

Since the baseline is measured once at the start, any subsequent changes in lighting will affect all the following shots. It might make more sense to measure the relative difference between each step, or to do both comparisons.

reyalp-jmac698-avtest-20231028.zip (69.1 kB - downloaded 5 times.)

Logged

Don't forget what the H stands for.