Measuring exposure performance (was Re: proposal - script shooting hooks)

c_joerg

This is an isoinc run from SX230. Meter96 looks more noisy as from S110 an G1x. Well, this cam has a smaller Sensorâ€¦

isoinc_SX230.csv

24.77KB

c_joerg

I would like to compare my cameras under identical conditions . But it's hard because the exposure time is constantly changing . It would certainly be helpful if you had a parameter for the exposure time .
Or any other idea?

Can I do something like set_tv96_direct(0) for 1s in the code?

reyalp

c_joerg
I would like to compare my cameras under identical conditions . But it's hard because the exposure time is constantly changing . It would certainly be helpful if you had a parameter for the exposure time .
Or any other idea?

Can I do something like set_tv96_direct(0) for 1s in the code?

That's a good idea. For now, you should be able to set the exposure you want by replacing

	local tv96 = get_prop(props.TV) + d_sv96 - start_ev_shift*96

with your own tv96 value. For one second, use 0.

c_joerg

This is a nearly identical isoinc run with G1x, S110 and SX230 with 1s exposure time in range ISO100 to ISO400. For me it looks like there is not really a difference between the cams. In the second plot I tried to normalize the curves.

c_joerg

Just for completenessâ€¦
May be a run around ISO1600 shows more differencesâ€¦

isoinc1s.zip

21.18KB

c_joerg

I don't think there should be oscillation in rawopint in constant lighting

I would also expect so on constant lighting.

if over or under protection isn't active, unless shutter speed is high enough to run into precision issues. However, I haven't actually tested like that, so maybe there is.

The second 100 pictures from rawopint_compare3, the part without modification show oscillation on constant lighting. Under protection was off, over not active and shutter speed around 1/4s. I see this in a couple of runs

The question is who constant the lighting was. I did it in a room without windows. The same environment as I used for all isoinc runs. The room has one filament lamp (200W / 50Hz) with non-electronically stuff inside. Iâ€™m not sure if a monitor would be better measure equipment.

Another note, in theory we can set the shutter speed directly in microseconds. The actual precision of the hardware is unknown (a shutter testing script like isoinc would be useful), but it should be possible to adjust exposure in steps smaller than 1/96th ev for some of the range. This isn't currently possible through script though.

Interesting stuff.
I've always asked myself, how the implementation of tv96 to exposure time works.
On ISO I expect amplifiers as we discuss before.
On exposure time I would expect a digital timer. But when the lowest resolution is microseconds than the timer must be larger than 16Bit. So when you can modify the timer, you can play with the lowest Bit. Do you know where the highest shutter speed is?

Update:
To run some constant measures with isoinc, I could set iso_start= iso_end=412 and shots_per_step=100 or?
To get more accurate values, it would be helpful to set meter_step=1?

rawopint_compare3.csv

36.68KB

reyalp

c_joerg
I've always asked myself, how the implementation of tv96 to exposure time works.

It follows the follows the APEX definition (how precisely isn't certain)

On exposure time I would expect a digital timer. But when the lowest resolution is microseconds than the timer must be larger than 16Bit. So when you can modify the timer, you can play with the lowest Bit. Do you know where the highest shutter speed is?

The exposure time is a 32 bit microsecond value, but I assume the flicker seen at high shutter speeds is related to physical limits. In the testing I've done, 1/16000 is noticeably darker than 1/8000, but I haven't got around to testing how close they are to the requested values or how consistent the results are. This should be relatively easy to now that we have raw meter.

At the other end, you can use a full 2000 seconds on some cameras, but quality suffers 😉

Update:
To run some constant measures with isoinc, I could set iso_start= iso_end=412 and shots_per_step=100 or?

I think so, though I don't promise the script correctly handles start=end

To get more accurate values, it would be helpful to set meter_step=1?

The default samples 57600 pixels, I wouldn't expect adding more to make much difference.

edit:
In my earlier comment, I said meter would tell you if the scene changes, but of course this isn't copletely correct, meter will change in the next frame if the requested exposure changes. Bv should should give you the absolute scene brightness, assuming there is no over or under exposure, and that the camera actually uses the requested exposure values.

In your compare3, the bv values show some slight variation.

c_joerg

if the scene changes, but of course this isn't copletely correct, meter will change in the next frame if the requested exposure changes.

Of course, I understand it like this, meter has to change when exposure changes

In your compare3, the bv values show some slight variation.

So I will do some investigation with different lighting situation including using a monitorâ€¦

c_joerg

I made 6 runs (each 100 shoots) under constant lighting condition with G1x. I thought it would be constant...

1) Filament lamp, indirect light, F4, 1/2s
2) Halogen lamp, indirect light, F4, 1/10s
3) LED lamp, indirect light, F4, 1/8s
4) LED lamp, indirect light, F4, 1/8s
5) Monitor, 60Hz, F4, 1/25s
6) Monitor, 60Hz, F16, 0.6s

Run 1) are the same condition as I made with all isoinc runs. There are changes from around 3/96 steps. Really worse is the halogen lamp. It looks like that the LED is better but she loses brightness over the time. It look like the monitor is the best solution for measure (when using longer shutter times). But also the monitor needs time, to get on the right temperature.

Notice: The senor temperature increases from 30 to 45 degree on the end, but it looks like that this has no influence.

The log file shows on sv96=412 ==> sv=101. I thought, sv96=412 is exact ISO100â€¦

isoinc_light.csv

70.72KB

exposeIT

Hi,
I follow your discussion and tests very interrested.

c_joerg
I thought it would be constant...

1) Filament lamp, indirect light, F4, 1/2s
2) Halogen lamp, indirect light, F4, 1/10s

For 50Hz AC the the light of your lamps have a frequency of 100Hz.
Every 10ms you will have a peak in brightness.
Noone knows really how quick the naked exposure-measurement of the camera is. (not exposure time)

So your exposure can differ depending to the position of the trigger.

A good idea to test this might be a 12V Halogen lamp with a good stabilized DC Power source.

reyalp

exposeIT
Noone knows really how quick the naked exposure-measurement of the camera is. (not exposure time)

In this case, the measurement is from the raw data, so it is exactly the exposure time.

It's not too surprising to get noticeable variation with a ~1/10th exposure time under an AC light source, catching one cycle more or less should be fairly significant.

c_joerg

For 50Hz AC the the light of your lamps have a frequency of 100Hz.Every 10ms you will have a peak in brightness.

It's not too surprising to get noticeable variation with a ~1/10th exposure time under an AC light source, catching one cycle more or less should be fairly significant.

I have not expected, to see something from AC on Filament or Halogen lamp. They are really sluggish. You can see the effect from 60Hz on the monitor at 1/25 and you can see also the effect from the monitor on live view. I think thatâ€™s more the not stabilized power supply from net. So I will do some more measure with Halogen lamp with different exposure times.

@reyalp
May be you should move this measure stuff to an own thread.

reyalp

c_joerg
I have not expected, to see something from AC on Filament or Halogen lamp.

Yes, this is a surprise to me too, I would expect them to have enough inertia that there wasn't much variation. What I mean was *if* a source flickers at ~50-100 hz would be expected to affect a 1/10th exposure.

If the source is actually the AC, just pointing at a darker subject and/or stopping down, adding ND to get a longer shutter time should smooth it out, but that's no good if you want to test very short exposures. In that case it seems like daylight is the best bet.

@reyalp
May be you should move this measure stuff to an own thread.

Good idea. Merged into the isoinc thread.

c_joerg

just pointing at a darker subject

All the measuring above, also with the monitor, I did with a gray card with a small x in the middle to focus.

In that case it seems like daylight is the best bet.

Yes of course.

But my intention is to find a light source, which is nearly reproducibly and nearly constant, to get accurate results on isoinc.

I made a couple of runs with the halogen 50Hz up to 4s. I got always the same result. Iâ€™m really sure; the variance in the runs comes from the not stabilized power supply from net. You can really see the shape of the curves, if you turn on a strong consumer. Results can be seen in isoinc_halogen_50Hz.csv.

Then I made 2 * 100 shoots with a 12V halogen. Yes, thatâ€™s looks really good. The first 100 shots I made with an old laboratory power supply (older 25 years). The second 100 with a newer switching Power Supply. Only on meter you can see a small drift over time on the laboratory power supply. So my next runs with isoinc I will always use the 12V halogen with the switching Power Supply. Results can be seen in isoinc_halogen_12V.csv. In the plot I removed the offset between both runs to get them in one plot.

It looks like that the monitor for higher exposure time is a good solution. One think, where I thought about is, do make a demonstration video (constant picture with changing brightness, or a simulated moon, which goes from one side of the monitor to the other side), which gives you reproducible results on rawopint, playing with the parameters. I know, for real testing off overexposure there is not enough dynamic. Iâ€™m not sure, if such a video is really helpful.

isoinc_halogen_50Hz.zip

11.72KB

exposeIT

reyalp
In this case, the measurement is from the raw data, so it is exactly the exposure time.

It's not too surprising to get noticeable variation with a ~1/10th exposure time under an AC light source, catching one cycle more or less should be fairly significant.

To clarify my view on this see the drawing below.

The 100 Hz sine wave is your scene light.

All internal camera measurements to determine the exposure data takes place at Half Press.
The measurement is presumably very short. We (I) don't know how long.
And we don't know at which position on the sine wave. (see the red squares)
But the brightness differs with the amplitude of the sine wave.

Anyway the camera will set all values according to its measurement.(aperture and exposure time ...)

The exposure time ( see green box) starts at Full Press collecting the light from the sine wave.
If the exposure time is long this will fairly integrate the light.
If the exposure time is short we will pick up significantly less light than given by the (wrong) measure.

So for 'exact' measurement we have two implications:

1) The measurement positiom on the sine wave period
AND
2) The exposure time

Maybe this has something to do with the flicker.

reyalp

exposeIT
All internal camera measurements to determine the exposure data takes place at Half Press.
The measurement is presumably very short.

You need to understand the actual script being used. The camera metering is used only for the initial exposure. After that, shooting is in continuous mode or with half press held down and never released, so camera metering is not a factor.

edit:
The script also has a "single" mode, in which case sv and tv are set explicitly by CHDK each shot. Cameras with an adjustable aperture should be set to a mode where it won't be changed.

c_joerg

@exposeIT

The 100 Hz sine wave is your scene light.

I donâ€™t think, there can be anything seen from 100Hz on a halogen lamp. The halogen lamp will integrate the light. May be there is some rest ripple on the top.

he measurement is presumably very short. We (I) don't know how long.
And we don't know at which position on the sine wave. (see the red squares)

I did 20 half press and I got always the same result from the camera measure. Ok I have only seen 1/3 EV steps on the display. The ripple where we talking about are less than 5/96 EV.

If the exposure time is short we will pick up significantly less light than given by the (wrong) measure.

I donâ€™t think so because of the high integrate light from halogen.
What happens, when you make 100 shots with 1/1000s against a halogen lamp? I donâ€™t think you will see significantly changes.
What is happens on video mode, when you make a video? I donâ€™t think a halogen AC lamp would make problems.

However, the DC solution would be the best way.

@reyalp

The script also has a "single" mode,

So I can make a comparable run on "single" and "quick" mode to see, how much the aperture flicker isâ€¦

c_joerg

Hello,

I made a couple of more runs (each run with 100 shoots) with isoinc with no increment (all shoots with ISO100), with my cameras G1x, S110 and SX2330. I normalized the meter and meter96 curves by subtraction of the median value of each 100 shoots., to see only the differences.

One thing that I was interested in how big the aperture flicker in single mode is. The most influence of aperture flicker you can see on G1x. I think this has to do with the absolute size of aperture. But itâ€™s only 1/96/EV and not happened in quick and continuous mode. So nothing to care about. On the SX230 I canâ€™t really measure it. This has probably to do with the 12bit resolution.

What you also can see is the influence of the sensor temperature. Especially on the first 100 shoots you see the influence. Also in meter96 not too much to take care about.

On S110 Unfortunately I run everything with Tv=1s (I used my isoinc with fixed Tv=1s). I'm not sure, which effect this have. I didnâ€™t look like clipping.

G1x
1) Quick, f=6.3, Tv=0.17s
2) Single, f=6.3, Tv=0.17s
3) Single, f=11, Tv=0.55s
4) Single, f=16, Tv=1s
5) Quick, f=16, Tv=1s
6) Quick, f=6.3, Tv=0.17s

S110
1) Quick, f=6.3, Tv=1s
2) Single, f=6.3, Tv=1s
3) Single, f=8.0, Tv=1s
4) Single, f=8.0, Tv=1s
5) Quick, f=8.0, Tv=1s
6) Quick, f=6.3, Tv=1s

c_joerg

SX230
1) Quick, f=6.3, Tv=0.16s
2) Single, f=6.3, Tv=0.16s
3) Single, f=8.0, Tv=0.25s
4) Quick, f=8.0, Tv=0.25s
5) Quick, f=6.3, Tv=0.16s

isoinc_ap.zip

30.06KB

c_joerg

I was just playing a little bit with my G1x, to understand a little bit more from the internal stuff, specially the relation between meter, meter96 and JPG data.

My knowledge before was, if I go with the exposure 1EV step down, this will half the RAW Data. So one Bit is approximately 1EV step. Yes I will nearly see this in the RAW Data.

What I have done with isoinc:
I set the G1x to manual mode with constant exposure time and constant ISO100 Then I made 12 runs with each 10 shoots on ISO100 by changing the initial EV shift from -8 to +3. As an object I used my gray card with constant light.

The first thing which I notice was that I got sometimes from the pre shot ISO200. Yes, we have seen it before and it looks like this is some stupid stuff from G1x. So I fixed that in the code from isoinc.

Yes, on the meter96 values it is really good seen, that I always go up on 1EV step.

There are a couple of thinks, which I donâ€™t understand jet

The G1x has a 14bit AD Converter. The pictures with +3EV are totally overexposed. Why I canâ€™t see 16384 in the RAW Data? Yes I know, it is an average of R G B Pixels. Is this the answer that only on color goes to full scale? Is it possible just to meter one color?

Why the RAW values are not going under 512? Yes I know a little bit about the black level but not really much. Does this mean, the values donâ€™t go lower? But Why? We lose 8 Bit on information on the AD converterâ€¦

On the end of the day, isoinc is really a great tool to investigate all about this. Great job.
A Tv inc option and a changeable step size (for longer ranges) could be really helpful also for other guys.

ISOINC_Konst3.csv

17.49KB

« Previous Page Next Page »