Dynamic Range & CHDK RAW Bits?

jmac698

Re: Dynamic Range & CHDK RAW Bits?

« Reply #20 on: 06 / December / 2012, 01:13:55 »

Advertisements

Yeah, it's normal for all sensors to have a masked area around the edges, to estimate black level.
So I took some dark frames and measured the noise myself. This is a sd1200is which is ccd and has amp glow in the upper left corner.
Oops, these are the jpg's, I'll have to do it again. Anyhow, I got average of 8.45, std=8.8
range 0 to about 62 (ecluding high 1%)
I have to double check these.
Yes I'm working on a script/program to make/process frames for noise, calibration and measurements.

Logged

jmac698

Re: Dynamic Range & CHDK RAW Bits?

« Reply #21 on: 13 / December / 2012, 09:28:48 »

So I got around to getting some stats on dark frames.
I found they get slightly brighter and the standard deviation decreases as the temperature lowers.

Someone said that only 10 bits of values actually existed in the RAW's; however I'm getting every value so it seems to be 16bit.

For example for sx260is I get (and I've posted this elsewhere a few times)
range 78-180
mean 128
median 128
std dev 2.6

And some more test conditions,
the next 3 analysis were done on the upper 12 bits,
so aren't really accurate

iso100, sx240, 15seconds
sx240 hot
11   2
10   8
9   1659
8   6,709,979
7   5,671,809
6   3359
5   51
4   10
3   2
2   1
1   0
0   0
mean 7.5
median 8
std dev 0.5

iso100, sx240, 59seconds
sx240 hot
12   4
11   6
10   28
9   9437
8   8,250,325
7   4,067,562
6   51094
5   6265
4   1470
3   471
2   169
1   36
0   12
mean 7.7
median 8
std dev 0.5

iso100, sx240, 59seconds
sx240 cool
12   0
11   1
10   9
9   1406
8   7,225,878
7   5,158,293
6   1280
5   11
4   2
3   0
2   0
1   0
0   0
mean 7.6
median 8
std dev 0.5

I think the std dev lowers with lower temperature, also dark noise seems to add to readout noise and adds some values to the lower range. I'm wondering if it's just two noises superimposed.

I also see values every step on the lower 8 bits of sd890is raw's, but it's strange looking, as if it's 12bits with a few pixels on nearby values.

Logged

jmac698

Re: Dynamic Range & CHDK RAW Bits?

« Reply #22 on: 13 / December / 2012, 09:36:40 »

Here's some stats on the extra area. The exact mapping can be gotten from dcraw.
sd890 10mp
full raw size
3684x2760
jpg size
3648x2736=9,980,926
extra
36x24=864, 86e-6 of area

sx240is 12mp
full raw size
4080x3036
jpg size
4000x3000=12,000,000
extra
80x36=2880, 240e-6 of area

So they've more than doubled the border pixels on the new sensor.

Logged

lapser

1093

Re: Dynamic Range & CHDK RAW Bits?

« Reply #23 on: 13 / December / 2012, 10:43:40 »

Quote from: jmac698 on 13 / December / 2012, 09:28:48

Someone said that only 10 bits of values actually existed in the RAW's; however I'm getting every value so it seems to be 16bit.

The pixels in the raw buffer are packed together across word boundaries. 10-bit sensors have a new pixel every 10 bits. The sx260 (which I have) has a 12 bit sensor, and the G1X is the only camera so far with a 14 bit sensor. Extracting the value of a single pixel requires unpacking code that is specific for each type of sensor. You'll find this code in the function: get_raw_pixel(x,y) in raw.c

Quote

For example for sx260is I get (and I've posted this elsewhere a few times)
range 78-180
mean 128
median 128
std dev 2.6

Each camera has a specification for black level called CAM_BLACK_LEVEL. It is 127 or 128 for the SX260, which is what you found as the mean dark frame level. Similarly, there is a white level, which is all bits set to 1, or 2^nbits-1. The dynamic range of the camera is: CAM_WHITE_LEVEL-CAM_BLACK_LEVEL

Quote

And some more test conditions,
the next 3 analysis were done on the upper 12 bits,
so aren't really accurate

They don't tell you much because most of the pixel data is bit shifted away.

Canon saves the masked area in their raw files, while most other cameras do not. The raw processor can compute the mean of the masked data and use that as the black level for the individual picture. There could even be a different "black" level for each color pixel. Presumably, this would make the images more accurate than using a fixed black level.

If you repeat your last test with the lower bits, you should find that the 10 bit sensor has values up to 1023 (white level), and the black level should be 31 or 32.

I've described how I discovered this information in this thread:
http://chdk.setepontos.com/index.php?topic=8997.0

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

jmac698

Re: Dynamic Range & CHDK RAW Bits?

« Reply #24 on: 13 / December / 2012, 16:02:36 »

I understood all that, but my brain failed to realize a simple mistake; that the values are right justified. Of course, working only with dark frames I never noticed how high the values could get.
So indeed, the proper way to convert the raw files is to use the lower bits.

Even though I used shifted bits, the changes in histograms are still valid, I'm sure there's an effect from cooling. I'll analyze it with the proper bits now.

You can expand the definition of dynamic range; of course what you said is true for the raw measurement, but change the ISO and see what the true dynamic range of the sensor is, assuming HDR techniques. Then you can fact in non-linearity of response to light (there's actually an S curve) and that's a different dynamic range.

You could hit the sensor white point with too low an ISO I guess. You could also talk about useful dynamic range within a certain noise threshold.

Logged