Histogram - page 10 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 5 6 7 8 9 [10] 11   Go Down

Histogram

  • 109 Replies
  • 24393 Views
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #90 on: 21 / January / 2013, 18:00:29 »
Advertisements
Thanks for posting that script. I'm glad to hear that it works. My next thing is to finish a new function called:

get_shot_histo(...bunch of parameters...)

It will return an Apex96 value that you can add to Tv96 to set the next picture exposure based on the histogram, and parameters. Then, the next shot would be in the ETTR range or ETTL range. You could do your script in 3 shots, or maybe a few more under certain conditions.

What I plan to do is start from the white level and count pixels going down 1 at a time towards black level, stopping when the conditions set in the parameters are met, similar to your script. Then I can convert the linear pixel brightness level into an ev96 exposure change for the next shot.

It works pretty well with the shot_meters, which add up all the pixel values and get the average, then calculate an exposure change for the next picture to be 2 f/stops below the 50% level.

tv96=get_tv96()+get_shot_meter(1)
set_tv_96_direct(tv96)
--shoot next picture and wait for get_shot_ready()

It might be useful to have that work as an option in the CHDK menu for auto bracketing or something. At this point, I'm only thinking of it as a Lua enhancement.
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #91 on: 21 / January / 2013, 18:48:28 »
Sounds good. I will do some thinking on ETTR myself.

BTW please 'throw' any testing towards me if you would like.

I also tweaked my script to make it more accessible to photographers. What I have done is restrict the input to test the top and bottom zones of the camera histogram to 0, 1/3, 2/3 or 1 stop. For the top zone (out of the five notional camera histo zones) 0 means I only check the histo count in the 4095th bin. For the 1/3 'stop' I use the count in the right most 2048/3 bins etc. I understand this  is a linear assumption.

For the bottom zone I do the same but use 128/3. That is test the 128 bin (n=0) or 128n/3 left most bins from 127, where n = 1, 2 or 3.

Cheers

Garry
Logged
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #92 on: 21 / January / 2013, 22:36:55 »
Thanks for the info. Can you explain the zones and bins thing a little for me? Maybe you have a link to a site that explains it? Would there be more zones with 12 and 14 bit sensors?
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #93 on: 21 / January / 2013, 23:28:55 »
Lapser

Start reading here http://www.normankoren.com/digital_tonality.html

I admit to simplifying things a bit, however, if you look at, say, a CHDK histogram, the five zones can be imagined to represent five 'stops'.  You can convince yourself of this by tracking how a simple image, ie a single colored wall, with a singe peak at 0Ev, moves one zone when you change the exposure by one stop.

This is not linearly correct, across the full histogram, however, I believe it is a pragmatically useful assumption.

If you take a 12 bit camera, the zone on the right will hold half of the tonal info, ie 2048 bins of the total 4096 bins. The next zone will hold 1024, the next 512, the next 256 and the fifth zone on the left 128 bins. That is, for a 12 bit Canon sensor the black level is at 127. Each bin collects the photons/charge at that tonal value, ie all five zones cover tonal values 128 to 4095.

But note that the right hand zone holds more bins than the left hand zone, ie 2048 vs 128. Hence the view that you should expose to the right, without clipping, to get as much tonal data as possible in your image.

What I did in my bracketing script was to divide the rightmost and leftmost zones into 1/3 Ev sub-zones. In other words the user can decide how much of these two end zones is zeroed out, ie 1/3 or 2/3 or 3/3 of the zone.

Of course the above is my simple understanding and I accept my language and maths may not be 100% accurate. But it is close enough or me.

Cheers

Garry
Logged
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #94 on: 22 / January / 2013, 01:10:11 »
Thanks for the good info. When you say "zeroed out", do you mean absolutely 0 pixels are in any brighter bin, or do you have a cutoff minimum number of pixels? It sounds like you go for zero, and it works, on the bright and dark ends of the histogram.

I'm also concerned about what happens with hot pixels in long exposures. I think they'll show up isolated in the brightest bin only. It might work to ignore the brightest bin, i.e. the hot pixel bin. Presumably, there will be pixels in the bins just below the brightest if the image is really overexposed.

I'll be working on the histogram to exposure compensation function soon. I'll post a build for you and see what you can do with it when it's ready. Thanks for all the help.

And did you ever look at the bins below the black level? Specifically, I wonder if bin 0 has a lot of pixels in it because of the zeroed out dead pixels the firmware zaps. They show up as colored dots in your dng images.
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #95 on: 22 / January / 2013, 09:46:28 »
Lapser

I should have written it like this: 'zeroed out'. In other words I consider any part of the histogram I am sampling to have reached zero when it falls below a threshold. The threshold is user selectable between 10 and 200 in steps of 10.

When I look at S95 or G11 RAW histograms in RawDigger I observe the following (all general scenes with mixed tones):
* You can 'guarantee' there will be no data below the 'black level', ie 127 and including at in the 0 bin. Note this is the RAW file as ejected by the camera and by Canon (ie I don't know what processing takes place between your sampling and the RAW creation in the camera). However, been I looked at your counts below the black point, they too were zero.
* When I underexpose, ie push the histogram far to the left, you do see 'image residue' data in the right hand end of the histogram (ie scene statistics), but the RawDigger actual RAW counts are, in any bin with 'image data' down to between 1-5, ie very low.
* If I take an image at the shortest shutter speed I can and at a dark image, I see a nice Gaussian peak in RawDigger at 127, I do not see a 'spike' at 4095, even after I have underexposed this far. In other words, I am not seeing any 'hot pixcels': my assumption being I would see these in the 4095 bin (but I could be wrong here?).

The above has told me that your count (the lapser-count), which is statistically down on the real count,  ie you sample the real RAW data, means that if I use a 'zero threshold' or 10-200 (to account for bin width), I should be OK. In other words if I use a lapser-count in the righthand 1/3 stop of the righthand zone (2046 bins wide), I can use 10 or 20 as my threshold, ie this is my 'zero'. Indeed this is what I am finding.

Of course, each scene is different and speculars could throw my 'zero' test off. However, I feel that for any given shoot, I would do a few test shots to set the thresholds (10-200) for that particular lighting environment, eg in a cathedral. Once I have my thresholds set up, I will capture away using auto bracketing, and burn my way through as many (cheap) 4Gb SD cards as I need! BTW I still have not worked out how to put CHDK on a 8Gb card or larger without a lot of 'mucking about'.

Bottom line: when I look at RawDigger I always see 0 data between 0 and 127ish. Also, if I underexpose I see very low scene-dependent RawDigger (image) data at the righthand end of the histogram, ie up towards 4095. On a very underexposed image I see nothing at 4095, ie if I assume hot pixcels would be there. Finally, I find a lapser-count of 20 works for me, but as I say, this may need to be adjusted to account for a particular scene.

Cheers

Garry

Logged
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #96 on: 22 / January / 2013, 12:46:19 »
Quote from: pigeonhill on 22 / January / 2013, 09:46:28
* If I take an image at the shortest shutter speed I can and at a dark image, I see a nice Gaussian peak in RawDigger at 127, I do not see a 'spike' at 4095, even after I have underexposed this far. In other words, I am not seeing any 'hot pixels': my assumption being I would see these in the 4095 bin (but I could be wrong here?).
Hot pixels only appear at shutter speeds over about 1 second. Most cameras start taking an automatic "dark frame" above one second, and subtract it from the picture to cancel out hot pixels. This doubles the time to take each picture, which is not always desirable. You can turn this off in the main CHDK menu under Raw/Dark Frame Subtraction (set it to OFF).

So to see hot pixels, turn off dark frames and take a 60 second exposure with the lens cap on and/or in a dark closet.

I was trying to figure out what all the dots are in the dng images you posted. They may be "dead pixels", which apparently show up as totally black, i.e. 0. They might show up in my histogram as 0 pixels, but I probably am skipping over them since my sample step size is around 50 pixels for a full image.

Maybe you could try taking a totally overexposed image, i.e. 60 second exposure in a bright room, and see if you see the dots in the dng file. If so, maybe Rawdigger could tell you their value, i.e. 0?
Quote
The above has told me that your count (the lapser-count), which is statistically down on the real count,  ie you sample the real RAW data, means that if I use a 'zero threshold' or 10-200 (to account for bin width), I should be OK. In other words if I use a lapser-count in the righthand 1/3 stop of the righthand zone (2046 bins wide), I can use 10 or 20 as my threshold, ie this is my 'zero'. Indeed this is what I am finding.
OK, that makes sense. The "lapser count" (has a nice ring to it), is an absolute number of pixels when the scale is 0. The total number of pixels varies, so it might be better to use a scale of 10000 or so. The result returned by get_histo_range(from,to,scale) is:
(lapser_count*scale)/total
A 0 scale just returns lapser_count, as you know. The original function used a scale of 100. For now, the total number of pixels should be between around 8,000 and 12,000, but that may change in future updates. A scale of 10,000 should give you the precision you need, plus it should work if the number of pixels increases or decreases.
Quote
Of course, each scene is different and speculars could throw my 'zero' test off.
One of my obsessions is sunset time lapses, which start out with the sun in the frame. That's about as "specular" as you can get, so if you figure that one out, let me know.
Quote
BTW I still have not worked out how to put CHDK on a 8Gb card or larger without a lot of 'mucking about'.
Try formatting your SDHC card with this program. Then add full CHDK, boot with firmware update, then go to Misc/SD card/ in the CHDK menu, and make it bootable (with write protect). Then you can add my custom builds. It works for my 64GB card.

http://www.ridgecrop.demon.co.uk/guiformat.exe
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Histogram
« Reply #97 on: 22 / January / 2013, 14:12:00 »
Quote from: lapser on 22 / January / 2013, 12:46:19
I was trying to figure out what all the dots are in the dng images you posted. They may be "dead pixels", which apparently show up as totally black, i.e. 0. They might show up in my histogram as 0 pixels, but I probably am skipping over them since my sample step size is around 50 pixels for a full image.

Yes, they are dead pixels - when read out from the sensor the value returned is 0.

I asked pigeonhill to use DNG version 1.3 in CHDK because this turns off the built-in CHDK bad pixel removal and instead inserts an opcode into the DNG file to tell processing software to do it instead. This also avoids the need to create the badpixel.bin file. The downside is that if your DNG processing software doesn't understand and use the opcode then it won't fix the dead pixels and you will see them in the image. So far only Adobe software has been found to do this properly.

You can force CHDK to do bad (dead) pixel removal by setting the DNG version to 1.1 and creating the badpixel.bin file (it will force you to do this before you can save a DNG file). The disadvantage of this is that recent CMOS cameras can have tens of thousands of dead pixels so it requires extra memory and processing time to fix them.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #98 on: 22 / January / 2013, 14:14:13 »
Lapser/philmoz

Took an S95 15sec exposure (ISO 80) and looked at the DNG and RAW. I had dark frame off.

Interesting.

The Canon RAW only shows the overexposed data, ie a spike at 4095 and no data anywhere else.

The DNG shows the spike at 4095 and a spike of about 700 at 0.

Does this make sense?

Cheers

Garry
Logged
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #99 on: 22 / January / 2013, 14:32:59 »
BTW I had already tried the >4GB procedure you outlined, but tried it again.

The firmware update occurs and I make the card bootable in CHDK and set the tab to locked.

But all I get is a canon message saying 'card locked'.

It's a mystery (well to me!).

Garry
Logged
Pages: 1 ... 5 6 7 8 9 [10] 11   Go Up
« previous next »
 

Related Topics