Shot Histogram Request

lapser

1093

Re: Shot Histogram Request

« Reply #30 on: 02 / December / 2012, 10:22:10 »

Advertisements

In my last post, I figured out how to correct for the black level by adjusting each of the 12,000 pixel samples with this formula:
p=((get_raw_pixel(x,y)-CAM_BLACK_LEVEL)*CAM_WHITE_LEVEL)/(CAM_WHITE_LEVEL-CAM_BLACK_LEVEL);

To avoid doing 12,000 new multiplications and 12,000 new divisions, not to mention subtractions and additions and truncations, I tried to do the correction to the final sum:
hsum=(((hsum-CAM_BLACK_LEVEL*npts)*CAM_WHITE_LEVEL))/(CAM_WHITE_LEVEL-CAM_BLACK_LEVEL);
This led me down the overflow road to long int land, which got more complicated than I wanted. I decided to sleep on it.

This morning, I realized that since my result is log2_96(hsum), I don't need to do any multiplications, since log(x*y)=log(x)+log(y). I only need to subtract the black level from each pixel, and not worry about scaling it to full range. The scaling factor is constant, and just adds a fixed amount to the log result. I'm only using the difference between 2 pictures, not the absolute log value. I can skip the scaling part, and thus skip the overflow problems. The new equation has only 1 extra subtraction:
p=get_raw_pixel(x,y)-CAM_BLACK_LEVEL;

I can also move this out of the 12,000 pixels loop, without any overflow:
hsum=hsum-CAM_BLACK_LEVEL*npts;

I also discovered in my research that CAM_BLACK_LEVEL is an approximation. The active area of the sensor is towards the right side. The left side contains a masked area that doesn't receive any light. The average value of the masked pixels can be used to find the actual black level, which is affected mostly by temperature, and probably varies from camera to camera. Canon raw images include the masked pixels, but dng images don't, although dng does contain a black level value. This should probably be computed from the masked pixels average when creating a dng. Maybe I'll try some black level experiments in the future.

Here are the results from simply subtracting the black level from each pixel. The target for the values on the right is -32. The jump between series of shots should be 384. However, I do a new half shoot before each series, so part of that jump may be a small exposure change from the half shoot. The average of all the jumps within the series is -31.5. So this should make an extremely accurate exposure meter, much better than half_shoot.

Code: (txt) [Select]

  ev+- | deltaHSUM96
  -192     0
  -160   -29 
  -128   -28
   -96   -32
   -64   -32
   -32   -29
     0   -32
    32   -33
    64   -30
    96   -31
   128   -32
   160   -29
   192   -33 (series average -30.8 )

  -192   368
  -160   -33
  -128   -29
   -96   -32
   -64   -33
   -32   -28
     0   -33
    32   -33
    64   -29
    96   -32
   128   -33
   160   -30
   192   -32 (series average -31.4)
   
  -192   376
  -160   -32
  -128   -29
   -96   -32
   -64   -33
   -32   -29
     0   -33
    32   -32
    64   -30
    96   -32
   128   -33
   160   -30
   192   -31 (series average -31.3)
   
  -192   376
  -160   -30
  -128   -32
   -96   -33
   -64   -30
   -32   -32
     0   -33
    32   -29
    64   -32
    96   -33
   128   -30
   160   -33
   192   -32 (series average -31.6)
   
  -192   379
  -160   -33
  -128   -33
   -96   -36
   -64   -32
   -32   -28
     0   -37
    32   -32
    64   -29
    96   -32
   128   -28
   160   -28
   192   -33 (series average -32.6)
             (average of all -31.5)

« Last Edit: 02 / December / 2012, 13:35:54 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #31 on: 02 / December / 2012, 11:22:24 »

I corrected the rounding error in my log2_96 function and did the test again. With the camera pointed at a blank wall, I took a series of shots, varying the shutter speed from -2 ev to +2 ev in 1/3 ev steps from the measured value of the initial shot, by keeping the shutter button half way down between shots using "shoot_full_only".

After each shot, I called my new function which adds up all the sampled pixel values in the resulting image, and returns 96*log2 of this sum. The absolute value is meaningless, but the difference between two shots tells you how much the exposure changed. In this test, it should be -32 each time, or 1/3 ev times 96. The results were perfect. Problem solved.

I'll post a test diff file and documentation of the new functions in a new topic, hopefully in a few days.

Code: (data) [Select]

  ev+- | deltaHSUM96
  -192     0
  -160   -31
  -128   -34
   -96   -33
   -64   -29
   -32   -34
     0   -33
    32   -30
    64   -34
    96   -32
   128   -30
   160   -33
   192   -33 
   average=32.166

  -192   384
  -160   -32
  -128   -34
   -96   -29
   -64   -34
   -32   -33
     0   -30
    32   -32
    64   -34
    96   -30
   128   -32
   160   -33
   192   -31
   average=32.00
   
  -192   382
  -160   -30
  -128   -33
   -96   -33
   -64   -30
   -32   -34
     0   -32
    32   -30
    64   -33
    96   -34
   128   -30
   160   -32
   192   -34
   average=32.08
   
  -192   385
  -160   -30
  -128   -32
   -96   -34
   -64   -29
   -32   -34
     0   -33
    32   -30
    64   -33
    96   -33
   128   -30
   160   -33
   192   -33
   average=32.00
   
  -192   385
  -160   -33
  -128   -34
   -96   -30
   -64   -26
   -32   -35
     0   -31
    32   -33
    64   -32
    96   -30
   128   -34
   160   -32
   192   -31
   average=31.75
   
   overall average=31.99

« Last Edit: 02 / December / 2012, 13:32:25 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Shot Histogram Request

« Reply #32 on: 05 / December / 2012, 03:01:15 »

Hmmm, I believe your logic is based on the assumption that a doubling or halving of the sum total of the raw sensor value from one image to the next, equates to a 1 f/stop change in image brightness.

I'm also assuming your using the existing logic in build_shot_histogram that only uses the top 10 bits of each raw value.

So your logic will only work if the dynamic range of the sensor is close to 10 f/stops.

If you were using your G1X then this will be reasonably close - the measured DR at base ISO is 10.8 stops for the G1X.

But the DR drops as the ISO rises so if you re-ran the test at a high ISO you might see very different results.

You are also likely to see quite different results on other cameras with different DR - the only measured value for the SX260 that I can find online is 7 stops (which I think is wrong). Another example, the G12 is reported as 11.2 stops at base ISO.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Shot Histogram Request

« Reply #33 on: 05 / December / 2012, 03:37:11 »

Quote from: philmoz on 05 / December / 2012, 03:01:15

I'm also assuming your using the existing logic in build_shot_histogram that only uses the top 10 bits of each raw value.

So your logic will only work if the dynamic range of the sensor is close to 10 f/stops.

I'm adding the sum of the pixel values before the bit shift, as I said in the other topic (I see what you mean, Waterwingz. You're right as usual.)

But the proof is in the data above. Over a range of 4 f-stops, I get an almost exact correlation. When I change the exposure (shutter speed) by 1/3 f-stop, the log2 of the pixel sum changes by the exact same amount. It should work over the entire dynamic range of the camera, no matter which camera it is.

I was kind of surprised to see how well it worked. But if you ever tried a program like LRTimelapse, it also measures brightness from the image and computes exposure compensation differences to feed back to Photoshop Lightroom. I'm doing the same thing with CHDK, but the exciting thing is I can feed back the exposure compensation to the camera, and have it correct the actual exposure, instead of the photo after the fact..

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #34 on: 05 / December / 2012, 14:15:23 »

To make things easier to follow, I'll post all my future progress here instead of in new topics (thanks Waterwingz). If I think a new topic is indicated, I'll check with Waterwingz first. To make things more coherent, I'll put unrelated ideas in different posts, all in this topic. That said, my next idea will be in the next post.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Proposed Lua functions

« Reply #35 on: 05 / December / 2012, 15:06:47 »

My code has gotten too complicated by mixing the old set_shot_histo and get_histo_range together with my new stuff. It also will be too hard to explain. So here's my new proposal.

1. Leave the old 2 routines the way they were. They will behave almost exactly like they have in the past (except fixing a few bugs).

But I don't think a histogram of the previous shot is useful. A histogram before taking a shot is very useful for framing the shot and setting exposure, but it doesn't tell you much after the shot, except giving an idea of over and underexposure in high contrast images. I think I can do that better in a new routine though.

2. I'll add two new routines:

To enable shot metering, you call:

npix=set_shot_meter(set_options,xp1,yp1,xp2,yp2,lowp,highp)

xp1,yp1,xp2,yp2 is the area of the image that will be used for metering. Values are in percent of the active area.

lowp,highp are the pixel levels in percentage for the "mini histogram" result (see below).

options: 0 disable, 1 enable, 2 set reference shot? future expansion

To disable shot metering, you call the set function without arguments or with option==0:

set_shot_meter(0)

To retrieve the data after a shot:

ev96rel, nlow, nhigh = get_shot_meter(get_options)

ev96rel is the change in ev from the reference shot, which can be added to the current exposure values so the next shot matches the brightness of the reference shot.

nlow is the number of pixel samples that are below the lowp value set above. This would be like:
nlow=get_histo_range(0,lowp), only the value would be more accurate

nhigh is the number of pixel samples that are above the highp value set above. It would be like:
nhigh=get_histo_range(highp,1023)

The number of pixels between nhigh and nlow would be: npix-nhigh-nlow. So you have a "mini histogram" with 3 ranges that is based on the actual number of pixels, and uses the full value of the pixel without bit shifting.

One of the "set_options" might be to use the actual, 12 or 14 bit pixel level as the cutoff, for greater accuracy. If you specify the high range as the white level, that could tell you how many totally blown pixels there are, or maybe how many hot pixels there are in a night image.

get_options could be used to return different data, like the actual log2(sum) instead of a relative value. That way you could save it, or compare it to an absolute value taken from a previous shot with the same area, without having a recent reference shot. You supply your own value from a reference shot taken in the past.

a possible set_option could be to use the entire image for the mini-histogram, and the area from set_shot_meter for the ev96rel value. That way, if something happened outside your metering area, like the sun setting or moon rising, you could detect it and adjust exposure if needed.

So I've got some more re-working of the code to do before I can put anything out. I welcome questions, comments or suggestions.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Hot Pixel removal on the fly?

« Reply #36 on: 05 / December / 2012, 15:19:37 »

In the past, Photoshop Lightroom would automatically remove hot pixels from an image when it loaded. You didn't even know they were there. I'm not sure it's still working, but it made me think.

One of the things I did in my histogram routines was store the maximum pixel value (WHITE_LEVEL) at the x,y position of each sample so I could see dots in the image. This told me if I was setting the area correctly, and helped me select pixels to sample representing all the colors, and as evenly distributed as possible.

Anyway, knowing I can change individual pixels, how about finding and correcting hot pixels? My understanding is that a hot pixel is a completely saturated single pixel. What makes it stand out is that the surrounding pixels are relatively dark. This could be detected by going through the raw image. Has anyone tried this? I know there are a lot of threads on hot pixels and raw images, but I'd love to have a go at it as a future project if the experts here think it might work.

It would be an interesting challenge to see how quickly it could be done. I accidentally used a step size of 1 in my histogram tests, and set the entire raw buffer to white. It added around 8 seconds, so I'm sure I could get through the entire image buffer much more quickly if I did it on purpose.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

First Test Script

« Reply #37 on: 06 / December / 2012, 02:38:25 »

I finally got a test script to work. Don't save or try to run this. I'll be changing the names of the functions before the final version. Skip the stuff at the top and scroll down to the main loop at the bottom. It shows how I use the new functions to set the exposure during half shoot. Just messing around with it a little, it worked great. When you point the camera at something light and start the script, take a few shots, and move the camera to something dark... you get an underexposed shot of the dark area that my routine measures. The next shot has the dark area exposed correctly. The exposure is one shot behind, as expected, but it works great! More tests to follow.

Code: (php) [Select]

--[[
@title New Time Lapse
@param m = interval
@default 3
--]]

function restore()
  release("shoot_half")
  hfile:close()
  shot_histo_enable(0)
end

function press_half()
  press("shoot_half")
  repeat sleep(50) until get_shooting()
end

function release_half()
  release("shoot_half")
  while(get_shooting())do sleep(50) end
end

nkey=6
keys={"set","menu","left","right","up","down"}
function keypress(k)
  repeat
    for i=1,nkey do
      if(is_pressed(keys[i])) then
        if(k==nil or k==keys[i]) then return keys[i] end
      end
    end
    sleep(50)
  until false
end

function nopress()
  repeat
    none=true
    for i=1,nkey do
      if(is_pressed(keys[i])) then
        none=false
        sleep(50)
      end
    end
  until none
end

fname="A/CHDK/LOGS/H_"..os.date("%y%m%d%H%M")..".LOG"

props=require("propcase")
pTV=props.TV
pTV2=props.TV2
props=nil

shot_histo_enable(1,3,3,50,25) --upper left
hfile=io.open(fname,"wb")
hfile:write(string.format("%s\n",os.date()))

print("press set / menu exits")
done=keypress()=="menu"
press_half()
hfile:write(string.format("tv0=%d\n pic tv tvd\n",tv))
while(not done)do
  get_histo_ev96rel(2) --reset building histo flag
  press("shoot_full_only")
  repeat
    sleep(100)
  until get_histo_ev96rel(3)==0 -- building histo flag set to 0 when building is done
  release("shoot_full_only")
  sleep(500)
  tvd=get_histo_ev96rel() -- get exposure difference of this shot and reference shot
  tv=tv+tvd -- sets exposure of next shot based on the reference shot (first shot after histo_enable)
  if(tv<0)then tv=0 end
  set_prop(pTV,tv)
  set_prop(pTV2,tv)
  sleep(500)
  nxp=get_exp_count()
  hfile:write(string.format("%d %d %d\n",nxp,tv,tvd))
  done=is_pressed("menu")
end
restore()

Note that the call to: shot_histo_enable(1,3,3,50,25) sets the measurement area to the upper left corner. It also sets a flag to make the next shot the reference shot. The output of get_ev96rel() is relative to the reference shot. The first value for the first picture is always 0, since that IS the reference shot.

« Last Edit: 06 / December / 2012, 02:53:23 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Proposed Lua functions

« Reply #38 on: 06 / December / 2012, 10:58:37 »

Quote from: lapser on 05 / December / 2012, 15:06:47

But I don't think a histogram of the previous shot is useful.

WRONG! I realized that I can find a relative exposure change from the full histogram based on a criteria like:
Calculate the exposure change needed so 3% of the pixels are above a specified value.

To find this, you would start from the top of the histogram. You would take the sum of the pixels from the top down to the point that the sum was 3% of the total sum. You would subtract this index from the top index to get the exposure increase needed.

Exposure changes move the histogram left or right by the amount of the change (linear).

To implement this exactly, I'll need the full histogram, not the bit shifted 10 bit version, which uses 1024 2-byte short int values (2K).

12-bit camera have 4 times the range of values, which increases the memory requirement to 8K.

14-bit cameras (the G1X) need 4 times that, or 32K.

This memory would only be activated by set_shot_histo(1), and released by the script terminating. I'll also have an option for just calculating brightness and not a histogram, which won't use any extra memory.

So do you think that increasing the temporary memory allocation by these amounts is OK?

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

First Time Lapse Video with new shot_meter functions

« Reply #39 on: 06 / December / 2012, 20:57:47 »

This was a good test in that it triggered a glitch that's easy to fix. But the scene was too bright to give a good test of the accuracy of the new shot_meter functions.

The first picture was so bright, the initial shutter speed was 1/2000, which is the input maximum for the sx260. Shot_meter kept asking for higher shutter speeds, and my lua script dutifully stored them in propcase TV. The camera maxed out near the beginning of the video at 1/3200 sec. But the program kept calling for higher and higher shutter speeds, finally reaching the nano-nano second range and maxing out a short integer at 32767. The next request was for 32770, which is a negative number corresponding to leaving the shutter open for around 10 years. Fortunately, the camera decided that 64 seconds was long enough.

The dot pattern at the upper left is the metering area, set by new parameters to set_shot_histo. The frame rate was 1 picture per second. With my new timing function, get_shot_ready(), I was able to use a 1 shot per second interval, with 0.4 seconds left over. So I should be able to shoot even faster than this in bright light. Of course, with a shutter speed of 64 seconds, the frame rate drops to less than 1 per minute.

I'll try again tomorrow with some out of range testing in the program, and the camera pointing away from the sun.

http://www.youtube.com/watch?v=gVft5lSjSMI#ws

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos