Find brightest pixel - LUA Scripting - CHDK Forum
« previous next »
Pages: [1] 2 3   Go Down

Find brightest pixel

  • 20 Replies
  • 11139 Views
*

Offline jmac698

  • ***
  • 138
Find brightest pixel
« on: 26 / December / 2013, 21:52:51 »
Advertisements
I'd really like a way to find the value of the brightest pixel.  I looked at motion detection and histogram, but they only apply to the small res live view, also motion doesn't have an option for brightest pixel. 

I have a few uses, one is to ensure no highlights are ever overblown (bypassing normal evaluative exposure), another is to find a good exposure for planets, which are only 20 pixels wide so nearly impossible to expose automatically.

Someone even made a get_pixel function, but my target is way too small to be seen in a live view, or if it it's shown, the down-scaling would be equivalent to an averaging and not an accurate value.

Are .cr2 images compressed? If not I could probably parse the raw myself in script.  Ideally I could read a raw buffer directly.
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Find brightest pixel
« Reply #1 on: 27 / December / 2013, 01:24:05 »
The brightest pixel will likely be a hot pixel, so if you want to find over-exposure, you need to detect a some larger number of bright pixels in proximity. Even a point source will end up saturating several pixels.

Quote
Are .cr2 images compressed? If not I could probably parse the raw myself in script.  Ideally I could read a raw buffer directly.
CHDK raw files are straight frame buffer dumps, uncompressed. See http://chdk.wikia.com/wiki/Frame_buffers#Raw

The packed little endian format will painful and extremely slow to process in script. You can look at core/raw.c for pixel reading functions.

Actual Canon CR2 files from cameras that have raw in the stock firmware are a completely different format, and are usually compressed AFAIK.

You might try http://chdk.wikia.com/wiki/Script_commands#get_histo_range
This makes a histogram of the raw buffer and allows you to get a count of the pixels in a certain range. Note however that for performance reasons, it doesn't sample every pixel in the raw buffer. See core/shot_histogram.c
Logged
Don't forget what the H stands for.
*

Offline jmac698

  • ***
  • 138
Re: Find brightest pixel
« Reply #2 on: 27 / December / 2013, 12:35:00 »
That was very helpful, thanks!

The code is here https://github.com/c10ud/CHDK/blob/master/core/shot_histogram.c
Code: [Select]
#define SHOT_HISTOGRAM_STEP 31 so every 31 pixels.  The code doesn't seem correct though, there's an extra 72 and 35 pixels strips for black level on my sensor.  Ideally it should count only the active area, but I haven't completely figured out where exactly it's counting.  Ideally also it should compensate for black level.  If I fix that up, could I submit a patch?

Btw, I have no hot pixels, is that unusual?
« Last Edit: 27 / December / 2013, 12:37:44 by jmac698 »
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Find brightest pixel
« Reply #3 on: 27 / December / 2013, 16:05:42 »
Quote from: jmac698 on 27 / December / 2013, 12:35:00
The code is here https://github.com/c10ud/CHDK/blob/master/core/shot_histogram.c
That isn't the CHDK source repository (it may be an up to date mirror or not, I don't know) The official source is at https://trac.assembla.com/chdk/browser/trunk/core/shot_histogram.c

If you want to work with the source, I suggest checking out from http://subversion.assembla.com/svn/chdk/trunk
Quote
  Ideally it should count only the active area, but I haven't completely figured out where exactly it's counting.  Ideally also it should compensate for black level.  If I fix that up, could I submit a patch?
You can always submit a patch. If you can clearly identify a problem and explain how your patch solves it while not making the code excessively complicated, there is a good chance we will accept it.

I'm not sure I agree that it should compensate for black level. Analyzing the masked portion of the sensor would require a camera specific definition for >100 cameras. If you just use the fixed black level defined in platform_camera.h, the script can do that just as well.

I do agree it should probably use active area. The reason it doesn't is historical: Active area was only defined for DNG support, and the shot_histogram code pre-dates that.

You may also want to look at http://chdk.setepontos.com/index.php?topic=8997.0
Quote
Btw, I have no hot pixels, is that unusual?
How have you determined that?

There are varying degrees of "hot" pixels: Stuck pixels that are always on, hot pixels with excessive dark current. The latter will show up at longer exposures. The stock firmware does a dark frame for long exposures (usually 3 seconds or more) which will also remove hot pixels.

From the factory, the camera has some list of bad pixels it knows to fix, so it wouldn't be surprising for a new camera to have no visible hot pixels, but over time more pixels will go bad.
Logged
Don't forget what the H stands for.
*

Offline jmac698

  • ***
  • 138
Re: Find brightest pixel
« Reply #4 on: 28 / December / 2013, 13:19:42 »
I've been doing these experiments, but I'll talk about it more in a noise thread:

dark frame, 10 degrees ccd temp, 16 minutes exposure
dark frame, 40 degrees ccd temp, 8 minutes exposure
They are cropped from the center of the raw sensor data.  Converted with
Code: [Select]
dcraw -D -b 1 -T IMG_6909.CR2 6909.tifAnd then adjusted for brightness/contrast
The virtually noise free image in the cold, dark frame is typical of the entire image, there's no really bright pixels at all in the whole frame.  Also, the create badpixels.bin thingy reports 0 bad pixels.

« Last Edit: 28 / December / 2013, 13:24:17 by jmac698 »
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Find brightest pixel
« Reply #5 on: 28 / December / 2013, 15:37:51 »
Quote from: jmac698 on 28 / December / 2013, 13:19:42
I've been doing these experiments, but I'll talk about it more in a noise thread:

dark frame, 10 degrees ccd temp, 16 minutes exposure
dark frame, 40 degrees ccd temp, 8 minutes exposure
Was Canon dark frame disabled?

Which camera?

Quote
Also, the create badpixels.bin thingy reports 0 bad pixels.
That only handles bad pixels known to the canon firmware, so it wouldn't count hot pixels. AFAIK some newer cameras handle badpixels differently and report none with this method.

There is no chance that a multi-megapixel consumer sensor would actually have zero bad pixels.

edit:
It is possible that the camera firmware is hiding all the hot pixels though, so for the purpose of this thread, finding the brightest pixels might actually work. It wouldn't be very portable though.
« Last Edit: 28 / December / 2013, 15:47:40 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: Find brightest pixel
« Reply #6 on: 28 / December / 2013, 15:41:13 »
Quote from: reyalp on 28 / December / 2013, 15:37:51
AFAIK some newer cameras handle badpixels differently and report none with this method. There is no chance that a multi-megapixel consumer sensor would actually have zero bad pixels.
Confirms my suspicion about my SX50hs reporting zero bad pixels.  I just never get that lucky in life   :P
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline jmac698

  • ***
  • 138
Re: Find brightest pixel
« Reply #7 on: 28 / December / 2013, 19:09:51 »
It's indeed an SX50HS.  I disabled dark frame subtraction in the CHDK menus, and also checked my log:
date,exp,ccd temp,opt temp,vbatt
12/26/2013 21:20:45,6907,9,-6,7693
12/26/2013 21:37:54,6908,10,-4,7556

I was aiming for an exposure of 16 minutes, and it took 17:09.

I've now analyzed that raw frame with http://www.rawdigger.com and it says
Blue pixels, min=0, max=188, avg=1.58, RMS=3.04, area sampled=1256x1304
The blue pixels had the worst results (though I don't see why there should be any difference in the colors if it's black).

I had to use an area because I noticed some light leakage.

Here's maybe a better test of hot pixels, this time using the whole frame and 15s exposure
Red pixels, min=0, max=384, avg=.783, RMS=1.45, area sampled=4176x3062
The image area doesn't correspond to what dcraw gives me.
These were from the Canon .cr2 files, the DNG from CHDK doesn't work.

Logged
*

Offline reyalp

  • ******
  • 14126
Re: Find brightest pixel
« Reply #8 on: 28 / December / 2013, 19:46:15 »
Quote from: jmac698 on 28 / December / 2013, 19:09:51
I had to use an area because I noticed some light leakage.
The "light leakage" is probably actually amp glow. Dark frame subtraction will help this, but for such long exposures you are probably better off making a master dark and subtracting it after the fact, rather than doing it in camera for each exposure.

Note that to create dark frames, you can use CHDK to force the mechanical shutter closed.
Quote
These were from the Canon .cr2 files, the DNG from CHDK doesn't work.
How does the DNG not work?

I would not be surprised if Canon cr2 has some bad pixels fixed or identified in some way. That's fine if you want to use the image, but if you want to look for bright pixels in the raw buffer, CHDK DNG is will give you a better idea of what is there.

FWIW, a 128 second ISO 80 master dark (average of 10 darkframes) for my d10 has 69 pixels with value 4095  (max possible), and 125 >=2048
Logged
Don't forget what the H stands for.
*

Offline jmac698

  • ***
  • 138
Re: Find brightest pixel
« Reply #9 on: 28 / December / 2013, 19:48:57 »
I took a dark frame with CHDK raw and am just converting it now, I'll report back later. 

I'll also verify all the DNG options and report which don't work.

I've thought of how to verify the light leak and use the close shutter method from http://chdk.wikia.com/wiki/Meteor_Intervalometer_with_Dark_Frame_Management

I'm familiar with amp glow and this looks different, as below.

ps amp glow comes from an electric field so it will be smooth in appearance, in one particular area, also I don't expect it in a modern sensor, and it might only apply to CCD (just some rules of thumb off the top of my head).

pps I might also mention that besides setting auto exposure for planets and stars, I'm also investigating noise with an aim to derive a formula to remove noise based on the variables temperature, ISO, and shutter speed, then I wouldn't need to take dark frames at the time, and can even go back and clean up some night shots I've previously made with the same camera.  This formula could also let me estimate the temperature of a picture when it was taken.  There's even  more you can do, like identify which camera a picture came from.
« Last Edit: 28 / December / 2013, 20:07:38 by jmac698 »
Logged
Pages: [1] 2 3   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal