DNG and badpixel.bin questions - General Help and Assistance on using CHDK stable releases - CHDK Forum  
« previous next »
Pages: [1] 2   Go Down

DNG and badpixel.bin questions

  • 12 Replies
  • 7003 Views
*

Offline thunderstorm

  • *
  • 46
DNG and badpixel.bin questions
« on: 25 / August / 2009, 13:15:24 »
Advertisements
Hello,
I just got my new SX10 (instead of my S5).
I have several questions about DNG and badpixels  :)

1. When i create badpixel.bin i do it inside the camera with the script. I understand there is no need to cover the lens since the picture being taked as a dark frame ?

2. I did it twice - one with my lens cap open (20,000 bad) and one with it closed (11,000 bad). Is it something i have to do every couple of days since the sensor is subject to aging/temp change/etc, because i see that those numbers are changing...
so just doing it once is not enough, i think.

3. Why DNG format requires badpixel table and RAW doesn't? It's just a RAW from the camera converted to more readable format. And if it do requires it, why i have option to select bad-pixel removal or not?

4. I saw that the RAW size is 15MB, and by calcs it means i have 12bit A/D sensor. Does this means that now the RAWS will be with four times dynamic range compared to my previous S5 with 10bit sensor ?
Logged
*

Offline reyalp

  • ******
  • 14003
Re: DNG and badpixel.bin questions
« Reply #1 on: 25 / August / 2009, 16:14:05 »
Quote from: thunderstorm on 25 / August / 2009, 13:15:24
1. When i create badpixel.bin i do it inside the camera with the script. I understand there is no need to cover the lens since the picture being taked as a dark frame ?
No, it just looks for pixels the canon firmware has set to 0 rather than the dark value (128 or so). How exactly the firmware decided which pixels to set to 0 is not clear, it is speculated to be based on a lookup table that involves things like shutter speed and temperature.
Quote
2. I did it twice - one with my lens cap open (20,000 bad) and one with it closed (11,000 bad). Is it something i have to do every couple of days since the sensor is subject to aging/temp change/etc, because i see that those numbers are changing...
so just doing it once is not enough, i think.
If you do it in the worst case that should be pretty good. The CHDK bad pixel code only actually changes the value if the pixel is 0 when the shot is taken, so having a more aggressive badpixel list doesn't hurt quality, it just slows down shooting very slight and takes some memory.
Quote
3. Why DNG format requires badpixel table and RAW doesn't? It's just a RAW from the camera converted to more readable format. And if it do requires it, why i have option to select bad-pixel removal or not?
The DNG specification requires it (actually, the latest version of the DNG spec allows this to be done much more intelligently but it's not widely supported et), and it is always done when DNG is selected. You cannot shoot DNG without it a badpixel.bin. The bad pixel option that you can select is a second, independent badpixel system: http://chdk.wikia.com/wiki/Badpixel_removal
Quote
4. I saw that the RAW size is 15MB, and by calcs it means i have 12bit A/D sensor. Does this means that now the RAWS will be with four times dynamic range compared to my previous S5 with 10bit sensor ?
Not really: http://chdk.setepontos.com/index.php/topic,2468.msg30570.html#msg30570
Logged
Don't forget what the H stands for.
*

Offline thunderstorm

  • *
  • 46
Re: DNG and badpixel.bin questions
« Reply #2 on: 25 / August / 2009, 18:36:21 »
I repeated the test and again got exacly two different bad pixel numbers: 22,391 and 11,408.
I understand both are saved in the firmware of the camera and each is used depending on the shooting conditions.

My questions are:
1. Are the badpixel tables are factory settings written specially for each camera ?
2. If each badpixel table is used for different shooting conditions, by using just one table (for example the smaller number) could result in black dots in the DNG picture or any other artefacts ? Why not store both tables (or at least the biggest one you get)

3. And question about the 12bit RAW:
I was reading the post about it as you linked me to. It clearly shows that only once in 4 value levels is used - showing it's 10bit A/D, so why the sensor delivers 12bit data? To waste memory and CPU time?  :)
Logged
*

Offline reyalp

  • ******
  • 14003
Re: DNG and badpixel.bin questions
« Reply #3 on: 25 / August / 2009, 18:54:03 »
Quote from: thunderstorm on 25 / August / 2009, 18:36:21
I repeated the test and again got exacly two different bad pixel numbers: 22,391 and 11,408.
I understand both are saved in the firmware of the camera and each is used depending on the shooting conditions.
This seems to match what fe50 said: http://chdk.setepontos.com/index.php/topic,4085.0.html

I'm not sure how widely this has been confirmed by experiment. ISTR the SX1 behaves differently.
Quote
My questions are:
1. Are the badpixel tables are factory settings written specially for each camera ?
Almost certainly. Each sensor will have a different set of defects. There's a bunch of stuff about defect mapping and factory adjustment in the firmware.
Quote
2. If each badpixel table is used for different shooting conditions, by using just one table (for example the smaller number) could result in black dots in the DNG picture or any other artefacts ?
Yes.
Quote
Why not store both tables (or at least the biggest one you get)
See my sig. We get the "bad" values by looking for zeros in a raw, what the camera does internally to set them is not clear. I suppose we could run the badpixel script for N iterations and pick the biggest list. It would be better to just use the canon table(s), but AFAIK they have not been identified. Again see sig ;)
Quote
3. And question about the 12bit RAW:
I was reading the post about it as you linked me to. It clearly shows that only once in 4 value levels is used - showing it's 10bit A/D, so why the sensor delivers 12bit data? To waste memory and CPU time?  :)
Good question. I assume it's either cheaper somehow or makes some processing they do easier. Older powershots only store 10 bit raw, but perhaps the availability of faster memory makes it advantageous to do work in 12 bit for some reason.
Logged
Don't forget what the H stands for.
*

Offline thunderstorm

  • *
  • 46
Re: DNG and badpixel.bin questions
« Reply #4 on: 25 / August / 2009, 19:28:01 »
By saying "We get it from", what do you mean? You talk about the method of getting bad pixels by searching for "0" in a RAW file on the computer, or do you mean the badpixel.lua that gets the canon's table of bad-pixels?

And i didn't understand your signature :)
Logged
*

Offline reyalp

  • ******
  • 14003
Re: DNG and badpixel.bin questions
« Reply #5 on: 25 / August / 2009, 19:56:16 »
Quote from: thunderstorm on 25 / August / 2009, 19:28:01
or do you mean the badpixel.lua that gets the canon's table of bad-pixels?
The badpixel script creates the badpixel list by looking for 0 value pixels in the raw buffer (technically the C code does this, the script just requests it, but the effect is the same.)
Quote
And i didn't understand your signature :)
The H stands for hack. CHDK is made by poking at the guts of poorly understood, undocumented proprietary mush.
Logged
Don't forget what the H stands for.
*

Offline _AVP_

  • *
  • 6
Re: DNG and badpixel.bin questions
« Reply #6 on: 21 / July / 2013, 08:19:45 »
Sorry for necroposting, guys. it's better than creating a new topic.

I've got my SX40 with badpixel.bin file created successfully (despite 0 count of bad pixels..) anyways.

My question is can anyone modify script of creating this file? Because it makes OVERBRIGHT shots looking for BLACK pixels. But i have RED "hot" pixels with long exposure shots i'd like to get rid of.

Or at least is there any way of manually creation  badpixel.bin ?
Logged
*

Offline reyalp

  • ******
  • 14003
Re: DNG and badpixel.bin questions
« Reply #7 on: 21 / July / 2013, 14:53:21 »
Quote from: _AVP_ on 21 / July / 2013, 08:19:45
I've got my SX40 with badpixel.bin file created successfully (despite 0 count of bad pixels..) anyways.
If it found 0 bad pixels, then it wasn't successful. (edit: see phils comment below) In all recent versions of CHDK (including 1.1), badpixel.bin must be created with the in-camera menu item, not the badpixel script.
Quote
My question is can anyone modify script of creating this file? Because it makes OVERBRIGHT shots looking for BLACK pixels. But i have RED "hot" pixels with long exposure shots i'd like to get rid of.
As above, you should not use the script, because it won't work with any build released in the last few years....
Quote
Or at least is there any way of manually creation  badpixel.bin ?
Well, we certainly won't stop you from opening it with a hex editor ;)

As I explained at some length in this very thread, the badpixel.bin file only contains pixels *known to the canon firmware to be bad*. Hot pixels that the firmware doesn't know about aren't included. To manually mark bad pixels, you should use the other badpixel system http://chdk.wikia.com/wiki/Badpixel_removal
« Last Edit: 21 / July / 2013, 16:07:25 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline philmoz

  • *****
  • 3449
    • Photos
Re: DNG and badpixel.bin questions
« Reply #8 on: 21 / July / 2013, 15:52:16 »
Quote from: reyalp on 21 / July / 2013, 14:53:21
Quote from: _AVP_ on 21 / July / 2013, 08:19:45
I've got my SX40 with badpixel.bin file created successfully (despite 0 count of bad pixels..) anyways.
If it found 0 bad pixels, then it wasn't successful. In all recent versions of CHDK (including 1.1), badpixel.bin must be created with the in-camera menu item, not the badpixel script.

My SX40 also reports 0 bad pixels, as does my G1X.

Since it's highly unlikely the sensors actually have no dead pixels, I suspect Canon has moved their dead pixel handling earlier in the capture process so it's now being done before the capt_seq hook.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline _AVP_

  • *
  • 6
Re: DNG and badpixel.bin questions
« Reply #9 on: 22 / July / 2013, 12:34:17 »
reyalp, thanks. Bot all i found is a theoretical information with dead links. Script (badpixel.lua) and tools (compiled show_bad_b) were removed from servers.

So in this way solution isnt obvious to me.
« Last Edit: 22 / July / 2013, 12:44:52 by _AVP_ »
Logged
Pages: [1] 2   Go Up
« previous next »
 

Related Topics