Bad pixels - page 2 - RAW Shooting and Processing - CHDK Forum
« previous next »
Pages: 1 [2] 3   Go Down

Bad pixels

  • 28 Replies
  • 23506 Views
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Bad pixels
« Reply #10 on: 28 / December / 2014, 17:13:30 »
Advertisements
Quote from: srsa_4c on 28 / December / 2014, 16:59:26
I think those "dead pixels" are deliberately made black by the firmware (depending on several factors - exposure time, temperature, ...). They are marked so to notify the JPEG engine that they need to be corrected (by interpolation). And I also believe that the original pixel values are not necessarily wrong. Attached is a (magnified) corner of a wide angle raw image taken by a CMOS sensor camera. The dark border is left as-is by the image pre-processing pipeline, marked bad pixels only appear in the inner area that is also corrected for vignetting. (In case someone is wondering - the large coloured spots are the result of demosaicing.)

We save the RAW/DNG image pretty early in the capture process, before the normal Canon code has done much with the RAW image data. I'm not sure where the firmware would be setting these pixels, unless it's in the Digic processor.

I'm pretty sure Canon use a list of bad pixels to correct JPEG images - scanning the whole image takes a lot of time. It would be pretty easy to test - modify the raw capture code to set a block of known good pixels to 0 then see if they are interpolated out of the JPEG image.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline reyalp

  • ******
  • 14133
Re: Bad pixels
« Reply #11 on: 28 / December / 2014, 17:37:23 »
Quote from: philmoz on 28 / December / 2014, 17:13:30
It would be pretty easy to test - modify the raw capture code to set a block of known good pixels to 0 then see if they are interpolated out of the JPEG image.
They are (on some/most cameras at least), this is how the manual badpixel removal / badpixel.txt "RAWconv" option works.
edit: By "does" I mean the jpeg process fixes 0 value pixels. This is presumably done in fast digic hardware rather than a separate pass scanning the raw.

Also, you don't have to modify the code now, just use lua rawop.set_pixel  :D

edit:
Manual section: http://chdk.wikia.com/wiki/CHDK_User_Manual#Manual_bad_pixel_removal
Relevant but horribly out of date wiki page http://chdk.wikia.com/wiki/Badpixel_removal
« Last Edit: 28 / December / 2014, 17:40:47 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Bad pixels
« Reply #12 on: 28 / December / 2014, 18:13:56 »
Quote from: reyalp on 28 / December / 2014, 17:37:23
Quote from: philmoz on 28 / December / 2014, 17:13:30
It would be pretty easy to test - modify the raw capture code to set a block of known good pixels to 0 then see if they are interpolated out of the JPEG image.
They are
Confirmed on my ixus115.
Quote
, this is how the manual badpixel removal / badpixel.txt "RAWconv" option works.
I was not aware of that, even though it's in the manual...
Quote
By "does" I mean the jpeg process fixes 0 value pixels. This is presumably done in fast digic hardware
This isn't even hard to do, they just need to modify the pixels as they are received from the A/D converter. No interpolation is needed at that point, so they don't need to buffer image data.
Logged
*

Offline ursamajor

  • **
  • 64
  • SX150IS, SX510HS
    • Per aspera ad astra...
Re: Bad pixels
« Reply #13 on: 29 / December / 2014, 00:42:11 »
Quote from: philmoz on 28 / December / 2014, 16:27:38
It's better to use the bad pixel list detected at high ISO - the cleanup code will ignore pixels in image that are not 0 even though they are in the list.

Phil.
Thank you. This is good to know.
Logged
SX510HS, SX150IS (retired) :D
*

Offline lupo

  • *
  • 1
  • Product photography beginner with Canon S5IS
    • Outdoor equipment shop
Re: Bad pixels
« Reply #14 on: 03 / January / 2015, 06:06:35 »
Quote from: ursamajor on 27 / December / 2014, 09:27:16
Indeed, no bad pixels in jpegs. Only in DNG 1.3 with some RAW edit softwares as RAW Therapee, ufraw etc. No bad pixels with DNG 1.1, so I switched on v. 1.1. Anyway, Adobe lightroom made the corrections with both DNG versions 1.1 and 1.3.

Thank you very much for the info, yesterday I tested with DNG 1.3 and CHDK RAW formats with my Canon S5IS, but with the same black pixels in the RAW (only) pictures, but haven't tested with 1.1 ... started to think about a new camera .... the change to 1.1 format fixed the problem.
Thanks again!

Quote from: philmoz on 28 / December / 2014, 16:27:38
...
If you want to use the built in bad pixel interpolation in CHDK (DNG 1.1) with the bad pixel list detected at low ISO, then when you shoot at high ISO you may end up with pixels not being corrected. It's better to use the bad pixel list detected at high ISO - the cleanup code will ignore pixels in image that are not 0 even though they are in the list.
Phil.

Thank you very much Phil!
« Last Edit: 03 / January / 2015, 06:17:37 by lupo »
Logged
*

Offline ursamajor

  • **
  • 64
  • SX150IS, SX510HS
    • Per aspera ad astra...
Re: Bad pixels
« Reply #15 on: 03 / January / 2015, 08:02:40 »
You're welcome! Indeed, DNG v. 1.1 is more compatible with various editing software. Until now, as far as I tested, only Adobe Lightroom corrects bad pixels in both DNG versions. RAW Therapee, Ufraw, Photofiltre Studio, Irfan View etc. only works with DNG 1.1. I switched to 1.1, though I usualy use Lightroom to edit my RAWs. But RAW Therapee is also between my preferences as editing software... so, now I have more compatibility.

I would be interested, if possible, that you post the number of bad pixels that your sensors have and its type...  :)
Logged
SX510HS, SX150IS (retired) :D
*

Offline CanonCOMPACT

  • *
  • 8
Re: Bad pixels
« Reply #16 on: 08 / March / 2015, 08:41:47 »
Hi,

When I read this thread I understand my case faced with raw files explained below.

I have some photos taken with my Canon A590IS with CHDK 1.2.0 on it. I had recorded the raw files in .dng format.

When I open this .dng file with Corel Paint shop pro X7 I see all these bad/stack pixel when zoomed. On the other hand when I open the same file with Photo shop cs6 they do not appears.  I think Photo shop automaically remove them.

By the way, is there a way of testing the camera's bad pixel count before purchasing a new one on the shop ? While the chdk is not for all models, could there be single script file suitable for all models?

What is the limit of bad pixels guaranteed by manufacturer in general?

I ckecked  the "create bad pixel.bin" command from chdk menü and it created a bin file after taking a shoot or two. How do I know  the bad pixel count of my sensor hereafter?




« Last Edit: 08 / March / 2015, 09:11:25 by CanonCOMPACT »
Logged
*

Offline reyalp

  • ******
  • 14133
Re: Bad pixels
« Reply #17 on: 08 / March / 2015, 14:38:03 »
Quote from: CanonCOMPACT on 08 / March / 2015, 08:41:47
I have some photos taken with my Canon A590IS with CHDK 1.2.0 on it. I had recorded the raw files in .dng format.
DNG 1.1 or DNG 1.3?
Quote
When I open this .dng file with Corel Paint shop pro X7 I see all these bad/stack pixel when zoomed. On the other hand when I open the same file with Photo shop cs6 they do not appears.  I think Photo shop automaically remove them.
Adobe programs are typically able to correct badpixels in DNG 1.3 files, while many 3rd party programs do not. If you use DNG 1.1, the pixels are patched on the camera, so they should not appear in any program. The downside of 1.1. is that you need to create badpixel.bin and it uses some memory. There is also a very small increase in processing time.
Quote
By the way, is there a way of testing the camera's bad pixel count before purchasing a new one on the shop ? While the chdk is not for all models, could there be single script file suitable for all models?
No. CHDK requires a unique "build" for each cameras model, and there isn't another known way to count bad pixels.
Quote
What is the limit of bad pixels guaranteed by manufacturer in general?
I'm not aware of any specific limits. Canon presumably has some limits for their factory QC process.
Quote
I ckecked  the "create bad pixel.bin" command from chdk menü and it created a bin file after taking a shoot or two. How do I know  the bad pixel count of my sensor hereafter?
If I remember correctly, this process lists the number found when you create the file. There is no simple way to get the number form the file later. If you use DNG 1.3, you could get the number by using some software to count the number of 0 valued pixels.
Logged
Don't forget what the H stands for.
*

Offline CanonCOMPACT

  • *
  • 8
Re: Bad pixels
« Reply #18 on: 09 / March / 2015, 10:03:09 »
Quote from: reyalp on 08 / March / 2015, 14:38:03
Quote from: CanonCOMPACT on 08 / March / 2015, 08:41:47
I have some photos taken with my Canon A590IS with CHDK 1.2.0 on it. I had recorded the raw files in .dng format.
DNG 1.1 or DNG 1.3?
Quote

I think it was 1.3.  I have updated to chdk 1.3.0 yesterday and had removed the cfg file beforehand.. So I am not exact sure.

You are right that bad pixel count is displayed on the LCD just after processing of two shoots, so you should not miss it. It was around 5450 pixels . As I remember , my ISO setting was 200 before test , I learned that it could affect the the result sligthly.
Logged
*

Offline CanonCOMPACT

  • *
  • 8
Re: Bad pixels
« Reply #19 on: 15 / March / 2015, 08:42:49 »
Hello again,

I have chdk 1.3.0. on my canon a590IS. As mentione above, I see different behaviour at processing the bad pixels by Corel paint shop pro X7 and PS CS6 on raw shooting.

I created the bad pixel.bin file from the chdk menü after pointing the camera to a White paper. The file resides under the chdk folder, and set the " manuel bad pixel removal option to averege.  Set the file extension to. DNG (1.3). 

When I set  the settings as above , I presume that all the bad pixels outputs from the sensor will be replaced with the average of the surrounding ones and there will be no bad pixel data in the saved raw Picture ( regardless of whatever of file extension is selected , dng or not). Now,  I expect that if I open this raw dng file with PS or Corel PSP I should not see any bad pixel in the opened image.  The result has not changed , PS CS6 does not show any bad pizel but Corel PSP X7 does.   What is wrong in my understanding , if any?

Note : If the manuel bad pixel removal by averaging is done by camera and raw is saved in that way to SD card, is not it that  it is no more important whether corel PSP recognizes or not the dng file properly? There should not be any bad pixel in the saved raw data.

Logged
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal