Badpixel Mapping for Different Exposures - page 4 - Feature Requests - CHDK Forum
« previous next »
Pages: 1 2 3 [4] 5   Go Down

Badpixel Mapping for Different Exposures

  • 48 Replies
  • 25493 Views
*

Offline dzsemx

  • ****
  • 302
  • [SX40HS][A650][A630]
Re: Badpixel Mapping for Different Exposures
« Reply #30 on: 12 / July / 2008, 19:43:40 »
Advertisements
my A650 has some warm pixels too... :(
Logged
*

Offline NewbieToobie

  • ***
  • 102
Re: Badpixel Mapping for Different Exposures
« Reply #31 on: 20 / July / 2008, 20:43:26 »
Has this feature been added into that collaborative build yet?

I found a small utility that lets you analyze your photos for hot-pixels.

Dead Pixel Test

This would be a good way to see if you are filtering out regular background noise or true hot-pixels. It shows you a graph of the noise level and any spurious hot-pixels, as well as their locations. It only works on TIFs and JPGs, and Olympus RAW files.

edit: Is there a way that the show_bad.exe program could define a range of values instead of mapping out all dark-frame noise too? Example: show_bad raw_file.cr2 > badpixel 60 128

Where 60 is the lowest threshold letting you keep regular noise that has real detail in it, and above 128 is all kept too. Just the hot pixels with values from 60 to 128 will be mapped. Otherwise you might be mapping out useful pixels. They may be warm but still be defining a star at night or edge detail or something.

By the way, is that threshold setting for show_bad.exe a number from 0 to 255 or is that the 10-bit value going from 0 to 1023?

Nevermind. I didn't realize how hotpixels are mapped out and how that threshold value works. I now know that it leaves alone anything below the threshold level. The noise where useful data might be. It still doesn't work on my S3 camera but now I understand what it's supposed to do.
« Last Edit: 22 / July / 2008, 17:11:15 by NewbieToobie »
Logged
*

Offline NewbieToobie

  • ***
  • 102
Re: Badpixel Mapping for Different Exposures
« Reply #32 on: 20 / July / 2008, 22:23:39 »
I'm confused on the filename to use. If I do that property-case 69 value from the debug menu on a Digic II camera, then I get values like these

64.0" = 64960
50.8" = 64992
40.3" = 65024
32.0" = 65056
25.4" = 65088
20.0" = 65120
16.0" = 65152
12.7" = 65184
10.0" = 65216
8.0"  = 65248
6.3"  = 65280
5.0"  = 65312
4.0"  = 65344
3.2"  = 65376
2.5"  = 65408
2.0"  = 65440
1.6"  = 65472
1.3"  = 65504
1.0"  = 0
0.8"  = 32
0.6"  = 64
0.5"  = 96

But the values returned by get_tv96, get_user_tv96, and get_user_tv_id are totally different

examples

16"
get_tv96 = -384
get_user_tv96 = +384
get_uset_tv_id = -12

4"
get_tv96 = -192
get_user_tv96 = +192
get_uset_tv_id = -6

1"
get_tv96 = 0
get_user_tv96 = 0
get_uset_tv_id = 0

0.5"
get_tv96 = +96
get_user_tv96 = +96
get_uset_tv_id = 3


Which values should be used for the exposure time filenames?

Logged
*

Offline LjL

  • ****
  • 266
  • A720IS
Re: Badpixel Mapping for Different Exposures
« Reply #33 on: 22 / July / 2008, 13:55:56 »
You should use the values you get with get_tv96, as my code uses the shooting_get_tv96() function to determine the exposure time being used (note also that I've never tested it with CHDK overrides).

Besides, the differences you get (between PC 96 and get_tv96) aren't really differences: notice how the positive values remain the same, while it's only the values that are negative in get_tv96 that become positive. That's because the propcase viewer doesn't know whether a value is supposed to be signed or unsigned, and treats everything as unsigned. "65152" is just the same value as "-384" when considered as an unsigned short rather than a signed short.


Technicalities aside, anyway, I'll give a table of the hopefully correct values to use, since yours seem to be just the same as the ones I get in my A720 (aside that you have 16" instead of my 15"). This is from platform/a720/shooting.c in the trunk, and you can find the equivalent for each other supported camera.

The second column gives you the filename to use, the third column gives you the shutter speed in seconds.

    { -12, -384, "15", 15000000 },
    { -11, -352, "13", 13000000 },
    { -10, -320, "10", 10000000 },
    {  -9, -288, "8",   8000000 },
    {  -8, -256, "6",   6000000 },
    {  -7, -224, "5",   5000000 },
    {  -6, -192, "4",   4000000 },
    {  -5, -160, "3.2", 3200000 },
    {  -4, -128, "2.5", 2500000 },
    {  -3,  -96, "2",   2000000 },
    {  -2,  -64, "1.6", 1600000 },
    {  -1,  -32, "1.3", 1300000 },
    {   0,    0, "1",   1000000 },
    {   1,   32, "0.8",  800000 },
    {   2,   64, "0.6",  600000 },
    {   3,   96, "0.5",  500000 },
    {   4,  128, "0.4",  400000 },
    {   5,  160, "0.3",  300000 },
    {   6,  192, "1/4",  250000 },
    {   7,  224, "1/5",  200000 },
    {   8,  256, "1/6",  166667 },
    {   9,  288, "1/8",  125000 },
    {  10,  320, "1/10", 100000 },
    {  11,  352, "1/13",  76923 },
    {  12,  384, "1/15",  66667 },
    {  13,  416, "1/20",  50000 },
    {  14,  448, "1/25",  40000 },
    {  15,  480, "1/30",  33333 },
    {  16,  512, "1/40",  25000 },
    {  17,  544, "1/50",  20000 },
    {  18,  576, "1/60",  16667 },
    {  19,  608, "1/80",  12500 },
    {  20,  640, "1/100", 10000 },
    {  21,  672, "1/125",  8000 },
    {  22,  704, "1/160",  6250 },
    {  23,  736, "1/200",  5000 },
    {  24,  768, "1/250",  4000 },
    {  25,  800, "1/320",  3125 },
    {  26,  832, "1/400",  2500 },
    {  27,  864, "1/500",  2000 },
    {  28,  896, "1/640",  1563 },
    {  29,  928, "1/800",  1250 },
    {  30,  960, "1/1000", 1000 },
    {  31,  992, "1/1250",  800 },
    {  32, 1021, "1/1600",  625 },
    {  33, 1053, "1/2000",  500 },
Logged
*

Offline NewbieToobie

  • ***
  • 102
Re: Badpixel Mapping for Different Exposures
« Reply #34 on: 22 / July / 2008, 16:59:01 »
Thanks for the clarification. I just tested the get_tv96 values returned when using the overrides and it seems to work. I read them with a script that just does a "print get_tv96", after a half-press to set the override value. Here's the low-end override values/filenames missing from your list, in case anyone needs them.

64.0" = -576
50.8" = -544
40.3" = -512
32.0" = -480
25.4" = -448
20.0" = -416

But I can't seem to get badpixel to work no matter what. By putting these filenames in a subfolder CHDK/BADPIXEL/ or just using the badpixel.txt file in the main CHDK folder. I tried keeping the amount of badpixels found to less than 1024 lines thinking it might be rejecting larger lists of pixels to map out but that didn't help either. This feature must be not working at all on the S3 cameras. I'm using that collaborative build. It doesn't work with the Allbest build either when used the original way.

I also found another file on Ewavr's site that comes in handy for finding where to set the hotpixel threshold. By using this with your black-frame raw image it lists how many pixels in each photo are at what luminosity values. It's easy to see the huge bunch of noise at the top of the list, then anything after that will be warm and hot pixels. There's less time-consuming experimenting trying to find out what is noise and what are warm/hot pixels.

show_bad_stat.zip

« Last Edit: 22 / July / 2008, 17:07:20 by NewbieToobie »
Logged
*

Offline ewavr

  • ****
  • 1057
  • A710IS
Re: Badpixel Mapping for Different Exposures
« Reply #35 on: 22 / July / 2008, 17:24:30 »
Quote from: NewbieToobie on 22 / July / 2008, 16:59:01
I also found another file on Ewavr's site that comes in handy for finding where to set the hotpixel threshold.

Main purpose of this tool was to determine how many bad pixels was remapped in factory (pixel value=0). This count must be independent on shooting conditions (in theory, but in my a710 I have always two numbers: 7473 and/or 3607). On bigger CCD (1/1.8") this amount is essentially less (~1000 for A650IS, ~1200 for IXUS700).
« Last Edit: 22 / July / 2008, 17:29:32 by ewavr »
Logged
*

Offline NewbieToobie

  • ***
  • 102
Re: Badpixel Mapping for Different Exposures
« Reply #36 on: 22 / July / 2008, 17:42:54 »
Thank you for that information about using show_bad_stat.exe. I was just now using it to test some raw files and that value at 0 was always the same (3257). I was wondering why. Though I haven't used it with any images at shutter speeds faster than 16 seconds yet. I might get a different number at higher shutter speeds. I'm most concerned about bad pixels for long exposures. Thanks for providing such a valuable little utility.
Logged
*

Offline LjL

  • ****
  • 266
  • A720IS
Re: Badpixel Mapping for Different Exposures
« Reply #37 on: 22 / July / 2008, 18:14:09 »
You must put the files in a "BADPIXELS" (plural) directory that is not inside the "CHDK" directory. Just F:\BADPIXELS\number if you're using Windows and your SD card is F:.
Logged
*

Offline NewbieToobie

  • ***
  • 102
Re: Badpixel Mapping for Different Exposures
« Reply #38 on: 22 / July / 2008, 19:18:25 »
Quote from: LjL on 22 / July / 2008, 18:14:09
You must put the files in a "BADPIXELS" (plural) directory that is not inside the "CHDK" directory. Just F:\BADPIXELS\number if you're using Windows and your SD card is F:.

Thanks! That was the problem. It works now. NICE

But I find that it is really limited when using it for long exposures, when it is really needed. It seems to only process the badpixels at the top of the image. A limit in how many it can filter? My 64 second ISO400 exposure I filtered out 9374 very warm pixels when making my badpixels file, filename "-576". But only a small band at the top of the image is affected.

I guess I'll have to play with that threshold limit to only filter out a small percentage of the most offensive ones. A shame it can't filter out more.

But ... I'm GLAD IT WORKS!

Thanks!

And to let you know, it DOES work with override shutter speeds. But as already mentioned, since those long shutter speeds produce so many more warm pixels it only filters out some at the top of the image.

Logged
*

Offline LjL

  • ****
  • 266
  • A720IS
Re: Badpixel Mapping for Different Exposures
« Reply #39 on: 22 / July / 2008, 19:36:08 »
Yes, there is a limit on the length of the badpixels file to be loaded - 4096 bytes, it was there before my changes...

But are you sure you really want to filter out that many pixels? Try taking another 64 seconds ISO 400 exposure. Are the same pixels as before warm?

Otherwise, that's simply noise, it's random, and you don't want to filter it out like that.

Also, you'll likely have a whole lot of very bright pixels in the top left corner of the image, with such a long exposure. That's noise caused by something warm that's next to the sensor, it's not bad pixels. I'm not really sure how to filter that out properly.
Logged
Pages: 1 2 3 [4] 5   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal