Histogram - page 11 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 6 7 8 9 10 [11]   Go Down

Histogram

  • 109 Replies
  • 25026 Views
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #100 on: 22 / January / 2013, 14:50:24 »
Advertisements
Just check the G11 DNG vs RAW (overexposed) and the same result, ie about 700 in the DNG at 0, but 0 in the RAW at 0!
Logged
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #101 on: 22 / January / 2013, 14:59:22 »
Quote from: pigeonhill on 22 / January / 2013, 14:14:13
The Canon RAW only shows the overexposed data, ie a spike at 4095 and no data anywhere else.
The DNG shows the spike at 4095 and a spike of about 700 at 0.
I'm assuming those are your 700 dead pixels that showed as colored dots in the dng image. Canon must be correcting them before saving them in its own raw format.
Quote from: philmoz on 22 / January / 2013, 14:12:00
Yes, they are dead pixels - when read out from the sensor the value returned is 0.
So a dead pixel is always 0, and a hot pixel is always 0x0FFF (12-bit camera)?

I always thought badpixel.bin was for hot pixels, but it sounds like it mainly removes dead pixels.

So if I went through the entire active area raw buffer looking for 0 pixels or isolated 0x0FFF pixels and set them to the neighboring (same color) pixel value (or average of its neighbors), would that solve the bad/hot pixel problem? I'm not saying it's worth doing, just trying to understand what's going on.

But it might be worth it if you're saving long exposure jpg only files with dark frames off and want to keep hot pixels out of the jpg. If I could get through the raw buffer in under 1 second, it would be an improvement over taking a 1 or more second dark frame after each shot.
Quote
recent CMOS cameras can have tens of thousands of dead pixels
Oh my! Maybe I'll test it by setting my histogram step size to 1 and then:
 print(get_histo_range(0,0,0))
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #102 on: 22 / January / 2013, 15:10:07 »
Quote from: pigeonhill on 22 / January / 2013, 14:32:59
BTW I had already tried the >4GB procedure you outlined, but tried it again.
The firmware update occurs and I make the card bootable in CHDK and set the tab to locked.
But all I get is a canon message saying 'card locked'.
So you got CHDK running with the firmware update? When you made the card bootable, did a popup window appear that said "finished"?

You have to have the file DISKBOOT.BIN in the root directory on a bootable card. Why don't you try putting my updated build on your card to make sure. You could just copy all the files from your smaller card that you know works over to the large card.

Then turn off the camera, take out the card, set write protect, put in the card, turn on the camera, and throw the camera against the wall if it says "card locked." It worked for me.
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #103 on: 22 / January / 2013, 15:29:59 »
Lapser

I give up, I've tried all options.

When I did the firmware update, it showed finished after I made the card bootable.

I tried your suggestion, ie transfer files.

All result in 'card locked'.

Maybe the S95 can not use greater than 4Gb CHDK, unless you go the multiple partition route: which I cant be bothered with.

Oh well, 4Gb Sandisk cards are cheap!!!

Cheers

Garry

PS I wont even both with the G11!!!
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: Histogram
« Reply #104 on: 22 / January / 2013, 15:57:12 »
Quote from: pigeonhill on 22 / January / 2013, 15:29:59
Maybe the S95 can not use greater than 4Gb CHDK, unless you go the multiple partition route: which I cant be bothered with.
The S95 was released in 2010 so it will only boot from a FAT16 formatted partition ( i.e.  4G or smaller ).
« Last Edit: 22 / January / 2013, 15:58:51 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline pigeonhill

  • *****
  • 1155
Re: Histogram
« Reply #105 on: 22 / January / 2013, 16:06:14 »
Thanks Waterwingz, I was beginning to think that was the issue. As I said, I will stick with 4Gb cards for my S95 and G11.

Cheers

Garry
Logged
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Histogram
« Reply #106 on: 22 / January / 2013, 17:23:24 »
Quote from: lapser on 22 / January / 2013, 14:59:22
I always thought badpixel.bin was for hot pixels, but it sounds like it mainly removes dead pixels.

So if I went through the entire active area raw buffer looking for 0 pixels or isolated 0x0FFF pixels and set them to the neighboring (same color) pixel value (or average of its neighbors), would that solve the bad/hot pixel problem? I'm not saying it's worth doing, just trying to understand what's going on.

CHDK can only fix dead (0 value) pixels.

It 'fixes' the 'bad' pixels by averaging the values from the nearest four pixels of the same color in the Bayer pattern (patch_bad_pixel in raw.c). Note this function will only fix a pixel if its value is 0.

There are two mechanisms for fixing bad pixels built in:

1. If saving DNG and using DNG version 1.1 it loads the list of bad pixels from the badpixel.bin file and fixes them just before saving the DNG file. The badpixel.bin file is created by searching two sample images for dead pixels and saving their co-ordinates to the file - it's a binary format.

2. You can manually create a file called badpixel or badpixel.txt that contains a list of X,Y co-ordinates to fix. These pixels are fixed in the call to patch_bad_pixels in raw_savefile if the 'bad pixel removal' option is enabled.

The reason CHDK uses the badpixel.bin/badpixel.txt files is that scanning the entire RAW buffer on every shot takes too long.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #107 on: 22 / January / 2013, 21:32:27 »
This is what I got when I sampled 100% of the pixels (step size 1). I did a dark and bright (totally white) shot. It looks like there are 24,288 dead pixels. "Hot pixels" in the dark frame shot are just a lot brighter than the surrounding pixels, but not all at white level. I've attached a small area of the dark frame shot that shows the hot pixels, and the lua script used to create the log. It took 11.5 seconds to create the histogram.
Code: [Select]
============================================================
Totally white shot of a bright light
60 seconds ISO 3200 SX260
============================================================
Tue Jan 22 17:38:56 2013
press half 32160
press full 32590
shot ready 104500
0 511 24288 **********************These are dead pixels
512 1023 0
1024 1535 0
1536 2047 0
2048 2559 0
2560 3071 0
3072 3583 0
3584 4095 12362592  ***************all other pixels very bright
    0 24288      4095 1628059
    1 0          4094 60
    2 0          4093 427700
    3 0          4092 62
    4 0          4091 582537
    5 0          4090 60
    6 0          4089 692629
    7 0          4088 70
    8 0          4087 781057
    9 0          4086 66
   10 0          4085 829938
   11 0          4084 64
   12 0          4083 842868
   13 0          4082 43
   14 0          4081 886493
   15 0          4080 39
   16 0          4079 933589
   17 0          4078 971279
   18 0          4077 19
   19 0          4076 3785960  **************all pixels are >=4076 except dead pixels
   20 0          4075 0
...
... rest is all 0's
...
  2386880 =npix nipx?= 12386880 4095 =topbin time= 11550

============================================================
Dark Frame shot (with some light leakage)
60 seconds ISO 3200 SX260
============================================================
Tue Jan 22 17:41:12 2013
press half 168840
press full 170280
shot ready 242310
0 511 12379636  ************almost all pixels are in the darkest bins
512 1023 6056   ************probably a little light leakage
1024 1535 696
1536 2047 135
2048 2559 115
2560 3071 67
3072 3583 28
3584 4095 147   ************most of these show up as colored dots, or hot pixels
    0 72286      4095 89
    1 15296      4094 0
    2 13167      4093 0
    3 12936      4092 0
    4 10993      4091 4
    5 10230      4090 0
    6 10469      4089 6
    7 11706      4088 0
    8 10273      4087 0
    9 10036      4086 0
   10 11038      4085 1
   11 12583      4084 0
   12 10614      4083 1
   13 10728      4082 19
   14 11352      4081 0
   15 14103      4080 0
   16 11173      4079 0
   17 11948      4078 0
   18 12723      4077 0
   19 16411      4076 0
   20 11933      4075 0
   21 13811      4074 0
   22 14482      4073 1
   23 18246      4072 0
   24 13099      4071 0
   25 15697      4070 0
   26 16880      4069 0
   27 21441      4068 0
   28 14012      4067 0
   29 17840      4066 0
   30 18990      4065 0
   31 25367      4064 0
   32 14905      4063 0
   33 21155      4062 0
   34 22095      4061 0
   35 30329      4060 0
   36 39734      4059 1
   37 45200      4058 0
   38 47958      4057 0
   39 63318      4056 0
   40 90361      4055 0
   41 49645      4054 0
   42 58352      4053 0
   43 83439      4052 0
   44 67285      4051 0
   45 47369      4050 0
   46 62994      4049 0
   47 91227      4048 0
   48 70370      4047 0
   49 47552      4046 0
   50 68705      4045 0
   51 98521      4044 0
   52 73098      4043 1
   53 46380      4042 0
   54 72929      4041 0
   55 104661     4040 0
   56 75157      4039 0
   57 43108      4038 0
   58 76468      4037 0
   59 109537     4036 0
   60 76369      4035 0
   61 39648      4034 0
   62 79952      4033 0
   63 98087      4032 0
   64 90772      4031 0
   65 38072      4030 0
   66 75544      4029 0
   67 109475     4028 0
   68 93132      4027 0
   69 35898      4026 0
   70 74871      4025 0
   71 117643     4024 0
   72 94369      4023 0
   73 32530      4022 0
   74 74426      4021 0
   75 125814     4020 0
   76 94709      4019 0
   77 28715      4018 0
   78 73223      4017 0
   79 132780     4016 0
   80 96751      4015 0
   81 25839      4014 0
   82 69994      4013 0
   83 140831     4012 0
   84 97866      4011 0
   85 23047      4010 1
   86 62941      4009 0
   87 150049     4008 0
   88 99421      4007 0
   89 20813      4006 0
   90 56208      4005 1
   91 158692     4004 0
   92 101187     4003 0
   93 17828      4002 0
   94 48245      4001 0
   95 166465     4000 0
   96 105678     3999 0
   97 14365      3998 0
   98 39618      3997 0
   99 174178     3996 0
  100 107878     3995 0
  101 11979      3994 0
  102 30933      3993 0
  103 179010     3992 0
  104 110326     3991 0
  105 10717      3990 0
  106 20821      3989 0
  107 180206     3988 0
  108 115760     3987 0
  109 9533       3986 0
  110 14937      3985 0
  111 175538     3984 0
  112 123248     3983 0
  113 8913       3982 0
  114 11215      3981 0
  115 163372     3980 0
  116 133507     3979 0
  117 8534       3978 0
  118 8648       3977 0
  119 137055     3976 0
  120 155136     3975 0
  121 8102       3974 0
  122 8291       3973 0
  123 73555      3972 0
  124 211810     3971 0
  125 7761       3970 0
  126 7676       3969 0
  127 7316       3968 0
  128 294710     3967 0
  129 7628       3966 0
  130 7350       3965 0
12386880 =npix nipx?= 12386880 4095 =topbin time= 11580
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Re: Histogram
« Reply #108 on: 22 / January / 2013, 21:51:39 »
Quote from: philmoz on 22 / January / 2013, 17:23:24
CHDK can only fix dead (0 value) pixels.
The reason CHDK uses the badpixel.bin/badpixel.txt files is that scanning the entire RAW buffer on every shot takes too long.
OK, thanks for the explanation. I'm more interested in correcting the "hot" pixels that show up in the jpg on my sx260 with long exposure night shots. I think Canon corrects the dead pixels when creating the jpg, doesn't it?

My testing in the last post showed that it takes 11.5 seconds to go through the raw buffer with get_raw_pixel() calls. Accessing the buffer more directly should speed things up a lot. I may have a go at it in the future.
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Histogram
« Reply #109 on: 22 / January / 2013, 22:03:01 »
Quote from: lapser on 22 / January / 2013, 21:51:39
I think Canon corrects the dead pixels when creating the jpg, doesn't it?

Yes.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
Pages: 1 ... 6 7 8 9 10 [11]   Go Up
« previous next »
 

Related Topics