Save RAW files in DNG format instead of RAW

ewavr

hotvedt
I have a problem with "show_bad_b.exe", it seems to always give "Total: 6069.".
I've just tested on 3 raw-files, one 1sec, one 10sec and one 64sec exposure.

10 sec and 64 sec is too big exposure time for this, maybe 1/2 - 1/5 sec is optimal.

All three badpixel.bin's is identical and 24276 bytes big.
Is there some bug in this program, or do i have way to many bad pixels?

This is typical value for 1/2.5" matrix. For 1/1.8" and bigger maybe typical value is ~700 (but statiscs is too small).

hotvedt

Yes, thanks for the quick reply.
I've used an 1/8 exposure now, and it works.

whim

This is typical value for 1/2.5" matrix. For 1/1.8" and bigger maybe typical value is ~700 (but statiscs is too small).

All I get on A620 (1/1.7") is 1940 (badpixel.bin ~8k)

used mode M, ISO 50, 1 - 1/2 - 1/4 - 1/8 - 1/25 - 1/50 sec

since i assumed that something was wrong if i didn't get variation, i then tried changing the
"Bad pixel removal" option ( off / RAWcon / Average ), but that had no effect, either ...

So does this mean my cam maintains only 1 bad pixel list ?

wim

ewavr

whim
All I get on A620 (1/1.7") is 1940 (badpixel.bin ~8k)

size of badpixel.bin = number of bad pixels *4

since i assumed that something was wrong if i didn't get variation, i then tried changing the
"Bad pixel removal" option ( off / RAWcon / Average ), but that had no effect, either ...

Of course, this option is applicable only for user's badpixel list (badpixel.txt)

So does this mean my cam maintains only 1 bad pixel list ?

This question for A620 developers from Canon 😁. Maybe yes.
BTW, A710 have 2 such lists, SX100 -3, IXUS700 - only one (IXUS700 matrix=a620 matrix). Maybe this depends on maxtrix type.

hotvedt

Would it be best to average a bunch of raw-files, and use the result to make the badpixel.bin ?

MartinBudden

Minor ammendments to camera.h and dng_hdr.c

ewavr,

I've made a couple of minor ammendments to these files:

a) I've added an empty ImageDescription (0x10E) tag. This is a mandatory tag for TIFF/EP (it is allowed to be empty, however).

b) I've changed some of the hard-coded values in dng_hdr.c into #defines and put them in camera.h. This allows greater flexibility for future cameras. So, for example, although currently all CHDK supported cameras are 10 bits per pixel, there is no reason why Canon might not bring out a 12 bit per pixel camera. The CHDK code assumes 10 bits per pixel in a number of places but the fewer places that this is assumed the easier it will be to port to such a camera. The dng code does not need to make this assumption.

My code is at:

chdkmartin - Google Code

Martin

MartinBudden

Thoughts on file extension

I'd like to be able to set the file extension to .DNG even if this is not visible when the camera is attached by a USB cable - I always transfer photos by physically removing the SD card. Some other members on this forum have also indicated this.

Of course it would be nice if the file was also visible over USB, but I don't think this is essential.

Martin

binsurf

MartinBudden
Thoughts on file extension

I'd like to be able to set the file extension to .DNG even if this is not visible when the camera is attached by a USB cable - I always transfer photos by physically removing the SD card. Some other members on this forum have also indicated this.

Of course it would be nice if the file was also visible over USB, but I don't think this is essential.

Martin

I agree, it's not essential to have it visible regardless.

whim

After a suggestion by fe50, i'm currently working on support for BADPIXEL.BIN creation
in CardTricks for people with a command line phobia 😀

In order to make this work properly on as many cams as possible, i would like to suggest
to the user which RAWs he/she needs to shoot to get a proper badpixel.bin.
In some cams, there's only one internal badpixel list, which means any single CRW
file is enough to create badpixel.bin; in others you need several CRWs, see quotes:

ewavr wrote in #33 in this thread:

It creates in current folder output file badpixel.bin and reports about defect pixel count. Try several RAW files to obtain stable result (approximate shooting conditions: minimal ISO, exposure time ~1/10s). Some cameras have two defect pixel lists, which are switched at exposure time ~1/25 s (my cam has 7473 and 3607 defect pixels in each list). Use list with maximum (but stable) defect pixel count, otherwise you can see multiple black points on image in some RAW convertors.

... and in #203:

A710 have 2 such lists, SX100 -3, IXUS700 - only one (IXUS700 matrix=a620 matrix)

So would an SX100 owner be so kind as to share which shutter speeds they had cycle through to arrive at
a correct badpixel.bin ?

TIA,

wim

PS

ewavr

PhyrePhoX
about badpixel file creation: is there a chance we can do this in camera? shoot an "old format" raw, go to file browser -> "right click" create badpixel file. no?

Yes, this feature in my plans.
Hmm, six check-ins in one day? I don't have time to refresh my local copy 😁.

... would of course be a lot better !!!

cedricb

we need a cross-platform solution... so in camera is the best... :😀

binsurf

Is there a way to use Lua for generating a badpixel.bin file? Or is it not robust enough yet to handle that?

ewavr

whim
So would an SX100 owner be so kind as to share which shutter speeds they had cycle through to arrive at
a correct badpixel.bin ?

On SX100IS: shutter speed 1/6 s. and longer - maximum bad pilxel count (at lowest ISO). But this is ISO dependent and may be temperature dependent.

... would of course be a lot better !!!

So, I started this work (bad pixels list creation in camera). Scan rate (using get_raw_pixel() function) is approximately 1 sec/Mp for digic3).

PhyrePhoX

considering it is a one-time thing only, this is pretty fast 😀

fudgey

PhyrePhoX
considering it is a one-time thing only, this is pretty fast 😀

Excuse me for being ignorant, but is this truly a one time thing? I mean, are we reading something Canon wrote to FLASH at a production test bench (which will gradually become more and more inaccurate due to new bad pixels), or something that will change when new bad pixels appear?

whim

@ewavr

So, I started this work (bad pixels list creation in camera). Scan rate (using get_raw_pixel() function) is approximately 1 sec/Mp for digic3).

Excellent ! I guess i'll stop work on doing the badpixel thingy in CardTricks ...
I tend to agree with PhyrePhox: time doesn't matter for a once-only process.

thanks,

wim

@fudgey
well... if you notice this in your DNG's, couldn't you just kill your badpixel,bin and have CHDK refresh it ?

PhyrePhoX

besides - one or two badpixels more or less - depending on the raw editing software you wont notice them anyways (for example adobe converter automatically detects and fixes bad pixels - Faststone Image Viewer does not).

ewavr

binsurf
Is there a way to use Lua for generating a badpixel.bin file?

So, I propose for this: small CHDK modification (diff attached) and lua script (also attached).
Tested on sx100is, produces same result as show_bad_b.exe

warning: this message is for developers only, script alone don't work properly with current CHDK.

diff.diff

1.88KB

badpixel.lua

382b

fudgey

whim
well... if you notice this in your DNG's, couldn't you just kill your badpixel,bin and have CHDK refresh it ?

Oh, I only asked because I was curious how this works and what's going on here. I don't really use RAW much at all and DNG is even worse because I don't have much software that eats it. I was under the impression that the old badpixels files for CRW interpolation have like four pixels. If we now have thousands, there's clearly a difference between the two.

To formulate the question better, are we creating badpixels lists by deducing things from photos or are we duplicating Canon's lists, or are we almost duplicating Canon's lists with some inaccuracies due to noise and new bad pixels?

If it's the first one, refreshing won't do squat, will it (unless the file has become corrupt)?

ewavr

fudgey
To formulate the question better, are we creating badpixels lists by deducing things from photos or are we duplicating Canon's lists, or are we almost duplicating Canon's lists with some inaccuracies due to noise and new bad pixels?

Badpixel.bin is attempt to duplicate Canon's list, because at present we cannot read it from camera's adjustment table.
So, because internal list can be refreshed only in service, creating badpixel.bin is one-time operation for most cameras.

reyalp

Rather than the lua script, how about checking if the users has a badpixel.bin, and creating it the first time the user takes a (raw or dng ?) shot without one ?

You could pop up a message when dng is enabled without one present.

You'd need to sanity check the exposure setting too maybe.

« Previous Page Next Page »