Save RAW files in DNG format instead of RAW

reyalp

MartinBudden
I would advise against averaging or summing dngs directly, presuming by dng you mean files containing the camera sensor output in mosaic (eg RGBG) form. This is because that, unless the images are precisely aligned, this is liable to produce false color artefacts.

Huh ? The the operation is performed on the individual mosaic components, and the final output is in the same mosaic format as the input. There is no opportunity for the wrong components to be combined. If the images aren't aligned, that will affect the output, just like any other stacking process without correct registration.

Note that the purpose of CHDK raw average is basically noise reduction.

cedricb

ewavr
2. no interest from users/developers.

... I've got a great interest about this feature for my S5, that's why I've originally point out these DNG tags... 😀

Ailurus fulgens

Hehe, i thought this is already in the trunk. I'm also interested in this, and i'm sure there are more people 😀

PhyrePhoX

yeah, noise reduction is the main advantage of in-chdk raw operation. the other raw operations are nice as well, of course.

PhyrePhoX

just tested on s3is. works wonders. i really would like to uncomment your code in the trunk right now, but i guess others should test as well.

this is great, a lot of my dngs up to now looked a bit weird when i manually tuned the WB, but now they look the same as the jpgs (color temperature wise). it's a shame i didnt notice the code was already there.

as always, good work ewavr. and of course thanks to howdy for pointing this out 😀

ewavr

PhyrePhoX
just tested on s3is. works wonders. i really would like to uncomment your code in the trunk right now, but i guess others should test as well.

Yes, it's interesting about cameras with digic3 propset.
Maybe this thread can be merged with "save dng ..." thread?

PhyrePhoX

merged it. though it isnt in order anymore, the merge function is a bit weird.

btw: Exiv2 - Image metadata library and tools
olympus uses two temperature sensors and stores these temperatures in exif. maybe we can use these adresses?

fudgey

I tried ewavr's DNG WB code on trunk 665 running on my a570is 1.00e (digic III, propset 2, vxworks). I shot to DNG in various wb settings (auto, manual, daylight and bulb) in P and M modes and a couple of ISO modes and opened them in gimp 2.6.1-1ubuntu3.

Compared to JPEGs, the colors are not exactly the same but they are certainly very close. From this quick test I'd say it works quite perfectly!

PhyrePhoX

update: digic III propset just tested (thx PlasmaHH in irc channel) - seems to work as well. so i guess we can "unlock" it in the trunk 😀

edit: added (uncommented 😁) in trunk.

ewavr

PhyrePhoX
edit: added (uncommented 😁) in trunk.

So, I found that WB settings also present in JPEG EXIF (in maker notes section):

DNG (for reading propertycase #100 values) :

dng_validate.exe -v DNG_9406.DNG

AsShotNeutral: 0.6039 1.0638 0.6309 (i.e. 1656(=1000/0.6039), 940, 1585 values in properycase #100).

JPEG EXIF (extracted by ShowExif):

dng_validate.exe -v -d 1024 IMG_9406.THM

Canon MakerNote: Offset = 678, Entries = 24
...<skip>...
*Tag13: 138 entries
...<skip>...
Long [91] = 940
Long [92] = 1656
Long [93] = 1585
Long [94] = 940

It is a pity that developmant of DNG4PS2 has been stopped :'(

PhyrePhoX

why do you say that? of course it is a pity, but with dng in chdk it isnt needed anymore (except for new cams to calculate color matrix)? no?

btw in ida you can find strings like colormatrix. shouldnt it be possible to get the colormatrix out of the binary?

radlpix

fudgey
Compared to JPEGs, the colors are not exactly the same but they are certainly very close. From this quick test I'd say it works quite perfectly!

Bringing the white balance into the DNG-files is a great improvement for CHDK! (This had to be said in my first posting ever here)

Concerning color differences between JPG and DNG it should be kept in mind, that the camera typically brings some color enhancement into the JPG even in myColors Neutral mode. And it depends on the RAW-converter that is used. For example I have the impression, that RawTherapee gives more saturation into the picture than UFraw. Nevertheless now the way to high quality color management is free and good color profiles for our cameras can be generated.

fudgey

PhyrePhoX
why do you say that? of course it is a pity, but with dng in chdk it isnt needed anymore (except for new cams to calculate color matrix)? no?

Shooting DNG is slower than CHDK CRW raw and requires 10's of kB's of RAM to function. It's more comfortable to use, but that aside it doesn't bring much to CRW+JPEG combination if (now that ewavr heroically found it!) WB would be dug from the JPEG makernotes tab by dng4ps-2.

PhyrePhoX

ok now i get it.
well, the slowness can be overcome when you set quality of the jpg to low. but we still need to find out that hook we can use/call after jpg is written ( https://chdk.kernreaktor.org/mantis/view.php?id=132 )

fudgey

PhyrePhoX
ok now i get it.
well, the slowness can be overcome when you set quality of the jpg to low.

No, that slows things down further unless you have a slow card. Scaling takes more time.

ewavr

PhyrePhoX
well, the slowness can be overcome when you set quality of the jpg to low.

No, because jpeg files are saved in background, raw files - in foreground.

fudgey
No, that slows things down further unless you have a slow card. Scaling takes more time.

I think that scaling time is near to zero (DSP part of digic is very fast).

PhyrePhoX

hm ok, so this feature wouldnt be useful for speed, but definitly for saving precious cardspace.
btw this background saving, couldnt this be used for the raws as well?
and have you read my post about the temperature (not color 😁) tags in exif? how difficult is it to add additional tags? as far as i understood exif, if your favourite image processing software doesnt know an exif tag, it is ignored. so in theory, we could add lots of tags - in theory.

fudgey

ewavr
I think that scaling time is near to zero (DSP part of digic is very fast).

JPEG continuos mode burst performs fastest in native resolution, slower when scaled down.

ewavr

PhyrePhoX
btw this background saving, couldnt this be used for the raws as well?

I don't know... But RAW files are 1.5-3 times bigger than jpeg with maximal resolution/quality, and RAM buffer for jpeg is not too big (~15 MB for A710, RAW file size ~9MB).

and have you read my post about the temperature (not color 😁) tags in exif? how difficult is it to add additional tags? as far as i understood exif, if your favourite image processing software doesnt know an exif tag, it is ignored. so in theory, we could add lots of tags - in theory.

We discussed this earlier. I don't know in which more or less standard tags this temperature can be stored (UserComment? DeviceSettingDescription? MakerNotes?) and in which form (binary/text). We cannot compel all software developers to read our non-standard tags.
p.s. You can read EXIF 2.2 standard, all EXIF tags listed on page 30 (printed page number is 24).

PhyrePhoX

well, olympus does this as well in the exif tags (they have sensor temperature and lens temperature in there somewhere, in dedicated tags).
well, i for one wouldnt be unhappy if it was added in the usercomment, because that field is a) empty as of now and b) every application reads that as default and c) we *could* extend this later with more stuff, or even maybe the user decides what non-standard info (like temperature, overrides etc) he would like to see in that usercomment, by the use of checkboxes). i dont know how hard it would be to add this, i tried once, but adding non-fixed values i couldnt achieve.

« Previous Page Next Page »