Save RAW files in DNG format instead of RAW - page 13 - Feature Requests - CHDK Forum

Save RAW files in DNG format instead of RAW

  • 392 Replies
  • 164506 Views
*

Offline PhyrePhoX

  • *****
  • 2254
  • make RAW not WAR
    • PhyreWorX
Re: Save RAW files in DNG format instead of RAW
« Reply #120 on: 19 / November / 2008, 20:34:10 »
Advertisements
yay, do it. i guess (hope?) development on this feature doesnt stop, once it is in the trunk :)

Re: Save RAW files in DNG format instead of RAW
« Reply #121 on: 20 / November / 2008, 02:09:19 »
Maybe if this version has no essential lacks, it can be added to trunk (after adding other IXUS/SD-series support) ?

I think it can be added to the trunk. A minor suggestion - can you merge camera_dng.h into camera.h? There's no run-time overhead to doing this and it simplifies updating the camera data (and adding new cameras).

Martin

Re: Save RAW files in DNG format instead of RAW
« Reply #122 on: 20 / November / 2008, 02:20:57 »
Now it works, but colors are not too good (but accectable for me ;))

Colors are good enough for me as well. However, for your interest, here's the full story (I omitted the color transform part when I explained the gamma because I thought you were doing monochrome thumbnails).

The output from the sensor is in a linear colorspace determined by the color filters on the sensor. sRGB is a non-linear colorspace that models the eye's response to light. In particular it models the color filters in the eye and the non-linear response of the eye. To get from sensor colorspace to sRGB you need to do a color transform and then apply the gamma correction. If the rgb output from the sensor is (X,Y,Z) and the linear sRGB colors are (RL,GL,BL) then:


[RL] = [              ] [X]
[GL] = [ color matrix ] [Y]
[BL] = [              ] [Z]


(Above denotes a matrix multiplication.) You then get from linear (RL,GL,BL) to sRGB by applying the gamma correction to each of the components (as described before).

Martin

*

Offline PhyrePhoX

  • *****
  • 2254
  • make RAW not WAR
    • PhyreWorX
Re: Save RAW files in DNG format instead of RAW
« Reply #123 on: 20 / November / 2008, 14:36:52 »
wow, i failed miserably getting temperature data into the exif. not as easy as it looks, but then again i am no coder :)


*

Offline vit40

  • ****
  • 276
Re: Save RAW files in DNG format instead of RAW
« Reply #124 on: 20 / November / 2008, 17:01:14 »
[ If the rgb output from the sensor is (X,Y,Z) and the linear sRGB colors are (RL,GL,BL) then:


[RL] = [              ] [X]
[GL] = [ color matrix ] [Y]
[BL] = [              ] [Z]



Well, slightly different. Output from sensor is obviously RGB, as there are R, G and B pixels, but it's not sRGB, because position of primary colors in CIE xy diagram are different, and another thing (as you noted), values are linear (proportional to amount of light). You need 2 matrix operations to get (linear) sRGB values - multiplying with inverted color matrix of the sensor to get XYZ values, and then with sRGB color matrix to get sRGB values. If you study the code in dcraw, you will see (besides color matrices for every camera, you will also find one sRGB matrix). After that comes gamma transformation to get non-linear, 8-bit sRGB values
« Last Edit: 20 / November / 2008, 17:03:33 by vit40 »

Re: Save RAW files in DNG format instead of RAW
« Reply #125 on: 20 / November / 2008, 19:58:41 »
Well, slightly different. Output from sensor is obviously RGB, as there are R, G and B pixels, but it's not sRGB, because position of primary colors in CIE xy diagram are different, and another thing (as you noted), values are linear (proportional to amount of light). You need 2 matrix operations to get (linear) sRGB values - multiplying with inverted color matrix of the sensor to get XYZ values, and then with sRGB color matrix to get sRGB values. If you study the code in dcraw, you will see (besides color matrices for every camera, you will also find one sRGB matrix). After that comes gamma transformation to get non-linear, 8-bit sRGB values

Agreed. My statement is a slight simplification of the full situation, but it was intended for explanation  rather than to implement the algorithm. And a transformation by matix A followed by matrix B is the same as a transformation by a matrix C, where C=B*A.

And I also omitted, for simplicity's sake, the color temperature transformation.

Martin

Re: Save RAW files in DNG format instead of RAW
« Reply #126 on: 20 / November / 2008, 20:10:57 »
wow, i failed miserably getting temperature data into the exif. not as easy as it looks, but then again i am no coder :)
Acutally, from a coding point of view, it is quite easy to include the temperature data. The problem is form a standards point of view: there are no tags defined by any standard (at least as far as I can find, and I have looked quite hard) that specify sensor temperature.

So the question is where to store the data. There are three main possibiliities:

a) the DNG private data tag
b) extend the DNG XMP tag to include sensor temperature
c) apply to Adode to add a new tag to the standard

Funnily enough I'm actually in favour of (c). Here's why:

i) The DNG private tag is really a place for the camera manufacturer to store proprietary information pertaining to the processing of the photo. The sensor temperature is generic information and does not effect how the sensor data is interpreted.

ii) Extending the XMP tag means including the XMP tag in the DNG file and then adding a field to store the sensor temperature. Well, if we are adding a field to the XMP standard, we might as well add a field to the DNG standard (both are owned by Adobe).

iii) Officially applying to Adobe for a new DNG tag acts as a sort of test case to establish CHDK as a legitimate player in the field. Sensor temperature is general purpose data that could be included by any camera manufacturer, and so asking Adobe to add a tag for this to the standard is a legitimate request.

So the question is, is there a recognized CHDK spokesperson, and could they apply to Adobe to ask for a new tag to be added to the standard?

Martin

*

Offline ewavr

  • ****
  • 1057
  • A710IS
Re: Save RAW files in DNG format instead of RAW
« Reply #127 on: 20 / November / 2008, 21:41:49 »
The problem is form a standards point of view: there are no tags defined by any standard (at least as far as I can find, and I have looked quite hard) that specify sensor temperature.

Yes, even in Canon maker notes sensor temperature is not found. But we can add MakerNote tag in own (maybe text) format.
« Last Edit: 20 / November / 2008, 21:48:33 by ewavr »


*

Offline PhyrePhoX

  • *****
  • 2254
  • make RAW not WAR
    • PhyreWorX
Re: Save RAW files in DNG format instead of RAW
« Reply #128 on: 21 / November / 2008, 01:16:09 »
Well its not only sensor temperature i'm after, also the optical mechanism temperature sensor data, because it quite accurately reflects real temperature. Also i'd like to see things like the 'real' camera orientation in it, as soon as it finds its way out of dataghosts experimental branch ;) also things like pethead adds in his timestamp mod,things like location,occasion and the like. Since this is subject to change,why not adding the info to the image description in ascii? x10E? And in chdk dng exif submenu you just tick off the things you want to have added in 'plaintext' to the image description, so later every gfx program can read it, things like "optical temp 24 / time since startup: 1-2-52 / etc"
I sure can apply to the dng board for adding that new tag, why not, but that will take ages for a) the dng board to decide and b) gfx app makers to react, if adobe acts in favour of us. Great work you have done so far, keep it up, thx!
Oh, i dont know if it is in there yet, a tag that shows if you were in bracketing mode and the number of the exposure. Canon does it in the jpg tags when i take bracketed shots with the original canon bracketing function of my s3is. Post production would get easier using this tag, imo :)
I'm sure there are alot more tags to add and to think of, but for now i'm running out of time, battery (cellphone :p) and ideas (brain didnt have coffee yet :p

*

Offline vit40

  • ****
  • 276
Re: Save RAW files in DNG format instead of RAW
« Reply #129 on: 21 / November / 2008, 02:31:15 »
Agreed. My statement is a slight simplification of the full situation, but it was intended for explanation  rather than to implement the algorithm. And a transformation by matix A followed by matrix B is the same as a transformation by a matrix C, where C=B*A.

And I also omitted, for simplicity's sake, the color temperature transformation.

Agree. I just didn't like the part where you stated " ... If the rgb output from the sensor is (X,Y,Z) ...", because values from sensor are not XYZ values (in CIE XYZ color space)

 

Related Topics