How to convert S95 CHDK CRW format to DNG

MartijnH

8

Re: How to convert S95 CHDK CRW format to DNG

« Reply #10 on: 02 / September / 2011, 12:17:14 »

Advertisements

The remark of Vit40 triggered me to do some research. From the DNG specification http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf and the CR2 specification http://lclevy.free.fr/cr2/ I understand that only two matrices for two illumination sources can be specified. I think that Dcraw supports two matrices, but the second one is rarely used. You can specify a second matrix in DNG4PS2, but it is not filled in for all predefined camera types.

As an experiment I took a picture of a Macbeth color checker on my computer screen and created three DNG files:
- with Adobe DNG converter from CR2
- internal with CHDK
- with DNG4PS2 from CHDK CRW

A difference for the DNG files I found out with Raw Therapee and Lightroom is:
- only CR2 DNG contains the Exif data to allow for automatic lens distortion correction and rescaling (with border removement) in for example Lightroom;
- the colors and as shot white balance for internal CHDK DNG were ok, but a slight exposure correction was needed;
- the as shot white balance for CHDK CRW had to be adjusted and a slight exposure correction was needed.

So, if you use CHDK raw (CRW or DNG) you loose automatic lens correction and may need some exposure correction. For CHDK CRW you may also need to adjust white balance.

For dive pictures I have no problem with adjusting white balance and exposure for every picture, but in the future I will use CR2 raw.

« Last Edit: 02 / September / 2011, 12:19:18 by MartijnH »

Logged

vit40

276

Re: How to convert S95 CHDK CRW format to DNG

« Reply #11 on: 02 / September / 2011, 13:05:07 »

Old dng specification used 2 color matrices for conversion. However, all adobe profiles in last years are up to dng 1.3, where color conversion is performed using two forward matrices, while two color matrices are used for WB. So there are 4 matrices in total in dng profiles. So it's not the same workflow as in dcraw which is using these matrices a bit incorrectly (whitebalancing is performed differently). However, ACR will work up to old dng specification and use only color matrices if there are no forward matrices in profile or in dng. I studied it in detail, since I needed that to make my dng profile generator program that is still unmatched by adobe, although lately they came quite close with "v3 profiles", as I documented on ACR forum several months ago.

I suppose all chdk dngs have baseline exposure tag = 0.5, while this value is generally different for each camera, hence the exposure difference between cr2 and chdk dng. You can check the value of this tag in a dng converted from cr2, using exiftool utility. With the same utility you can also change it, so results will be the same.

dngs made from crw have no WB info, since dng4ps2 doesn't know how to extract this info from jpeg, because it is in manufacturer info tag, and it's position depends on the camera model. A while ago I made an utility to copy this value from jpg into dng for my compacts. Without that, WB set to "as shot" is the same as WB Auto in ACR

You can use lens profiles with every kind of raw/dng files in ACR, just you have to select it manually - if you have it for your lens. But lens profiles provided by adobe so far seem to ... well ... need some refinement ...

« Last Edit: 02 / September / 2011, 13:09:26 by vit40 »

Logged

MartijnH

8

Re: How to convert S95 CHDK CRW format to DNG

« Reply #12 on: 03 / September / 2011, 04:59:14 »

You are absolutely right vit40!

With exiftool it is easy to derive the S95 Adobe color matrices from an S95 CR2 image converted with Adobe DNG converter 6.4. They are:

Color Matrix 1
Illuminant: Standard Light A
Matrix: 1.5419 -0.8268 -0.0232 -0.0656 0.8621 0.2437 0.0163 0.0404 0.5172

Color Matrix 2
Illuminant: D65
Matrix: 1.3440 -0.5896 -0.1279 -0.1236 0.9598 0.1931 -0.0180 0.1001 0.4651

Hence, the illuminant should be D65 and not daylight as I assumed.

It appears that it does not work to put both matrices in DNG4PS2 (the latest beta), Exiftool displays the use of illuminant 1 together with matrix 2, so you have to choose one of them.

The baseline exposure of CHDK DNG and DNG4PS2 is -0.5, while it should be 0 and a correction of 0.5 exposure is needed.

« Last Edit: 03 / September / 2011, 05:01:39 by MartijnH »

Logged

vit40

276

Re: How to convert S95 CHDK CRW format to DNG

« Reply #13 on: 05 / September / 2011, 16:14:03 »

In case there is only one color matrix in dng or dng profile, it doesn't matter what is the value of illuminant tag. Can be D65, Daylight, Tungsten, whatever. These tags are important only in case there are 2 color matrices, because in that case ACR will interpolate a color matrix from matrix 1 and 2, depending on color temperature set in ACR and values of illuminant tags

If you like canon colors, you can use camera profiles for S95 that are, I suppose, provided by Adobe. In that case it doesn't matter what color matrix is in a dng, because all 4 matrices will be taken from selected profile. Matrix from a dng will be used only in case you select "embedded" profile. After much experimenting, I concluded that Canon standard camera profile is better than simple matrix conversion in most situations for several reasons ... I suppose many would not agree with me and try to prove it with macbeth color checker that has several colors, but real life has many more colors so this checker and whole calibration stuff based on it is actually of limited use ...

Logged

MartijnH

8

Re: How to convert S95 CHDK CRW format to DNG

« Reply #14 on: 11 / September / 2011, 09:48:47 »

Thanks, you are again right Vit40. In practice it does not matter what illuminant is mentioned for one matrix, but it is stilll nice to hnow for which illuminant the matrix is measured and optimal.

I nearly never used the embedded colors, because I also liked de Adobe Standard colors in Lightroom. Thanks for explaining the difference. There is a small problem that Lightroom does not show you which camera profile is used. When the dng contains the right Exif tag for the camera (I assume), Lightroom uses automaticly its camera profile and names it Adobe Standard. The camera profiles, including for Canon Powershot S95, are in

C:\Documents and Settings\All Users\Application Data\Adobe\CameraRaw\CameraProfiles

For lensprofles Lightroom shows you which profile is chosen. The lensprofiles, including for Canon Powershot S95, are in

C:\Documents and Settings\All Users\Application Data\Adobe\CameraRaw\LensProfiles

Logged

vit40

276

Re: How to convert S95 CHDK CRW format to DNG

« Reply #15 on: 13 / September / 2011, 08:10:24 »

Quote from: MartijnH on 11 / September / 2011, 09:48:47

There is a small problem that Lightroom does not show you which camera profile is used. When the dng contains the right Exif tag for the camera (I assume), Lightroom uses automaticly its camera profile and names it Adobe Standard.

Really? I don't use LR, but according to this (old) webpage

http://www.thelightsright.com/UsingCameraProfilesWithLightroom2

you can select profile on calibration page like in ACR

Logged

MartijnH

8

Re: How to convert S95 CHDK CRW format to DNG

« Reply #16 on: 16 / September / 2011, 18:06:23 »

Well this has been changed in Lightroom 3

http://help.adobe.com/en_US/Lightroom/3.0/Using/WS939594D8-4279-41b4-B8E9-B06BC919EC7C.html

"Adobe Standard [...] The Profile pop-up menu displays only one Adobe Standard profile for your camera"

Logged

icameisaw

21

Re: How to convert S95 CHDK CRW format to DNG

« Reply #17 on: 12 / June / 2020, 19:49:02 »

I know it's been a while since anyone posted this, but when I try this, DNG4PS-2 will see the CRW and JPG files once I set up the camera, but when I try to convert the files to DNG, there is a stream of

Code: [Select]

Error: Can't find camera profile for this file
Z:\Shared\RAW\IMG_1071.CRW

messages and nothing is output.

Here's the export of the camera profile:

Code: [Select]

CameraCount=-1
[Camera000]
Id={11111111-11110000-08a420c3-32f90020}
Name=PowerShot S95
ShortName=S95
SensorWidth=3744
SensorHeight=2784
CroppedWidth=3648
CroppedHeight=2736
ActiveOriginX=64
ActiveOriginY=22
ActiveOriginRight=16
ActiveOriginBottom=2
BlackLevel=127
WhiteLevel=4095
Mosaic=1
BitsPerUnit=12
[Camera000/Matrix1]
Illuminant=21
Matrix=1.344000 -0.589600 -0.127900 -0.123600 0.959800 0.193100 -0.018000 -0.100100 0.465100
Mult=1.0 1.0 1.0
[Camera000/Matrix2]
Illuminant=-1
Matrix=0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
Mult=0.0 0.0 0.0

Is it the Id that's wrong? How on earth are you supposed to know what it is?

I've looked through the CHDK source code and can't spot any obvious Id in platform/S95. (I am now even more in awe of the people who worked out what was where in each camera though...) Searching for that string, or sensible bits of it, doesn't find it anywhere in the trunk tree.

If I look at some of the cameras DNG4PS-2 supports natively, and will export the settings for, the ID does vary, slightly:

A430 - Id={11111111-11110000-08a420c3-32f90026}
A450 - Id={11111111-11110000-08a420c3-32f9001E}

.. do I need to try every other couple of last two hex digits?

Logged

reyalp

14117

Re: How to convert S95 CHDK CRW format to DNG

« Reply #18 on: 12 / June / 2020, 22:13:21 »

It's been a very long time since I did anything with DNG4PS2, but I don't think that ID is anything you need to set or manipulate.

Under settings -> camera types you may need to set the slot that matches your cameras resolution to the correct camera.

Logged

Don't forget what the H stands for.

icameisaw

21

Re: How to convert S95 CHDK CRW format to DNG

« Reply #19 on: 13 / June / 2020, 04:31:59 »

Thanks - because the program had the S95 set up and I had JPGs for most of the files, I didn't think I needed to do that.

Once I did, it worked..

.. but only with the files that didn't have associated JPGs. The ones that did had the 'can't find camera profile' error, so something clearly does want setting up differently, because that's supposed to be helpful!

It turns out that the 'Name' has to be "Canon PowerShot S95" - rather than anything else, like "PowerShot S95" - presumably because that's the name in the JPGs.

On the off-chance that it might make a difference (being recognised as being from a known camera when something looks at the DNG rather than it going "'PowerShot S95'? I know about a 'Canon PowerShot S95', but this isn't that..") I've made that change.

Thank you again - I know this is something you typically only need to do once!

Logged