Repository of DNG profiles?

DavidB

40

Repository of DNG profiles?

« on: 19 / August / 2010, 01:41:54 »

Advertisements

I've generated a custom DNG profile for my A470. It's a dual-illuminant profile generated from photos of a ColorChecker target under both D65 and StandardA illuminants, so should make a good profile to set as default for all DNGs from this camera (setting the defaults can be done within Adobe Camera Raw).
Now the colours from this camera match the colours produced by my other cameras, and when I switch back to the Embedded profile (i.e. the matrix saved by the camera) some of the colours (especially the blues) seem a bit washed out.

So I'm set, but would anyone else like a copy? Is there already an online repository where these things can be put for the benefit of those CHDK users who don't have their own profiling targets and software?

Logged

reyalp

14118

Re: Repository of DNG profiles?

« Reply #1 on: 19 / August / 2010, 01:56:10 »

You could start a page on the wiki, and host files on drop.io if needed.

If turn this into a matrix in the same format as found in http://tools.assembla.com/chdk/browser/trunk/include/camera.h you could also correct the embedded profile.

Logged

Don't forget what the H stands for.

DavidB

40

Re: Repository of DNG profiles?

« Reply #2 on: 19 / August / 2010, 02:12:59 »

Quote from: reyalp on 19 / August / 2010, 01:56:10

You could start a page on the wiki, and host files on drop.io if needed.

When I get some free time (ha!

) I'll look into that. Something simple I can do.

Quote

If turn this into a matrix in the same format as found in http://tools.assembla.com/chdk/browser/trunk/include/camera.h you could also correct the embedded profile.

Unfortunately it can't be simplified down to that form, although there may indeed be some opportunity to improve the matrix.
The dual-illuminant profiles actually contain two tables, one for each lighting condition, and the RAW processors interpolate between these based on the selected white balance (a very simplified explanation). So even if the default table in the code was improved, you should still be using a proper DNG profile for best results. And once you are, the embedded matrix doesn't matter. For unusual lighting conditions (e.g. low-pressure sodium streetlights) you're best off using a customised profile for those particular conditions, but the dual-illuminant profile is an excellent default.

Logged

DavidB

40

Re: Repository of DNG profiles?

« Reply #3 on: 20 / August / 2010, 12:34:26 »

Ok, I've put up a wiki page on DNG profiles: Calibrating colors for DNG processing.
Currently it only has a profile for the A470, so if you have any DNG profiles that you think are good enough for general use, please add them!

If any CHDK users are in Melbourne, Australia and would like to help me generate profiles for their camera, please drop me a line and we'll see what we can organise.

Logged

reyalp

14118

Re: Repository of DNG profiles?

« Reply #4 on: 01 / September / 2010, 23:53:32 »

From the DNG spec, it looks like we should be able to have dual illuminant profiles in the DNG for those cameras where someone has the equipment to generate them. Including them both in camera DNG would require some ifdefs or something in the DNG code, but doesn't look too hard. Possible ? Useful ?

I'm clueless about all this fancy color stuff

Logged

Don't forget what the H stands for.

DavidB

40

Re: Repository of DNG profiles?

« Reply #5 on: 02 / September / 2010, 00:15:53 »

The camera-generated profiles are currently DNG 1.1-spec. Profiles were added in 1.2, and the current version is 1.3. So there are a few things we need to tweak/add, but it should be possible.

The custom profiles I've been generating for my cameras are just under 3kB, which is a small overhead to put into the code.

Logged

DavidB

40

Please add profiles to the Wiki !

« Reply #6 on: 07 / September / 2010, 23:12:35 »

Just doing integration testing: the guts of the code are working. My cameras are each embedding a dual-illuminant profile into their DNG files!

The files still have color matrices, so DNG v1.1 processing software should see no difference (other than the matrices being derived from the camera's custom profile) but DNG 1.2+ processing software (e.g. ACR 4.5+, Lightroom 2+) get to use the profile automatically without having to install anything special.

Once I've done a bit more testing I'll package up the diffs for addition to the trunk.

Please, if you have a DNG profile for your camera which produces better colors than the simple color matrix that CHDK currently embeds, please add it to the Wiki profile page so they can be integrated into the CHDK distribution in the future.

Logged

vit40

276

Re: Repository of DNG profiles?

« Reply #7 on: 08 / September / 2010, 07:57:10 »

Are you planning to integrate profiles into code? I don't use dngs, because they additionally slow down the camera, but I think it would be good to put them into some folder on SD card, separately from the code, so users can change profiles without rebuilding the code. These profiles could be universal - contain only color matrices (one or both), like now, or forward matrices also, or also some of lookup tables like your profile

Anyway, I don't see a big advantage of having profile embedded in a dng. Only ACR and LR work with these profiles, and if you have them installed on the computer, it's easy to put appropriate profiles also. But maybe someone else will find it useful, it's up to personal taste

As about profile you attached, a took a brief look. So, in comparison with various profiles provided by Adobe, where difference between CM1 and CM2 is relatively small (in terms of position of primary colors in xy diagram), your profile has position of blue primary in daylight matrix very far outside xy diagram (while its position in tungsten matrix look quite ok). That's why blue colors are much more saturated. I tried it with my A650 (different sensor, but similar pixel size, so colors using As shot WB are quite good) and most real life colors including blue look quite ok, only very saturated blue on my test images are way oversaturated and too dark, so that result is even outside PhotoPro color space, although scene color is inside sRGB. Can't be different no matter which sensor, because of position of blue primary in that color matrix. But don't worry, even profiles provided by Adobe are faulty in blue range - there are currently only two sets of very last profiles (for 550D and D3) where they took the right approach (well, one thing still needs to be addressed), using only conversion with very big lookup table (90x16x16) and without matrix operation (in fact, with dummy forward matrix).

Real thing would be making a tool that would produce these profiles out of the camera. Idea is like this: put a special custom made CHDK raw file on SD card and develop it to jpeg using CHDK function. Then, comparing output and input, calculate a lookup table in the profile. Maybe I'll do this somewhere in the far future. I'm currently working on a calibration program for my 400D, because Camera profiles for 400D from Adobe are not very acurate, job is almost finished, so I have most of the code for this job, only time to finish this and my enthusiasm is running out ... Tried DNG profile editor first, but it's intended only to do slight modifications on existing profiles for existing cameras - it's useless if you want result that looks identical (or almost) to jpeg from camera (if that is the goal, of course)

« Last Edit: 08 / September / 2010, 07:59:28 by vit40 »

Logged

DavidB

40

Re: Repository of DNG profiles?

« Reply #8 on: 08 / September / 2010, 10:19:26 »

Quote from: vit40 on 08 / September / 2010, 07:57:10

Are you planning to integrate profiles into code? I don't use dngs, because they additionally slow down the camera, but I think it would be good to put them into some folder on SD card, separately from the code, so users can change profiles without rebuilding the code. These profiles could be universal - contain only color matrices (one or both), like now, or forward matrices also, or also some of lookup tables like your profile

Anyway, I don't see a big advantage of having profile embedded in a dng. Only ACR and LR work with these profiles, and if you have them installed on the computer, it's easy to put appropriate profiles also. But maybe someone else will find it useful, it's up to personal taste

Yes I am embedding the profile into the code. It's only about another 2-3 kB of data for each DNG file, and it falls back cleanly to DNG v1.1 behaviour (i.e. simple color matrices) if your processing software doesn't deal with profiles. Those matrices come from the profile: you'd just be missing out on things like the extra hue/sat mappings.

Currently I've taken the easy way out and am doing the "hard" work of merging the profile into the DNG file via pre-processing on the host rather than putting the .dcp file on the card and getting the in-camera code to do that (it's not just a case of copying the profile as a "chunk" into the DNG file). We might add in-camera integration in the future of course, especially as that will allow us to support per-individual-camera profiles.

Please see this thread for further discussion of the feature, and issues related to profile names.

Quote

As about profile you attached, a took a brief look. So, in comparison with various profiles provided by Adobe, where difference between CM1 and CM2 is relatively small (in terms of position of primary colors in xy diagram), your profile has position of blue primary in daylight matrix very far outside xy diagram (while its position in tungsten matrix look quite ok).

While the Adobe DNG Profile Editor only allows you to build a D65 ("daylight") and a Standard Light A ("tungsten") matrix, they don't have to be for these illuminants (the illuminant tags specify the scope of each matrix). But yes I do try to stick to illuminants similar to these for default profiles, usually generating them with the X-Rite software and sometimes tweaking them a bit with the Adobe software.

I've been happy with the real-world results from these profiles so far, but maybe more testing/tweaking is required?

Quote

Real thing would be making a tool that would produce these profiles out of the camera. Idea is like this: put a special custom made CHDK raw file on SD card and develop it to jpeg using CHDK function.

Step back a minute.... Is your goal to produce results that match the colors of the Canon-rendered sRGB JPEGs, or to produce results that match the colors from the RAW files from all your other cameras?

In my environment (I'm a professional photographer working in a color-managed environment, all the way out to print) I want colors that accurately represent the real world, and we have all our cameras producing consistent color. Each of our cameras (currently only less than 10 bodies, although we do have images from previous cameras in our library too: a combination of DSLRs, an EVIL, and some CHDK compacts) is individually profiled (although we only have one of each of the compacts and thus are able to treat them as per-model profiles for now).
These dual-illuminant profiles are just defaults: where possible we generate custom profiles for each lighting situation. Especially unusual artificial lighting (e.g. in architectural work where each room can be different) but also for outdoor lighting (e.g. I'll usually generate several profiles per day when in Antarctica, mainly because it's now so easy).
I must admit I have no particular interest in the colors that Canon decided their JPEGs should have.

Logged

fudgey

1705
a570is

Re: Repository of DNG profiles?

« Reply #9 on: 08 / September / 2010, 13:28:33 »

Quote from: DavidB on 08 / September / 2010, 10:19:26

Step back a minute.... Is your goal to produce results that match the colors of the Canon-rendered sRGB JPEGs, or to produce results that match the colors from the RAW files from all your other cameras?
...
I must admit I have no particular interest in the colors that Canon decided their JPEGs should have.

I'd be thrilled to have profiles/color matrices/whatever in DNG that closely match Canon's colors. I find them usually good enough, while RAW/DNG from the a570 I don't find good at all, AFAIK because our color matrix is not a good one.

If there would be an near-automatic tool to harvest Canon's colors, I'd probably be using DNG a whole lot more. Getting the matrix for would be an easy part of the porting process (if I'm not mistaken, there are ported cameras without a matrix because most people including me don't know how to author one properly or lack the tools for the task).

Getting the camera calibrated is taking it one step further and I'm probably not the only one who doesn't really need that but would still sometimes like to work with 10-bit colors on a lossless compressed image.

Logged