New project: camera color profile calculation

DawMatt

Re: New project: camera color profile calculation

« Reply #200 on: 06 / June / 2008, 00:56:47 »

Advertisements

Quote from: lukg on 01 / June / 2008, 06:59:41

Thanks lks, if you have a real gretag color chart you can provide better results than everyone else. Keep trying, play with focus and zoom.
I have a question for the author or anyone knows...does this method depends on the palette provide in the first post?? because i tried with different color palette with horrible results (errors of about 100000%).

I asked Denis the same question a little while back. Apparently the process isn't tied to the Gretag Macbeth color checker, though I have yet to try calibrating against anything else.

I'm ordering a C1 IT8.7 calibration chart from here and am planning to use some other techniques to try to calculate the color matrix for my camera. Will be interesting to see how this impacts the quality of the DNG results.

Thanks,
Matt

Logged

DawMatt

Re: New project: camera color profile calculation

« Reply #201 on: 06 / June / 2008, 01:20:14 »

Quote from: lks on 01 / June / 2008, 21:14:42

I'm thinking part of the problem is that we need to also do a calibration for Tungsten light (Type A). I'm guessing that's why the colors shift a bit with different color light, even after correcting for the white balance.

This is likely to be part of the issue.

Denis requested us to use the daylight white balance, but I think what was intended was that we take the photo in daylight and use the daylight white balance. Offline he's indicated we should use the appropriate white balance setting for the light source rather than artificially set an inappropriate white balance setting.

Researching a little further, CIE states we should be using two different illuminants for colorimetry:

Standard illuminant A - "standard" indoor tungsten lighting
D65 - "Average daylight" @ 6500K, similar to what you would see around the middle of the day

These are the light sources used in camera matricies that DNG4PS-2 inherited from dcraw (17 and 21, respectively), so it makes sense that the second item is what Denis was referring to when he meant daylight. Unfortunately this type of lighting isn't that easy to come by unless you get some 6500K fluro tubes (with a CRI of 80+, preferably 90+). Purists wouldn't be satisfied with those but they are the closest that I can find to practical D65 lighting.

My calibration photos were taken under natural light or 5000K energy saving bulbs. From what I've read the CRI of energy saving bubls is ~0 so I wouldn't recommend this route, and I think my better results came from natural lighting.

Bottom line: even lighting across the whole screen is important, and ideally should be natural daylight. The camera should be manually (not automatically) set to the correct white balance and you should publish the lighting and white balance type along with the matrix..

Thanks,
Matt

PS I'm learning all of this color management stuff as I go along. Its amazing how much material there is out there about this!

Logged

DawMatt

Re: New project: camera color profile calculation

« Reply #202 on: 06 / June / 2008, 01:44:31 »

Hi,

Quote from: lks on 02 / June / 2008, 20:58:42

Now that I know it is possible to get 22 areas, I will try some other time to take a better calibration picture. My guess is that one has to push the exposure to the right enough so that the white patch is 245+ in intensity. I can just bracket a few exposures and see if any work better.

The image is underexposed on purpose. The reason is to minimise the amount of tone mapping performed by the camera when preparing the JPEG. If you try to expose the photo perfectly, the JPEG colors will be tone mapped which means comparisons to the RAW colors to be skewed. Would recommend you don't try to expose the photo perfectly or you will undermine the reason for taking the photo in the first place.

Thanks,
Matt

Logged

lukg

162
Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a

Re: New project: camera color profile calculation

« Reply #203 on: 06 / June / 2008, 06:40:51 »

Thanks Matt, what you think about my latest matrix with 24 patches found. It's the best matrix I used till now (still a blue dominance, but I noticed that in all my calibrations).
Anyway, are you saying that it is "impossible" (and useless) trying to get the DNG closer to the jpg (in normal situation) because of tone mapping?

« Last Edit: 06 / June / 2008, 07:26:12 by lukg »

Logged

DawMatt

Re: New project: camera color profile calculation

« Reply #204 on: 06 / June / 2008, 10:17:50 »

Quote from: lukg on 04 / June / 2008, 13:23:48

Ok, I did some crazy experiments. I took 10 pictures (-1.1/3exp) in continuos mode. Averaged RAWs on camera, and jpgs using Photomatix. This is the results of the first calibration (at 1.00 brighness):
...
All 24 patches found. That's a good start i think.

Interesting experiment. Not quite sure I'd trust the results for matrix generation though. Denis's original requirements for taking the calibration photo are design to minimise software interference with colors (i.e. underexpose to reduce JPEG tone mapping). Using software to average photos out goes against that philosophy at the very least, and depending on how the software works could upset the integrity of the matrix generated. I don't understand the math enough to know one way or the other though. Maybe I'm just being paranoid.

Quote from: lukg on 04 / June / 2008, 19:42:21

I have an idea for Artden,
...
having the possibility to choose a weight value to assing for each areas in the calibration process (like ACR script).
Here the screenshot of what i am talking about:ACR Calibrator L

I think that only selecting 22 of the 24 color patches is the closest the current version comes to implementing this technique. I've been trying to avoid changing the core matrix generation code (I don't understand it yet) so will leave it up to Denis to comment on the idea. But please keep in mind these are only approximate matricies. Generating a precise color matrix is considerably beyond the scope of the program, the Gretag Macbeth Color Checker, and many of the other variables part of our current process.

Quote from: lukg on 06 / June / 2008, 06:40:51

Thanks Matt, what you think about my latest matrix with 24 patches found. It's the best matrix I used till now (still a blue dominance, but I noticed that in all my calibrations).

As above, I think using averaged photos is dangerous. But as you've noted the results have a color dominance (cast) I think you already know that the matrix is better, but not quite ready for widespread usage. Have you considered changing the lighting around your monitor when taking the calibration shot? Maybe that is introducing a cast/reflections/etc?

Quote from: lukg on 06 / June / 2008, 06:40:51

Anyway, are you saying that it is "impossible" (and useless) trying to get the DNG closer to the jpg (in normal situation) because of tone mapping?

Not exactly. What I'm saying is that it is to be expected that there will be some variation between JPEG and RAW files. Canon will try to make the JPEGs look as good as possible because not everyone edits the photos afterwards. This means they will tone map, recover highlights, remove noise, sharpen - basically do a whole bunch of stuff that we would like better control over in Photoshop or other programs. With RAW we have the "negative" - the input before all of that processing occurred - so we can't expect it to look the same as Canon's process JPEGs (at least not until after we've worked on them in our graphics editor of choice).

So its useful for us to try to make the DNG look like the JPEG, to verify the correctness of the color matrix, but just as long as we keep in mind the Canon standard processing so we don't waste time trying to replicate that via color matrix tweaking.

Thanks,
Matt

Logged

lukg

162
Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a

Re: New project: camera color profile calculation

« Reply #205 on: 06 / June / 2008, 12:52:57 »

ok ok! Thanks Matt for your replies!
My averaging experiment was meant to reduce the overall standard deviation, since I saw that in calibration picture the noise was still there even at 80iso. The purpose was to obtain uniform colors and help the program in the calculation (since I think the program calculates mean in each area).
Averaging is not a real manipulation, since it applies a simply math mean to every pixel obtained from the 10 identical pictures taken (in the jpg and in the raw too of course).
I think this is a better way to take a calibration picture cause a mean is surely "more" realistic/reliable of a single value (noise is variable).
Since I suppose (I hope) that averaging is a "linear" operation, i don't think that this could influence negatively on the results, since it is applied on both raw and jpg in the same way. At the contrary it gives me very good results (compared to previous matrix).

Here a comparison between the DNG and the JPG of the camera (white point obtained picking up the bottom-left square)

I am sure that there will be a better matrix outside, but this work is driving me crazy

« Last Edit: 06 / June / 2008, 13:44:48 by lukg »

Logged

DawMatt

Re: New project: camera color profile calculation

« Reply #206 on: 06 / June / 2008, 18:24:51 »

Quote from: lukg on 06 / June / 2008, 12:52:57

Since I suppose (I hope) that averaging is a "linear" operation, i don't think that this could influence negatively on the results, since it is applied on both raw and jpg in the same way. At the contrary it gives me very good results (compared to previous matrix).

My biggest concern is that we aren't performing the same operation across both JPEG and RAW, because we are using different software to do both. We are taking it on faith that Photometrix (sp?) does exactly what we expect it to. It may be performing some other manipulation we aren't aware of. With the in-camera RAW averaging we have the source code available so can know exactly what it is doing.

The results look OK so maybe the end justifies the means, but I'm still think someone should check we aren't introducing suble issues into the matrix.

Thanks,
Matt

Logged

lukg

162
Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a

Re: New project: camera color profile calculation

« Reply #207 on: 07 / June / 2008, 06:16:01 »

mmm I understand what you mean....I don't know how CHDK does the average (maybe it's just a RAWsum/n.picture)

...but for the jpg I did it by hand in photoshop and compared with the one I did in photomatix. I think averaging is a simple "arithmetic/geometric/harmonic mean" operation, I hope it is a "standardized" operation

Quote

The results look OK so maybe the end justifies the means, but I'm still think someone should check we aren't introducing suble issues into the matrix.

I agree with you..for the moment I'll use my new matrix, since I have nothing better.

« Last Edit: 07 / June / 2008, 19:16:44 by lukg »

Logged

lks

16
[A570IS, S5IS]

Re: New project: camera color profile calculation

« Reply #208 on: 07 / June / 2008, 16:28:22 »

The new matrix is quite good in daylight conditions. Significantly better than any of the past matrices, including the 720is.

In lower temperature light (2800 to 4000 degrees) the colors and white balance of the new matrix get out of whack. I will try again to make a calibration under Tungsten light -- it hasn't worked well the first few times.

Logged

lukg

162
Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a

Re: New project: camera color profile calculation

« Reply #209 on: 07 / June / 2008, 19:12:35 »

Quote from: lks on 07 / June / 2008, 16:28:22

The new matrix is quite good in daylight conditions. Significantly better than any of the past matrices, including the 720is.

In lower temperature light (2800 to 4000 degrees) the colors and white balance of the new matrix get out of whack. I will try again to make a calibration under Tungsten light -- it hasn't worked well the first few times.

Glad that you find the matrix good! I agree that we need a matrix for lower temperature!

Logged