DNG profiles : the end of RAW color problem? - page 2 - RAW Shooting and Processing - CHDK Forum  
« previous next »
Pages: 1 [2] 3   Go Down

DNG profiles : the end of RAW color problem?

  • 24 Replies
  • 17051 Views
*

Offline lukg

  • ***
  • 162
  • Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a
Re: DNG profiles : the end of RAW color problem?
« Reply #10 on: 23 / August / 2008, 06:46:43 »
Advertisements
Quote
Did LKS take photos of a real chart on you behalf?

He owns a real gretag chart.
I could have taken pics of my screen by myself if it wasn't so.  :P
« Last Edit: 23 / August / 2008, 06:48:18 by lukg »
Logged
*

Offline DawMatt

  • *
  • 36
  • Canon IXUS 860 IS
Re: DNG profiles : the end of RAW color problem?
« Reply #11 on: 23 / August / 2008, 07:45:47 »
Are you trying to produce a matrix free DNG in the hope it won't interfere with the profile/recipe in ACR 4.5?  If that is the reason you asked the question, I think you will find the result would not change even if you could.  e.g. if you used DNG4PS2 to apply a completely incorrect color matrix, then used exiftool to reset the make/model info to the correct values, I'm sure ACR would still pump out the same output.

What lighting was used for the calibration photos?  If you read through the profile doco (and indeed most calibration standard documentation) you need shots taken under both Standard Illuminant A & D65 lighting to create a complete and accurate profile.  When my chart turns up I'll be experimenting with indoor lighting and direct sunlight (at different times in the day) to see if I can emulate the two lighting conditions.  D65 lighting seems fairly difficult to get hold of so will need to rely upon a white balance card/direct sunlight to verify when is the right time to take the shot.

So what exactly don't you like about your current profile?  The color checker doesn't look quite right when profiled, or that not all shots adjusted with the profile turn out with the color you were expecting?

Thanks,
Matt
Logged
*

Offline lukg

  • ***
  • 162
  • Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a
Re: DNG profiles : the end of RAW color problem?
« Reply #12 on: 23 / August / 2008, 08:42:05 »
Quote
Are you trying to produce a matrix free DNG in the hope it won't interfere with the profile/recipe in ACR 4.5?

Yes! That was my intention. I don't know if it's a stupid thing, if it is possible or not, I thought that if we have had the chance to skip the "imperfect" calibration in dng4ps, we could have obtain better results.

Quote
then used exiftool to reset the make/model info to the correct values

I'm afraid i didn't understand (i've never used exiftool). What I've seen is that, obviously, different matrixes give different outputs in camera raw. How could a matrix free DNG give the same output of a non-matrix free (which of them?), since different non-matrix free DNGs give different output?  :D :D :D :D
Another thing is that if the matrix is not 100% perfect, it can give different level of fidelity for different pictures. Maybe a picture is 95% good with that matrix, maybe another picture is 70%. A little error in RED calibration for eg. doesn't affect too much red color, but could have important consequences in other colors. Now consider what it would be if not only the red color were little wrong, but green and blue too.
Despite the fact we don't know if a matrix is good or not, by using DNG Profiles you will get an "almost" perfect camera profile, BUT, if the matrix is not 100% perfect, you will have a profile that is 100% good only for the picture you have used for the DNG profiling.
In other words you have an incorrect matrix (maybe good for the calibration picture used in DNG4ps calibration), and you have an incorrect profile (perfect but only for the picture used for DNG profiling).

Quote
What lighting was used for the calibration photos?

I don't remember, but I think he used "daylight" in the camera, or picking up the white balance directly from the chart (the second square in last row). Anyway, these are small errors that weigh far less than the errors of the approximated matrix where we started from.
However, using my matrix and the DNG profile in ACR (created in DNG Profiler), I get better results than without camera profile (specially the red channel).

Quote
So what exactly don't you like about your current profile? The color checker doesn't look quite right when profiled, or that not all shots adjusted with the profile turn out with the color you were expecting?

Now I like it very much, but that doesn't mean it is perfect  :D
The color checker (screen pic) after DNG4PS calibration was perfect (for my eyes). Some pictures too, a few others not.
The gretag chart (real pic) after DNG profiling was super perfect (for my eyes). Many pictures too, a very few others not.
In conclusion, I think that the camera profile is not 100% good, just because it is based on a not 100% good matrix, and this lead us to have an impredictable color fidelity.
Anyway, i'm using an Eos 450d these days so I shoudn't have color problems anymore; I gave my S5 to my father just to have some fun (no color problems for him, he doesn't even know what raw is).  :D

« Last Edit: 23 / August / 2008, 08:46:44 by lukg »
Logged
*

Offline vit40

  • ****
  • 276
Re: DNG profiles : the end of RAW color problem?
« Reply #13 on: 23 / August / 2008, 10:07:10 »
There is no need to have a DNG without color matrix. When you select a profile in ACR, a matrix in the profile is used, along with recipe (additional color adjustments using HSV lookup tables) and matrix in a dng is ignored. So calibration in DNG4PS(2) doesn't have any influence to the result

There were no comments on my "prebeta profiles" for A620. It's understandable - DNG4PS2 users didn't see them, as I made profiles using DNGs made with DNG4PS 1.14. But, as ranocchio observed, dng4ps 1.1.4 adds the string "PowerShotA620" into the dng, instead "PowerShot A620" (with space), so if you are using DNG4PS2, my profiles wouldn't appear in ACR

In the meantime, I tuned them to "beta 1 version" and switched from DNG4PS 1.14 to DNG4PS2.  I'm very satisfied with the final result - colors from A620 raws are VERY close to colors from 400D raws (both using Adobe beta 1 profile). Also, it turned out that Adobe recipe is very close to A620 in-camera color rendition - you just have to lower brightness from default value 50 to 25 and contrast from 25 to 20. And other thing - tag baseline exposure in dng should be set to zero. Unfortunately, there is no option to customize this tag in DNG4PS2 (value 50/100 = +0.5 EV is written), so I had to write a small program for changing this tag

On the other side, profiles for 400D, provided by Adobe, that should return the same result as 400D using picture styles Standard, Landscape etc ... are quite far from that. Well, I hope that they will do better job in beta 2, 3, 7 ... 314 ...
« Last Edit: 23 / August / 2008, 11:41:21 by vit40 »
Logged
*

Offline lukg

  • ***
  • 162
  • Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a
Re: DNG profiles : the end of RAW color problem?
« Reply #14 on: 23 / August / 2008, 12:46:20 »
Quote from: vit40 on 23 / August / 2008, 10:07:10
There is no need to have a DNG without color matrix. When you select a profile in ACR, a matrix in the profile is used, along with recipe (additional color adjustments using HSV lookup tables) and matrix in a dng is ignored. So calibration in DNG4PS(2) doesn't have any influence to the result

Yes, this is not the point.
What I experimented, as i remember well, is that when I made the profiling process, DNG profiler editor uses as starting point my DNG (with matrix and "almost corrected" colors) and it has calculated the ACR profile using the "manipulated" dng.
I tried to create a profile using 2 different DNG created from the same CRW (gretag chart pic).
One DNG created with my matrix. The second one created with the DNG4PS2 embedded matrix.
I obtained 2 different profiles.
Obviously the 2 gretag chart DNG, using their relative profiles, look identical. The other pictures don't. (the combination mymatrix+itsProfile gave me better results).
Matrixes are not considered by ACR anymore, but the Profile is. And it has been made using that matrix, so it is matrix dependent. In the end, you surely get a wonderful color fidelity with the calibrated and profiled gretag chart pic, but not so perfect for other pictures (it depends on how good was the matrix included in DNG that has been used for profiling).
Maybe I did something wrong, but this is what I experimented.
« Last Edit: 23 / August / 2008, 19:00:05 by lukg »
Logged
*

Offline vit40

  • ****
  • 276
Re: DNG profiles : the end of RAW color problem?
« Reply #15 on: 23 / August / 2008, 13:18:31 »
Yes, profile editor is another story. It uses matrix embeded in your DNG as a staring point (unless, I think, DNG has reference to some profile - in that case it uses matrix and recipe in that profile)

I calibrated these profiles this way (took several hours):

- made custom test image, consisting of Gretag table + 720 HSV combinations to check almost entire color space
- displayed it on monitor; 400D showed me 5350K, tint +10. It was ok for me
- made a photo of it with 400D and A620 with "exposure to the right", so that green channel was just at saturation level; needed a combination of dcraw and my program to check that exposure is ok
- converted both raws to dng, set baseline exposure to zero (it was +0.5 on A620 and -0.25 on 400D), to get comparable results
- moved positions of primary colors in A620 DNG with DngSetMatrix until I came so close, that all RGB values of Gretag macbeth were within tolerance about +/-1 (in range 0..255); on A620, lowering black level from 5 to 3 was needed
- using profile editor, opened 400D raw and exported Adobe standard beta 1 recipe
- opened A620 raw, opened that recipe and exported an "Adobe standard beta 1" profile for A620

Now, the fun begins. It turned out that color matrix for Adobe standard profile was different than a matrix for usage the old way (4.4). They recalibrated 400D in the meantime, so result from A620 and 400D, using this profile, was different, although not much. So a longer procedure was needed - move primary colors, make a profile, save a jpg, compare with 400D jpg and again and again ... until I got very close and was satisfied. In this case, green primary was slightly moved towards white pointe (so green is slightly less saturated), while red and blue were moved slightly anti-clockwise, so that R and B hue was about 2 degree higher

Also, it turned out that, for example, difference in saturation for macbeth green patch of only 1% results in error of almost 10% in some very green patch (0, 200, 0 for example), so it wasn't easy job to come close to correct calibration

That's it ...


« Last Edit: 23 / August / 2008, 13:20:05 by vit40 »
Logged
*

Offline DawMatt

  • *
  • 36
  • Canon IXUS 860 IS
Re: DNG profiles : the end of RAW color problem?
« Reply #16 on: 25 / August / 2008, 08:07:52 »
Hi,

I had expected the DNG Profiler to ignore the matrix in the camera, because building an accurate profile on a flawed foundation just doesn't make sense.  But maybe I was wrong.  Looking at the tutorial on Adobe Labs it doesn't seem that the profile is relative to the base matrix.  Will need to look into it when my chart arrives.

While you are at it look at tutorial 6 (at the same URL) which shows how to make a dual illuminant profile.  This is important because camera sensors respond differently dependent upon the lighting conditions.  Your issues with color might be due to one photo being taken in conditions similar to the calibration photo, and the less pleasing one being taken under markedly different lighting conditions.

Thanks,
Matt

PS It doesn't really matter if the camera was in Daylight white balance mode when the photo was taken.  What matters is if the light was actually "Daylight" (i.e. 6500K or close to it).  If you can get one chart photo, evenly lit, in this lighting and another at Standard Illuminant A (i.e. tungsten, 2850K, something like a normal household light bulb) then you will be well on your way to making a good profile for use with all your photos.
Logged
*

Offline DawMatt

  • *
  • 36
  • Canon IXUS 860 IS
Re: DNG profiles : the end of RAW color problem?
« Reply #17 on: 25 / August / 2008, 08:37:15 »
Quote from: vit40 on 23 / August / 2008, 13:18:31
I calibrated these profiles this way (took several hours):
Wow, that is pretty hard core.  I was hoping it would be a lot simpler than that.  Even knowing all the steps to try is pretty impressive.  How did you educate yourself regarding this level of device color calibration?
Logged
*

Offline vit40

  • ****
  • 276
Re: DNG profiles : the end of RAW color problem?
« Reply #18 on: 25 / August / 2008, 09:34:04 »
Well, with some basic knowledge of physics and programming, and reading some stuff on the internet. And some experimenting ... Anyway, this time was much shorter than time needed to develop CHDK (or even DNG4PS2), so my big thanks to all great programmers that spent so much time for this ... 

Of course, much shorter way would be to buy some color calibrator and a target (in my case) or to buy some DSLR (in case of CHDK developers) and/or camcorder ...  but more boring also ...
Logged
*

Offline lukg

  • ***
  • 162
  • Eos 450D+18-55is+55-250is & Powershot S5is - 1.01a
Re: DNG profiles : the end of RAW color problem?
« Reply #19 on: 25 / August / 2008, 12:21:44 »
Quote from: DawMatt on 25 / August / 2008, 08:07:52
it doesn't seem that the profile is relative to the base matrix.

I got 2 diffent profiles from two different "matrixed" DNG (both from the same CRW). Not only in theory, but they were also visibly different after the profiling progress (in the color list box).
Anyway, using dng4ps2 and adobe tool gives enough accurate colors.

« Last Edit: 25 / August / 2008, 13:30:15 by lukg »
Logged
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics