DNG meta data determining RAW converter crop behavior often of mediocre quality

koshy

1096

« on: 30 / August / 2014, 20:05:56 »

Advertisements

In working with RAW(DNG) data from all cameras currently on AutoBuild which don't shoot native RAW I found that the meta data determining RAW converter crop behavior often is of mediocre quality. This could mean one of two things:

a) That different physical cameras of the same model vary in this regard. That should be spot checked to confirm or rule out. To do that a few DNG files from other people's cameras should suffice. Ideally just shots of a more or less uniformly lit wall or up into the sky.

b) That the ActiveArea specifications for ~half of the CHDK camera models are just not accurate enough.

To go on with this let me elaborate a bit on the matter. Most facts will be known to the developers but to get a basis to discuss this I should still phrase it out. Let's look at an arbitrary example - I chose IXUS 870.

Code: [Select]

  | 8)  Model = Canon DIGITAL IXUS 870 IS
  |     - Tag 0x0110 (26 bytes, string[26])
  | | 18) DefaultCropOrigin = 10 10
  | |     - Tag 0xc61f (8 bytes, int32u[2])
  | | 19) DefaultCropSize = 3648 2736
  | |     - Tag 0xc620 (8 bytes, int32u[2])
  | | 20) ActiveArea = 8 14 2764 3682
  | |     - Tag 0xc68d (16 bytes, int32u[4])

This meta data determines the crop chose by the RAW converter. It's important to realize here that ActiveArea specifies pixels that show the scene and are not fully covered or partially shaded by covering material. No RAWConverter should show pixels outside of ActiveArea, they get cropped away on conversion. DefaultCropOrigin and DefaultCropSize however are just recommendations. Adobe products like PS Camera RAW and LR do adhere to that recommendation. Anything based on dcraw does not. Which is why an accurate ActiveArea is important to me and also an explanation for the poor quality of the ActiveArea specs on various CHDK cameras. When using Adobe's products 10px around the image get cropped beyond ActiveArea so it can be hard to get a good value.

I'm attaching two images. One is I870's full sensor's upper left and lower right corner, the other is the same for the current ActiveArea.

It can be seen that the ActiveArea is 2px too small on the left and right and 13px too small at the top. Evaluating the full sensor shows that it could be 4px larger towards the bottom.

The order of the rectangle coordinates for ActiveArea is: top, left, bottom, right.
Camera.h instructs that for ActiveArea:

Code: [Select]

X1, X2  sensor - needs to be divisible by 4
Y1, Y2 sensor - needs to be divisible by 2

So that needs to be taken into account.

With my above measurements ActiveArea for I870 should be (in DNG spec order) 21 16 2760 3680
With the Camera.h division requirements that becomes: 22 16 2760 3680

Or would it be better to say:

Code: [Select]

CAM_ACTIVE_AREA_Y1 = 22
CAM_ACTIVE_AREA_X1 = 16
CAM_ACTIVE_AREA_Y2 = 2760
CAM_ACTIVE_AREA_X2 = 3680

All this was verbose mode... What I'd like to do is spot check a couple of DNGs (of any cam you're willing to supply one of) to see if the fully masked pixels match precisely between two cameras of the same type. If they do I'd like to measure actual ActiveArea dimensions for all cameras where I see fully masked / shaded pixels in RAW conversions. Implementing such results in CHDK seems trivial enough but I'd like to know beforehand that there is interest to include these findings. Otherwise the endeavor would become pointless.

Edit: Changed one "RAW" into "RAW(DNG)"

I870_full_sensor.jpg (91.98 kB, 844x226 - viewed 112 times.)

I870_current_ActiveArea.jpg (89.08 kB, 844x226 - viewed 56 times.)

« Last Edit: 31 / August / 2014, 14:48:32 by koshy »

Logged

Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)

waterwingz

12138

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #1 on: 30 / August / 2014, 21:39:42 »

At the risk of being pedantic, I think you are interchanging the use of the terms RAW and DNG ?

CHDK RAW files are just binary dumps of pixel information as it comes from the sensor. There is no additional meta information and no concept of active areas.

I believe that what you are describing here is the four pieces of information embedded in the DNG header for the DNG file created around the RAW information? This information comes from four #defines in each camera's CHDK platform_camera.h file. E.G :

Code: [Select]

    #define CAM_ACTIVE_AREA_X1              12  
    #define CAM_ACTIVE_AREA_Y1              12
    #define CAM_ACTIVE_AREA_X2              4440
    #define CAM_ACTIVE_AREA_Y2              3334

There are at least a couple of forum threads about how those values are determined but it largely comes down to the care and diligence of the person who did the original port (i.e. quite variable) and their guess about which rows & columns in the sensor data were actually completely active.

So I think what you are asking is two things First of all, are the DNG file header active area values optimal? And second, are there variations in the sensor masking that will result in different "optimal" values.

Edit : this also opens up the same question for cameras that support native RAW when looking at the CHDK DNG files for those cameras

« Last Edit: 30 / August / 2014, 21:44:11 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #2 on: 30 / August / 2014, 23:35:52 »

Quote

It's important to realize here that ActiveArea specifies pixels that show the scene and are not fully covered or partially shaded by covering material.

My philosophy for CHDK has been to include all pixels that show recognizable image data, even if they aren't at the same level as the rest of the sensor.

My rational is
1) most people who really use raw will crop their shots anyway
2) Occasionally, salvaging those border pixels will be worthwhile
3) If the active area does not include them, then you can only access them by hacking the DNG header..
4) It's not always clear where the edge of the "good" pixels is.

Some discussion can be found in
http://chdk.setepontos.com/index.php?topic=11591.msg113570#msg113570

Note that in CHDK 1.3, you adjust the default crop per Phil's suggestion in that thread. The ability to set "full" may negate #3 above.

In general, the active area and default crop are is at the mercy of the person who did the port. Some of them are quite bad, and fixing the ones that are obvious wrong would be a good thing.

Logged

Don't forget what the H stands for.

koshy

1096

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #3 on: 31 / August / 2014, 16:29:29 »

Quote from: waterwingz on 30 / August / 2014, 21:39:42

At the risk of being pedantic, I think you are interchanging the use of the terms RAW and DNG?

I changed the one use of RAW I could find to which that would apply in the original post. As the subject and meta data implies this is about DNGs. I'm not interested "RAW" binary dumps of sensor data and didn't produce any.

Quote from: waterwingz on 30 / August / 2014, 21:39:42

I believe that what you are describing here is the four pieces of information embedded in the DNG header for the DNG file created around the RAW information?

There are more than the four. The four define the ActiveArea.
There are also DefaultCropOrigin and DefaultCropSize. The latter only get used by Adobe products like PS, PSE (Adobe Camera Raw for those) and LR.

Quote from: waterwingz on 30 / August / 2014, 21:39:42

So I think what you are asking is two things First of all, are the DNG file header active area values optimal? And second, are there variations in the sensor masking that will result in different "optimal" values.

I am not asking the first. I am saying that they are not optimal as about 50% of the cameras I have show fully black and/or partly shaded pixels within their ActiveAreas.
I am asking if there is interest to change that and indeed am asking if the second might conflict with that wish.

Quote from: waterwingz on 30 / August / 2014, 21:39:42

There are at least a couple of forum threads about how those values are determined but it largely comes down to the care and diligence of the person who did the original port (i.e. quite variable) and their guess about which rows & columns in the sensor data were actually completely active.

I am in the unique position to have DNG test data for the 112 models that don't natively shoot RAW readily available and I'm willing to check that. It'll take time and effort of course so I was interested to find out if it would be welcome / appreciated.

Quote from: waterwingz on 30 / August / 2014, 21:39:42

This also opens up the same question for cameras that support native RAW when looking at the CHDK DNG files for those cameras

Yes in general it would but I can't help with that at present time. I only have the native RAW data from those.

« Last Edit: 31 / August / 2014, 16:33:01 by koshy »

Logged

Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)

koshy

1096

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #4 on: 31 / August / 2014, 16:30:03 »

Quote from: reyalp on 30 / August / 2014, 23:35:52

Quote
It's important to realize here that ActiveArea specifies pixels that show the scene and are not fully covered or partially shaded by covering material.
My philosophy for CHDK has been to include all pixels that show recognizable image data, even if they aren't at the same level as the rest of the sensor.

As I was finding fully Black pixels inside the ActiveAreas that goal would need work, too. I don't agree on the matter of including recognizable image data, even if they aren't at the same level as the rest of the sensor but maybe that can be worked out. I'll write more on it below.

Quote from: reyalp on 30 / August / 2014, 23:35:52

1) most people who really use raw will crop their shots anyway

They might. I prefer not having to as most often I rigorously frame in camera but that's a matter of personal preference. In what I'm working on these borders can interfere so I'll deal with them one way or another (the other would be in the post processing chain).

Quote from: reyalp on 30 / August / 2014, 23:35:52

2) Occasionally, salvaging those border pixels will be worthwhile

I tried out of curiosity. I found varying exposure levels and it didn't work out well. We are talking about 10px or so out of usually several thousand. Cases where salvaging those border pixels will be worthwhile might exist but they'll be rare I'd reckon.

Quote from: reyalp on 30 / August / 2014, 23:35:52

3) If the active area does not include them, then you can only access them by hacking the DNG header..

You can always modify that header after exposure should you choose to. The question is what should the default behavior be.

Quote from: reyalp on 30 / August / 2014, 23:35:52

4) It's not always clear where the edge of the "good" pixels is.

Yes, as interpolation will have its part but one can look at non interpolated output which might help.

Quote from: reyalp on 30 / August / 2014, 23:35:52

Some discussion can be found in
http://chdk.setepontos.com/index.php?topic=11591.msg113570#msg113570

Note that in CHDK 1.3, you adjust the default crop per Phil's suggestion in that thread. The ability to set "full" may negate #3 above.

Thank you very much for this link. Very informative.

The ability to adjust this mostly only influences Adobe products and not other software based on dcraw. (Out of lack of interest I didn't check third parties I know not to be using dcraw code) Dcraw ignores DefaultCrop etc. So only changing it to "Full" which will change the ActiveArea, too would have an influence there.

Personally I'm not too interested in the DefaultCrop for aforementioned reasons but would like to point out that having it default to "JPEG" in CHDK 1.3 makes sense for one group of cameras and not for another. For those where you can actually get JPEG and DNG data to align pixel precisely it makes sense. For those which use lens designs that strongly favor field curvature over astigmatism as seems to be Canon's preference since a few years it does not make sense. For those you'll need all pixels you can get to have the largest area to later use lens correction software on. I don't know if ever someone made an authoritative list which cameras do use lens correction in post processing so here is one: All DIGIC IV and later cameras currently supported by CHDK with the exception of A1100, A2100, D10, IXUS 95, IXUS 100, IXUS 980, SX10, SX20, SX120 do use it. DIGIC III and DIGIC II cameras generally don't. Those that do should default to ActiveArea those that don't should default to JPEG in terms of crop.

Quote from: reyalp on 30 / August / 2014, 23:35:52

In general, the active area and default crop are is at the mercy of the person who did the port. Some of them are quite bad, and fixing the ones that are obvious wrong would be a good thing.

To conclude I could imagine doing the following:

- providing three sets of measurements
a) top, left, bottom, right fully masked (black pixels) on sensor
b) top, left, bottom, right partially shaded pixels not at the sensor's general exposure level
c) for those where it's relevant X, Y position of upper left JPEG corner on sensor

a) can be used to define ActiveArea according to reyalp's preference.
b) can be used to define it according to mine.

Since there already is a crop setting in CHDK we could have both (replacing current ActiveArea setting) where both settings would influence Active Area (as now does Full Sensor) and make DefaultCropOrigin 0,0 and Crop size the full ActiveArea. I'd use Active Area b for the JPEG setting (as a fall back where the RAW converter doesn't support the Crop meta tags)

Well, this is probably too long a reply as it is... So let's stop here.

« Last Edit: 31 / August / 2014, 16:38:05 by koshy »

Logged

Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)

reyalp

14079

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #5 on: 31 / August / 2014, 17:01:18 »

Quote from: koshy on 31 / August / 2014, 16:30:03

Quote from: reyalp on 30 / August / 2014, 23:35:52
2) Occasionally, salvaging those border pixels will be worthwhile
I tried out of curiosity. I found varying exposure levels and it didn't work out well. We are talking about 10px or so out of usually several thousand. Cases where salvaging those border pixels will be worthwhile might exist but they'll be rare I'd reckon.

Probably true. I wonder if you can reliably recover them with a flat field?

Quote

So only changing it to "Full" which will change the ActiveArea, too would have an influence there.

Right, what I meant was the "Full" allows you to access the dark borders. But you have to know in advance that you want them.

Quote

All DIGIC IV and later cameras currently supported by CHDK with the exception of A1100, A2100, D10, IXUS 95, IXUS 100, IXUS 980, SX10, SX20, SX120 do use it. DIGIC III and DIGIC II cameras generally don't. Those that do should default to ActiveArea those that don't should default to JPEG in terms of crop.

I think the capability may have been introduced in Digic IV, but it's possible that some lenses do not need it. I actually thought the D10 did this, but on double checking it appears not. The wide angle is quite distorted, but so is the jpeg.

Quote

To conclude I could imagine doing the following:

- providing three sets of measurements
a) top, left, bottom, right fully masked (black pixels) on sensor
b) top, left, bottom, right partially shaded pixels not at the sensor's general exposure level
c) for those where it's relevant X, Y position of upper left JPEG corner on sensor

a) can be used to define ActiveArea according to reyalp's preference.
b) can be used to define it according to mine.

Since there already is a crop setting in CHDK we could have both (replacing current ActiveArea setting) where both settings would influence Active Area (as now does Full Sensor) and make DefaultCropOrigin 0,0 and Crop size the full ActiveArea. I'd use Active Area b for the JPEG setting (as a fall back where the RAW converter doesn't support the Crop meta tags)

Well, this is probably too long a reply as it is... So let's stop here.

FWIW, my current procedure for setting the active area is described at the bottom of https://www.assembla.com/spaces/chdkptp/wiki/DNG_Processing

I wouldn't be averse to defining both a) and b) somehow, but up to now we've had trouble even getting the most basic definitions. Same for tuning the default crop vs jpeg.

With reasonably consistent exposure conditions, it's possible the masked and partially masked regions could be extracted automatically.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #6 on: 31 / August / 2014, 17:27:48 »

Just a quick note: the "partially shaded border" is much less shaded on images taken with zoom. It exists because vignetting correction is not applied to the border area.

Logged

koshy

1096

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #7 on: 31 / August / 2014, 20:17:27 »

Quote from: srsa_4c on 31 / August / 2014, 17:27:48

Just a quick note: the "partially shaded border" is much less shaded on images taken with zoom. It exists because vignetting correction is not applied to the border area.

Thank you very much for that quick note. It could explain what I found in my tests where the exposure level difference varied across the shaded strip. Revisiting the matter with using zoom one can see that zooming lessens the shaded pixels. This will require some thought.

Quote from: reyalp on 31 / August / 2014, 17:01:18

Quote from: koshy on 31 / August / 2014, 16:30:03
Quote from: reyalp on 30 / August / 2014, 23:35:52
2) Occasionally, salvaging those border pixels will be worthwhile
I tried out of curiosity. I found varying exposure levels and it didn't work out well. We are talking about 10px or so out of usually several thousand. Cases where salvaging those border pixels will be worthwhile might exist but they'll be rare I'd reckon.
Probably true. I wonder if you can reliably recover them with a flat field?

It would be quite tricky to recreate the "vignetting correction" on the outer pixels but theoretically feasible if you're really up to taking it that far...

Generally this is interesting. Yet another digital processing step in what's supposed to be "RAW". Do you know of others? I take it that this is on the other side of DIGIC and cannot be influenced by CHDK?

« Last Edit: 31 / August / 2014, 20:18:59 by koshy »

Logged

Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)

waterwingz

12138

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #8 on: 31 / August / 2014, 20:22:40 »

Quote from: koshy on 31 / August / 2014, 20:17:27

Generally this is interesting. Yet another digital processing step in what's supposed to be "RAW". Do you know of others? I take it that this is on the other side of DIGIC and cannot be influenced by CHDK?

Once again, did I miss something here? The RAW image file is every pixel available from the sensor. The DNG has the same information, but some "meta info" is added to point out which parts of the sensor data are actually likely to be useful. Nothing to do with CHDK here - it's all there. Just pick what part of the sensor data you want to work with in "post processing".

Logged

Ported : A1200 SD940 G10 Powershot N G16

koshy

1096

Re: DNG meta data determining RAW converter crop behavior often of mediocre quality

« Reply #9 on: 31 / August / 2014, 20:45:18 »

Quote from: waterwingz on 31 / August / 2014, 20:22:40

Once again, did I miss something here? The RAW image file is every pixel available from the sensor. The DNG has the same information, but some "meta info" is added to point out which parts of the sensor data are actually likely to be useful. Nothing to do with CHDK here - it's all there. Just pick what part of the sensor data you want to work with in "post processing".

The question is what one calls RAW. Strictly speaking the sensor is an analog device that turns photons into electrons. The electrons are stored in pixel wells and after exposure are converted into a digital signal. If an ISO has been dialed in the analog signal gets amplified prior to conversion. RAW as in "intensities as read by the sensor" is the output of the analog/digital converter. With the sensors in these cameras we find that it looks like a fixed value is set up as an intentional bias for "BlackLevel" which makes the output more uniform across different sensors.
Now srsa_4c implied that the sensor also applies a "vignetting correction" on its digital side (which isn't applied to border pixels, which in turn explains the shaded pixels we find in full sensor RAWs). That is what you might have missed. Such a "vignetting correction" would depend on focal lengh as a variable. It is part of what we get as a sensor dump and we cannot control it. Assuming it is done correctly it is a beneficial thing. Still it makes you wonder what else happens where you'd expect just plain "intensities as read by the sensor".

Logged

Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)