Cropping the sensor for DNG saving - page 2 - Feature Requests - CHDK Forum
« previous next »
Pages: 1 [2] 3   Go Down

Cropping the sensor for DNG saving

  • 26 Replies
  • 10302 Views
*

Offline reyalp

  • ******
  • 14039
Re: Cropping the sensor for DNG saving
« Reply #10 on: 15 / June / 2014, 16:13:26 »
Advertisements
Quote from: RaduP on 15 / June / 2014, 08:51:54
Out of curiosity, is there any criteria for which cameras have a second buffer and which do not?
Generally the ones with more physical RAM, which also tend to be the more expensive ones.

If you are thinking about optimizing DNG, you should figure out what gains are available before worrying too much about possible implementations.

Writing a full size 4104x3048 17.89 MB CHDK raw (full sensor, all one write, no reversing) on my D10 takes  ~3 sec on a slow card = 6 MB/s. The active area is 4072x3030 = 17.65 MB. So if you removed all the inactive area with no overhead, you would save writing 0.24 MB = 0.04 sec.

I would certainly take this if it were free, but in reality there would be some runtime cost to set it up, and also code complexity. It's not unlikely that the former would be a significant fraction of the savings.

Moving to a class 10 card would give me a much better gain, and the time saving available from skipping the inactive area would be correspondingly smaller.

A possibly better approach would be to use lossless compression on the data. The cameras that support native raw already do this, so there must be some way to do it quickly on the available hardware. In theory, this could be done in parallel with writing, as we do for reversing. Lossless compression should typically get you something like 15% reduction in data size, which should result in a corresponding saving in write time. However, even if it's theoretically possible, cameras without two raw buffers could still be problematic.

Another factor is that the masked area can actually be useful, because they give a measure of the black level for the specific exposure. The MaskedAreas DNG tag can be used to explicitly support this, though it would need to be set up for each camera.
Logged
Don't forget what the H stands for.
*

Offline RaduP

  • *****
  • 909
Re: Cropping the sensor for DNG saving
« Reply #11 on: 15 / June / 2014, 16:38:13 »
Well actually I was thinking that the biggest time gain would be from not having to crop the edges in post processing. I was thinking that the size/speed gain is secondary, but I didn't know about your changes, and I thought it was something trivial to implement. On cameras with two buffers it should still be relatively simple, no? But on the cameras with just one buffer, it's probably not worth the effort.
Logged
*

Offline reyalp

  • ******
  • 14039
Re: Cropping the sensor for DNG saving
« Reply #12 on: 15 / June / 2014, 17:09:01 »
Quote from: RaduP on 15 / June / 2014, 16:38:13
Well actually I was thinking that the biggest time gain would be from not having to crop the edges in post processing.
What cameras, and what are you using to process your DNGs?

If the port is set up correctly, the truly inactive parts of the DNG should never be shown in normal raw software. It is possible that some ports don't have the active area defined correctly.

There is some additional complication: On the real sensor, there is masked area (=no image data, should never be included in active area) some border area that has image data, but looks a bit different, and normal image area.

Generally, CHDK defines the active area as anything that has recognizable image data on it, which means that DNGs have dark borders. The reasoning is that you might want to recover these in a pinch, and they would generally be inaccessible in normal raw processing software if not included in the active area.

DNG also lets you specify a default crop, which some image processors use instead of the active area. In CHDK, this is the same size as the jpeg, centered in the active area.

If you want a different default crop or active area, it should be relatively easy to add a step to your workflow to use some exif tool to change them.
Quote
On cameras with two buffers it should still be relatively simple, no?
It would still complicate things, and as I said earlier, some people may want at least some of the masked area.
Logged
Don't forget what the H stands for.
*

Offline RaduP

  • *****
  • 909
Re: Cropping the sensor for DNG saving
« Reply #13 on: 15 / June / 2014, 17:27:44 »
I am using the SX510 HS, and UFRaw as well as Raw Therapee. The borders have absolutely no useful image data, and are dark.
Logged
*

Offline reyalp

  • ******
  • 14039
Re: Cropping the sensor for DNG saving
« Reply #14 on: 15 / June / 2014, 17:45:54 »
Quote from: RaduP on 15 / June / 2014, 17:27:44
I am using the SX510 HS, and UFRaw as well as Raw Therapee. The borders have absolutely no useful image data, and are dark.
If you upload a sample DNG somewhere, we can fix the active area.

edit:
The DNG should have a reasonably well exposed image, other than that content or settings don't matter.
« Last Edit: 15 / June / 2014, 17:49:32 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline RaduP

  • *****
  • 909
Re: Cropping the sensor for DNG saving
« Reply #15 on: 15 / June / 2014, 17:59:12 »
Logged
*

Offline reyalp

  • ******
  • 14039
Re: Cropping the sensor for DNG saving
« Reply #16 on: 15 / June / 2014, 19:48:23 »
Quote from: RaduP on 15 / June / 2014, 17:59:12
Thanks!

www.eternal-lands.com/CRW_0059.DNG
Thanks, I've updated this in trunk changeset 3463. It should be on the autobuild shortly. Please check that DNGs are still OK.

This port had both the default crop and active area set to the full sensor size, so you get very large borders before.

I set the active area using the procedure described at the end of https://www.assembla.com/spaces/chdkptp/wiki/DNG_Processing

I set the default crop to jpeg size from the canon specs http://www.usa.canon.com/cusa/consumer/products/cameras/digital_cameras/powershot_sx510_hs#Overview

Note that this is ~300 pixels shorter than the active area, so if your DNG software forces you to use the default crop, you will lose a bit of the image. error, see below.
« Last Edit: 15 / June / 2014, 20:29:53 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline RaduP

  • *****
  • 909
Re: Cropping the sensor for DNG saving
« Reply #17 on: 15 / June / 2014, 20:25:34 »
Quote
Note that this is ~300 pixels shorter than the active area, so if your DNG software forces you to use the default crop, you will lose a bit of the image.

What do you mean by 300 pixels?
The Canon specs is 4000x3000, and the DNG is 4160 by 3054 (as reported by Raw Therapee).
Logged
*

Offline reyalp

  • ******
  • 14039
Re: Cropping the sensor for DNG saving
« Reply #18 on: 15 / June / 2014, 20:26:57 »
Quote from: RaduP on 15 / June / 2014, 20:25:34
Quote
Note that this is ~300 pixels shorter than the active area, so if your DNG software forces you to use the default crop, you will lose a bit of the image.

What do you mean by 300 pixels?
The Canon specs is 4000x3000, and the DNG is 4160 by 3054 (as reported by Raw Therapee).
Whoops, I read the wrong spec. Fixing...

edit:
In changeset 3464.
« Last Edit: 15 / June / 2014, 20:29:32 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline RaduP

  • *****
  • 909
Re: Cropping the sensor for DNG saving
« Reply #19 on: 15 / June / 2014, 20:31:04 »
Thanks!
Is there any reason not to leave the default crop to the sensor useful data though? I can see this being a small problem at the wide angle, where there are some dark corners, but at full zoom it looks pretty ok.
Logged
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics