RAW subtract - Feature Requests - CHDK Forum
« previous next »
Pages: [1] 2   Go Down

RAW subtract

  • 14 Replies
  • 14486 Views
*

Offline DBgit

  • *
  • 14
RAW subtract
« on: 22 / July / 2008, 09:29:45 »
Advertisements
Hi,

I would like to have the possibility to perform manually a dark frame subtraction between two raw files already stored in camera. In other words, I want a "raw subtract" feature along with "raw merge" and "raw average".

Here is why I think it could be useful:
Let's say I take several long exposures with the automatic dark frame subtraction off. Later on, I could then manually take a few dark frames (with the lens cap on). I then merge all the raw exposures together in a new raw file and all the raw dark frames in another raw file with "raw merge".  With a "raw subtract" feature, one could then subtract these two raw files one from the other to get a high quality raw file with minimal noise!

The big advantage would be to perform the dark frame subtraction at the RAW level (i.e. before the demosaicing process). This can theoretically eliminate the presence of artifacts that can appear when the dark frame is subtracted after the demosaicing process. Another advantage would be to have a single raw file to post-process (and store on our HD) instead of two (or more).

I took a rapid look at "raw_merge.c" and it think this "raw subtract" feature could be implemented by simply changing a few "+="to "-=" in the 8 lines beginning with "row[i+XX] += ...". It might also require a few other minor changes like (i) limiting the number of raw files selected to two, (ii) adding back the BLACK_LEVEL instead of removing it, and (iii) a final verification to ensure that no intensity values are negative or exceeding 1023.

What do you think?
« Last Edit: 28 / July / 2008, 04:06:52 by PhyrePhoX »
Logged
*

Offline LjL

  • ****
  • 266
  • A720IS
Re: RAW subtract
« Reply #1 on: 22 / July / 2008, 14:08:20 »
This is related, though not in itself similar, to the various requests for automatic dark frame creation (search for "creation of darkframes").

But there's certainly no problem, as you say, implemention subtraction even without yet having automatic dark frame creation.

The best thing, I think, would be to just have a new option "Deferred" in the "Noise reduction" / "Dark frame subtraction" selection, which will do your shot(s) and then, later, shoot the same number of dark frame, with the same shutter speed, merge them, and subtract them from your shots.
Logged
*

Offline fe50

  • ******
  • 3152
  • IXUS50 & 860, SX10 Star WARs-Star RAWs
    • fe50
Re: RAW subtract
« Reply #2 on: 22 / July / 2008, 15:47:00 »
Wouldn't this (shooting a dark frame later) bring more noise ?
I thought that the intensity and the arrangement of the noise is more or less dependend from things like the temperature of "hot" parts near the sensor, so a dark frame shot under other circumstances (cold camera) couldn't bee as good for noise reduction as a "direct" DF shot, or do i misinterpreting this ?
Logged
*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: RAW subtract
« Reply #3 on: 22 / July / 2008, 17:03:38 »
Yes an older (or more in the future) dark frame is not as good as one shot right before or after each shot. Useful and probably better than nothing, but worse.

Btw, would it be a good idea (for long exposure shots, that is) to shoot a dark frame before and after each shot, and then do some cunning processing (or less cunning like averaging each pixel) to join the two dark frames so that the result would better represent the camera's state during the actual shot instead of before or after the shot? For instance would this further reduce purple fringing, some of which is always visible even after dark frame reduction in long exposures?
Logged
*

Offline LjL

  • ****
  • 266
  • A720IS
Re: RAW subtract
« Reply #4 on: 22 / July / 2008, 18:21:38 »
Sure, it's theoretically better to shoot a dark frame after (or even before, too) each actual shot.

But hey, I'm sure you see the convenience of shooting a one-hour-long exposure that actually lasts one hour, and then going back home while your camera shoots the dark frame. It's still generally much better than shooting them the following day, as far as temperatures go!

Besides, there are times when you just want to emulate a single, long exposure, and you really don't care for having a long gap after each exposure.

Also, don't be mistaken thinking that, if the dark frame is shot shortly after the exposure, then the noise will be more "similar" in the sense that the same pixels will have noise. It won't.
It will be similar on average, that's all that the temperature changes can modify! You're still not removing any noise (except for the hot pixels), you're just subtracting some spurious brightness from the image - on average.

Another interesting thing to mention is that a dark frame can be simulated if you have a bias frame. A bias frame is a frame that only records electronic noise from outside the sensor. If you have that and a dark frame for any exposure / temperature, you can approximate the "right" dark frame for your exposure time and temperature.
See Wikipedia about this.
Logged
*

Offline DBgit

  • *
  • 14
Re: RAW subtract
« Reply #5 on: 22 / July / 2008, 22:10:39 »
Logged
*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: RAW subtract
« Reply #6 on: 23 / July / 2008, 13:35:15 »
Ah yes amp glow, that's what I meant obviously.
Logged
*

Offline reyalp

  • ******
  • 14110
Re: RAW subtract
« Reply #7 on: 25 / July / 2008, 15:36:39 »
Dark frame control :D

Note that the cameras have temperature sensors (it uses that as part of the decision whether to do a raw subtract or not in the original firmware) so it may be possible to have it alert you to the need for a new dark frame. Or automatically create one whenever the temp or exposure settings change by more than a certain amount.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14110
Re: RAW subtract & temperature
« Reply #8 on: 27 / July / 2008, 19:59:26 »
Back to the original topic:
FWIW, I now have raw subtract working on the camera, and pre/post darkframe raw writing controlled by options and available from ubasic.

Still to do:
  • write a proper UI for raw subtract. I have in mind two options:
    • create darkframe. Takes the pre and post darkframe raws of a given exposure (which in my current code have the same number but different names) and creates a darkframe.
    • apply dark frame. Takes a darkframe, generated above, and subtracts it from a list of files.
  • clean up the mess mess in raw_savefile. It current does a lot of stuff that doesn't actually have to do with saving files, like bracketing, save first raw only and bad pixel removal. Calling raw_savefile twice makes this much worse. I have it working with those options (I think...), but it's a mess.
If any coders want to look at the code, I'm happy to post a diff, but it's pretty raw  :-[ at the moment so unless someone wants to work on it, I'll hold off until it's a bit nicer.

If anyone has feedback on how they'd like it work, please speak up.
Logged
Don't forget what the H stands for.
*

Offline DBgit

  • *
  • 14
Re: RAW subtract & temperature
« Reply #9 on: 27 / July / 2008, 21:47:20 »
Quote from: reyalp on 27 / July / 2008, 19:59:26
FWIW, I now have raw subtract working on the camera, and pre/post darkframe raw writing controlled by options and available from ubasic.


Cool! Thanks for your work reyalp. I'll certainly take a look at it once you are ready.

The UI you suggest looks OK to me. However, it would be much better (and faster) if we could get a real control of the shutter to create directly a dark frame (i.e., without taking a pre and post raws of a given exposure...). If I remember correctly Dataghost already found a way to close (or open) the shutter during an exposure.

Anyway, if this type of control is not possible, your approach to create a dark frame is certainly useful. Of course, another solution would be to put the a cap in front of the lens!  :P
Logged
Pages: [1] 2   Go Up
« previous next »
 

Related Topics
SimplePortal 2.3.6 © 2008-2014, SimplePortal