I tried some software and learned more about DNG, so, lets continue.
I'm trying to choose between DNG and CRW format for use with CHDK. Please correct me if I'm wrong somewhere.
Just to avoid confusion, "CRW" RAW in CHDK is just a dump of the sensor frame buffer. It is not related to the actual Canon CRW/CR2 which are propriety formats based on TIFF. This means that any software which uses CHDK RAW must be updated for every new sensor.
DNG can contains badpixel info, but RAW cannot. Desktop software can use DNG built-in badpixel info for correct output image. For CRW, desktop software can use non built-in badpixel file. So, result of DNG and CRW conversion can be the same.
Sort of. There are several complications.
There are two different categories of bad pixels: Pixels known to the firmware to be bad, and those that aren'tThe former are set to 0 by the canon firmware, while the latter will be "hot" or "stuck". The canon firm "fixes" the 0 value pixels when it creates a jpeg, but the 0 shows up in the raw data.
Further complicating this, most cameras have multiple Canon bad pixel maps, so which pixels are set 0 might depend on exposure length, ISO and sensor temperature. Additionally, some cameras handle them in a different way, such that no zero value pixels are seen in the raw.
DNG bad pixel handling depends on the DNG versionThe DNG 1.1 spec requires that "bad" pixels be interpolated over in the raw data.
In DNG 1.3, the header can just specify DNG opcodes which specify that pixels with a particular value should be interpolated over by the DNG software, but a lot of software doesn't support this.
CHDK DNGIn CHDK, the DNG code only directly addresses the 0 value known bad pixels, but there is a separate system you can use to handle hot pixels for both DNG and jpeg.
If you use DNG 1.1, then you must create badpixel.bin and bad pixels are interpolated over on the camera. badpixel.bin is purely a speed optimization, because we don't know how to directly access the Canon list and searching for 0 value pixels every time is too slow.
If you use DNG 1.3, then you do not need badpixel.bin and 0 value pixels are not modified on the camera. Your raw processor must understand DNG opcodes, or you must have some other way to tell it to interpolate over the affected pixels.
PTP remote shoot is differentSince I gather from other posts you are using PTP remote shoot, this deserves a mention. If you remote shoot with DNG, then the data is always 1.3 format, using opcodes. However, in chdkptp remoteshoot you can use the -badpix=n option to interpolate over pixels with a specific value. (note however that this depends stuff your java/chdkptp port may not implement)
CHDK has different systems for known (0 valued) bad pixels and hot pixelsAll of the above stuff about DNG refers to pixels the camera already knows are bad. The "badpixel" file can be used to specify an arbitrary list of pixels that should either be interpolated over
or set to 0, depending on the "manual bad pixel removal" setting.
Camera color profile is more complex thing. As far as I understand, not DNG nor the RAW contain color profile, i.e. processing depends on desktop software.
CHDK DNG do contain a color matrix, but the quality varies between ports and there no simple, documented process to make a good calibration (dng4ps2 can be used for this, but in practice it is difficult or impossible to get it to work on current cameras). If you come up with a better calibration, we can include it in the source.
The important part of choosing DNG/CRW - what software I need for convert DNG/CRW to well-known image formats like PNG, using badpixel info and color profiles. For my purpose, I need multiplatform software(preferable open-source) with ability to batch conversion. I.e. Photoshop and other Windows/Mac is not usable.
If dcraw + CHDK framebuffer dump RAW works for you, then use it. Most other programs that understand CHDK RAW are based on dcraw.
For DNG processing, I have had good results with raw therapee. You can create settings files to apply standard adjustments to things like color, hot / dead pixels etc, and output 8 or 16 bit png. AFAIK it can be used in to do batch conversions from the command line, but I have not used this.
There are also a lot of open source image processing libraries available (imagej, scipy + scikit-image, opencv...) I don't know off hand which ones might support DNG or bayer data.
chdkptp itself also has some limited DNG processing capability:
https://www.assembla.com/spaces/chdkptp/wiki/DNG_ProcessingP.S. I read http://chdk.wikia.com/wiki/Badpixel_removal, but could you confirm: as far as I understand, "badpixel file" used only for camera-side badpixels removing, but "badpixel.bin file" used only for desktop-side camera pixels removing.
Not really, but hopefully my comments above clear this up.
