Combining header and raw files.

Microfunguy

4037

Combining header and raw files.

« on: 26 / December / 2009, 17:13:59 »

Advertisements

I am being a little lazy because someone probably knows the answer and can respond quicker than I can find the solution.

I want to combine 1.hdr with each file *.raw to produce *.dng.

copy /b 1.hdr + *.raw *.dng does not work, it generates a single file.

Any ideas ?

Logged

ewavr

1057
A710IS

Re: Combining header and raw files.

« Reply #1 on: 26 / December / 2009, 17:26:34 »

for %%f in (*.raw) do copy /b 1.hdr + %%f %%~nf.dng
(inside .bat file)

Logged

Microfunguy

4037

Re: Combining header and raw files.

« Reply #2 on: 26 / December / 2009, 17:33:17 »

Thanks !

I would never have worked that out, works fine and the files open in dcraw-based programmes and CS4.

Logged

cybercom

268

Re: Combining header and raw files.

« Reply #3 on: 27 / December / 2009, 22:29:26 »

Quote from: ewavr on 26 / December / 2009, 17:26:34

for %%f in (*.raw) do copy /b 1.hdr + %%f %%~nf.dng
(inside .bat file)

@ewavr:

I still have absolutely no idea what anything you post here means, but I do know that it's coming from a genius!

Thanks for all your contributions over the years here.

Logged

~~~~~~~~~~~~~~~~~~~~~

<")%%%><<

Microfunguy

4037

Re: Combining header and raw files.

« Reply #4 on: 28 / December / 2009, 08:50:25 »

Quote from: cybercom on 27 / December / 2009, 22:29:26

I still have absolutely no idea what anything you post here means

For anyone else who is interested ... assign each file that has the raw extension to an 'internal' variable 'f'.

Do a binary copy, do not treat hex '1a' as an end-of-file character, just copy it.
Add the raw file to 1.hdr.

Take just the filename part of 'f' (not the extension) and add the DNG extension to it.

That is the name for the joined files.

Repeat for all other raw files.

Logged

reyalp

14111

Re: Combining header and raw files.

« Reply #5 on: 28 / December / 2009, 23:10:39 »

If you are making a DNG out of CHDK raw, you must also reverse the byte order somewhere.

Logged

Don't forget what the H stands for.

Microfunguy

4037

Re: Combining header and raw files.

« Reply #6 on: 29 / December / 2009, 08:08:18 »

Quote from: reyalp on 28 / December / 2009, 23:10:39

If you are making a DNG out of CHDK raw, you must also reverse the byte order somewhere.

Of course I must, and I have

Your assembler code makes it reasonably fast.

I do not reverse twice because the 640x480 lowest-quality JPG's will be discarded.

That has the added bonus of showing the area I have cut-out from the raw image.

David

« Last Edit: 29 / December / 2009, 08:13:08 by Microfunguy »

Logged

reyalp

14111

Re: Combining header and raw files.

« Reply #7 on: 29 / December / 2009, 20:06:28 »

Quote from: Microfunguy on 29 / December / 2009, 08:08:18

Of course I must, and I have

Hmm, then why save the header separately ?

Maybe OT, but in CHDK, I'd like to add a "fast DNG" option of some kind that saves the header + a CHDK raw (not byte reversed, not bad pixel patched). I don't want to generate an invalid DNG, so I thought about saving the header separately, but even in that case I think that making a new "not-dng" format (essentially, a new header + dng header + raw data) that is trivially convertible to DNG would make more sense. Less trouble to keep all the data together, and less stress on the canon filesystem code, which eventually falls down with too many image files.

The advantage of saving the header separately would be that you could fall back on tools that understand existing CHDK raw, but that doesn't apply to your cropped DNGs either, since most the CHDK raw tools use size to figure out which camera it is.

Logged

Don't forget what the H stands for.

ewavr

1057
A710IS

Re: Combining header and raw files.

« Reply #8 on: 30 / December / 2009, 02:44:45 »

Quote from: reyalp on 29 / December / 2009, 20:06:28

Hmm, then why save the header separately ?

Maybe header even not saved?

Quote

Maybe OT, but in CHDK, I'd like to add a "fast DNG" option of some kind that saves the header + a CHDK raw (not byte reversed, not bad pixel patched). I don't want to generate an invalid DNG, so I thought about saving the header separately, but even in that case I think that making a new "not-dng" format (essentially, a new header + dng header + raw data) that is trivially convertible to DNG would make more sense. Less trouble to keep all the data together, and less stress on the canon filesystem code, which eventually falls down with too many image files.

Personally I don't like idea to increase number of non-standard RAW formats.
Maybe we can increase speed of DNG saving:
-using some DNG 1.3 tags to avoid bad pixels patching. Unfortunately, dcraw still not supports such tags

.
-copying with reversin whole RAW buffer to another buffer to avoid double reversing. In this case RAW/DNG can be saved even after shoot. Example of "delayed raw" imlementation in my private build - http://trac.assembla.com/a710/browser/core/raw.c

« Last Edit: 30 / December / 2009, 02:49:50 by ewavr »

Logged

reyalp

14111

Re: Combining header and raw files.

« Reply #9 on: 30 / December / 2009, 03:27:58 »

Quote from: ewavr on 30 / December / 2009, 02:44:45

Personally I don't like idea to increase number of non-standard RAW formats.

Yes, I agree. But thinking about what would actually be easiest for me to use, this seemed like the least bad option.

If we made a documented standard CHDK raw header, it would make things easier for the tools that now guess based on file size.

Quote

Maybe we can increase speed of DNG saving:
-using some DNG 1.3 tags to avoid bad pixels patching. Unfortunately, dcraw still not supports such tags .

and bad pixel patching is only a small part of the time, most of it is reversing the bytes. Bigger benefit would be not needing badpixel.lua, and not having to worry about different canon badpixel tables.

Quote

-copying with reversin whole RAW buffer to another buffer to avoid double reversing. In this case RAW/DNG can be saved even after shoot. Example of "delayed raw" imlementation in my private build - http://trac.assembla.com/a710/browser/core/raw.c

Oh, that's quite interesting. How much memory can you allocate in the other memory areas before problems start ? The other thought (or possibly combined with this) is reversing one block, start writing, and reverse the next block while the first is writing. ISTR you tried this before and ran into problems, but the hardware must be capable of it ?

Logged

Don't forget what the H stands for.