Save RAW files in DNG format instead of RAW

PhyrePhoX

there is a dcraw changelog? where is it? havent found it yet 😀

whim

@phox

goto Decoding raw digital photos in Linux and scroll or search for this line:
"This file contains the entire history of dcraw.c since its conception on February 23, 1997"

wim

PS just found it, not tried yet

ewavr

In #626 added visibility files with 'dng' extension via USB for A710 and SX100 (turn on option in misc. menu).
If this option is enabled, files with other extension cannot be transferred:
A710 - 'thm', 'dps'
SX100 - 'cr2'
Tesed in Windows XP and Ubuntu 8.10 explorers.
Do not try erase 'dng' files via USB! Camera or ignores it (a710) or crashes (sx100).

for develops: Menu string is 'hardcoded', because I cannot invent correct translation to English.

PhyrePhoX

judging by the adresses you used i guess you found them in ram? so, not possible to find via IDA?

ewavr

PhyrePhoX
judging by the adresses you used i guess you found them in ram? so, not possible to find via IDA?

Of course, I using IDA for this:
(a710)
some_f_for_dng = 0x89E8
0x89E8 + 0xFFF13950(canon_data_src from boot.c) - 0x1900 = 0xFFF1AA38 (from here pointer to function (sub_FFE4C390) copied to RAM address 0x89E8 at camera/CHDK startup).
second_ext_for_dng = 0x8A0C
0x8A0C + 0xFFF13950(canon_data_src from boot.c) - 0x1900 = 0xFFF1AA5C (from here pointer to '.THM' copied to RAM address 0x8A0C at camera/CHDK startup).

But no all extension can be replace by 'dng'. Easiest case if 'cr2' files are visible via USB (for a710 'cr2' files are not visible).

P.S. For example, for a710 list of file extensions which camera scans in different folders at USB connection, located at 0xFFE68580 (in ROM) - jpg, crw, avi, wav, mrk, dps, fir.
Second list of extensions which can be transfred via USB located at 0xFFF1AA48 (and copied to RAM at startup) - fir, mrk, jpg, crw, avi, thm, wav. You can see that these lists are different.

laetilodie

In the new firmware are you try to store DNG shutter opening time instead of time of starting DNG creation ?

ewavr

laetilodie
In the new firmware are you try to store DNG shutter opening time instead of time of starting DNG creation ?

Yes, I already implemented it, but found than Canon uses shutter closing time for jpeg exif :'(
Unfortunately shutter closing event (after image exposure and before dark frame exposure) currently not handled by CHDK.

edit: Now I can try to add exposure time to shutter opening time to obtaing correct exif time.

edit2: Done, but I am not sure that all cameras uses shutter closing time for JPEG EXIF. Can somebody test this assumption (with Tv=5-10 sec and dark frame=on)?

edit3: EXIF standard says (about DateTimeOriginal tag): "For a DSC the date and time the picture was taken are recorded."

laetilodie

I try with my S2is and DNG EXIF time is 10s after JPG EXIF time with Tv=10s and dark frame=on
Firmware 623

edit : Same result with the 625

ewavr

#627: more precise date/time in DNG.

laetilodie

Not working for me, always 10s between JPG and DNG with firmware 627.

fudgey

a570 100e:

All code very similar to a710. Extension support is the same as for a710:
scan: jpg, crw, avi, wav, mrk, dps, fir
transferable: fir, mrk, jpg, crw, avi, thm, wav

boot.c: long *canon_data_src = (void*)0xFFF2D320;
0xfff3216c - 0xFFF2D320 + 0x1900 = 0x4E4C + 0x1900 = 0x674C
0xfff32190 - 0xFFF2D320 + 0x1900 = 0x4E70 + 0x1900 = 0x6770

stubs:
DEF(some_f_for_dng, 0x674C)
DEF(second_ext_for_dng, 0x6770)

I haven't tested this yet, though. That's for thm like the a710.

ewavr, where does that 0x1900 come from? I found 0x674C by comparing code and it's good, but wouldn't know how to do it the other way around...

ewavr

laetilodie
Not working for me, always 10s between JPG and DNG with firmware 627.

You compare time of creation of files or EXIF data? Date/time in both EXIF must be equal.

ewavr

fudgey
ewavr, where does that 0x1900 come from?

From any boot.c, boot() function:

long *canon_data_dst = (void*)0x1900;

    for(i=0;i<canon_data_len/4;i++)
	canon_data_dst[i]=canon_data_src[i];

Initialized data segment in any VxWorks/DRYOS camera started at 0x1900.

laetilodie

ewavr

laetilodie
Not working for me, always 10s between JPG and DNG with firmware 627.

You compare time of creation of files or EXIF data? Date/time in both EXIF must be equal.

Sorry not 10s but 1s between EXIF JPG and EXIF DNG with Tv=10s and dark frame=on.
Same problem as other firmware.

PhyrePhoX

ewavr

PhyrePhoX
judging by the adresses you used i guess you found them in ram? so, not possible to find via IDA?

Of course, I using IDA for this:
...

okay, will try to follow this for s3is and a620.
i just added the badpixel.lua to the svn (see Changeset 629 - chdk - Trac )
should run on all cameras.
i noticed something: when you enabled dng and then later remove badpixel.bin (for example you just save the cfg, not the badpixel.bin when you change sdcard) you can still create dngs. shall we somehow check on startup for the existence of badpixel.bin?

ewavr

PhyrePhoX
when you enabled dng and then later remove badpixel.bin (for example you just save the cfg, not the badpixel.bin when you change sdcard) you can still create dngs.

Of course, because badpixel.bin is copied to memory.

shall we somehow check on startup for the existence of badpixel.bin?

If dng option is turned on and this file is not present at camera startup, dng option will be turned off automatically (see conf.c : conf_change_dng()).

PhyrePhoX

hm, i was sure i checked conf.c for it 😁
should have known better 😁

fudgey

fudgey
a570 100e:

stubs:
DEF(some_f_for_dng, 0x674C)
DEF(second_ext_for_dng, 0x6770)

I've tested the above now on a570 100e, it appears to work. I'm able list and transfer DNG files shot using CHDK on Ubuntu's default nautilus+libgphoto2 desktop tools.

As I said before, this is similar to a710 (disables .DPS and .THM). I also tested USB deletion and it does not work with a570 either, gphoto2 gives a clean error and camera does not crash. So just like a710 here as well.

a570 101a should work with the same exact stubs, I checked from the disassembly but obviously can't test it.

The setting is a bit out of place in MISC menu, though, I would've expected to find it in RAW menu along with the other DNG options.

ewavr

PhyrePhoX
okay, will try to follow this for s3is and a620.

So, in both cameras required stuff is absent 🙁.

fudgey
I've tested the above now on a570 100e, it appears to work. I'm able list and transfer DNG files shot using CHDK on Ubuntu's default nautilus+libgphoto2 desktop tools.

Good, I will search this functions for other cameras.

The setting is a bit out of place in MISC menu, though, I would've expected to find it in RAW menu along with the other DNG options.

We already have a million useful and useless options in RAW menu. Maybe we need DNG submenu in RAW menu.

PhyrePhoX

ewavr

PhyrePhoX
okay, will try to follow this for s3is and a620.

So, in both cameras required stuff is absent 🙁.

damn. on the other hand, i really never ever used usb to transfer files, so we should also add dng extension for cameras that dont support moving dngs via usb. and for those people who need the function that the associated files are deleted when you delete a jpg - well, we have to take a look at the raw purge feature again.

The setting is a bit out of place in MISC menu, though, I would've expected to find it in RAW menu along with the other DNG options.

We already have a million useful and useless options in RAW menu. Maybe we need DNG submenu in RAW menu.

yes, this is true. i'm sure we will find a solution 😀

« Previous Page Next Page »