Time for better folder naming for saving RAW/DNG ?

waterwingz

12138

« on: 21 / June / 2014, 12:32:02 »

Advertisements

Currently, there are two choices for where CHDK stores RAW/DNG images. They are saved either in the same folder as their associated JPG or in a seperate folder.

Storing in the same folder suits a lot of people. But the mechanism for storing in a seperate folder is probably not that useful due to the way it's currently coded.

As it stands, when RAW/DNG are stored seperately, they are all put in a single folder called A/DCIM/100CANON. That seems a little strange. At a minimum a folder name like A/DCIM/100RAW might have made more sense? Looking at the code, it seem it might originally have been intended that the "100" was to match the JPG folder number (at least on cameras that don't encode the date in the folder number) :

Code: [Select]

#define RAW_TARGET_DIRECTORY    "A/DCIM/%03dCANON"
sprintf(dir, RAW_TARGET_DIRECTORY, 100);

But now it's hard coded as "100".

It seems there might be some options here :

change 100Canon to 100RAW or 100DNG or 100CHDK or RAW_DNG to make it easier for newbies to find
start using folders that parrrallel the JPG folder names - something like nnnRAW for cams that don't use date coded folders and dateRAW for those that do.
store RAW/DNG files in a top directory other than A/DCIM/ (like A/RAWDNG/) and then match the JPG file convention in the subdirectories beneath the new main directory

Coding this looks simple - it's just a question of how the various generations of cameras might react to folder names they are not expecting. But as 100Canon works, I think it should be okay.

Side note : the sx510hs , ixus140_elph130 , ixus255_elph330hs , ixus132_elph115 all have a bug where RAW/DNG files are each stored in a separate folder (with folder names like A/DCIM/101___09/ETC_0112.TMP/) under their JPG's folder . This has something to do with how _GetImageFolder works on those cameras and I think could be fixed by inserting 0x00 in the 15th character in the returned folder name string.

Logged

Ported : A1200 SD940 G10 Powershot N G16

philmoz

3449

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #1 on: 21 / June / 2014, 17:40:42 »

Using a different directory name may prevent the CHDK RAW images from being visible when the camera is connected to a PC (assuming you can find an extension that makes them visible in the first place).

I don't think this is a big issue; just saying it might be a side effect.

The folder name issue on the four cameras you mentioned could be a finsig problem with it finding an incorrect address for _GetImageFolder. I'll take a look.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

waterwingz

12138

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #2 on: 21 / June / 2014, 18:27:00 »

Quote from: philmoz on 21 / June / 2014, 17:40:42

Using a different directory name may prevent the CHDK RAW images from being visible when the camera is connected to a PC (assuming you can find an extension that makes them visible in the first place).

Yea - that's kind of what I was thinking when I mentioned "it's just a question of how the various generations of cameras might react to folder names they are not expecting". As long as we left the current option for putting file in the same directories as the JPG (and maybe putting them all in 100Canon?) then adding the ability to put them in a whole separate directory structure should not hurt. Don't use that option if it doesn't fit your workflow?

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14000

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #3 on: 21 / June / 2014, 22:42:51 »

Quote from: waterwingz on 21 / June / 2014, 12:32:02

Side note : the sx510hs , ixus140_elph130 , ixus255_elph330hs , ixus132_elph115 all have a bug where RAW/DNG files are each stored in a separate folder (with folder names like A/DCIM/101___09/ETC_0112.TMP/) under their JPG's folder . This has something to do with how _GetImageFolder works on those cameras and I think could be fixed by inserting 0x00 in the 15th character in the returned folder name string.

I'm pretty sure ixus140 does not suffer from this, since it uses the workaround you mention, and has since I checked the port in.

We really need to encourage better testing to prevent this kind of thing from being included in ports that get checked in.

A lua script that verifies that string.format('%s/IMG_%04d.JPG',get_image_dir(),get_exp_count()) gives the last shot jpeg would be a useful addition to the tests.

Logged

Don't forget what the H stands for.

reyalp

14000

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #4 on: 21 / June / 2014, 22:46:33 »

As far as the original question goes:

100CANON is basically broken on the cameras that use date based naming, so I don't think it should be the default there. IMO, "in dir with jpeg" should probably be the default. Allowing images to be saved somewhere completely outside of the DCIM tree might be a useful option.

If you want to be able to download with (non-chdk) USB, the in folder with jpeg is likely the best choice anyway (assuming it works).

Logged

Don't forget what the H stands for.

nafraf

1308

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #5 on: 22 / June / 2014, 00:19:19 »

Quote from: reyalp on 21 / June / 2014, 22:42:51

Quote from: waterwingz on 21 / June / 2014, 12:32:02
Side note : the sx510hs , ixus140_elph130 , ixus255_elph330hs , ixus132_elph115 all have a bug where RAW/DNG files are each stored in a separate folder (with folder names like A/DCIM/101___09/ETC_0112.TMP/) under their JPG's folder . This has something to do with how _GetImageFolder works on those cameras and I think could be fixed by inserting 0x00 in the 15th character in the returned folder name string.
I'm pretty sure ixus140 does not suffer from this, since it uses the workaround you mention, and has since I checked the port in.

The same workaround was used in sx510, I have not seen that bug. Perhaps the cause of confusion are comments out of date comments in platform/sx510/shooting.c.

Logged

waterwingz

12138

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #6 on: 22 / June / 2014, 01:11:07 »

Quote from: reyalp on 21 / June / 2014, 22:42:51

I'm pretty sure ixus140 does not suffer from this, since it uses the workaround you mention, and has since I checked the port in.

Quote from: nafraf on 22 / June / 2014, 00:19:19

The same workaround was used in sx510, I have not seen that bug. Perhaps the cause of confusion are comments out of date comments in platform/sx510/shooting.c.

I ran into this issue testing RAW/DNG on the Powershot N. A little bit of "grepping" took me to the comments on the other ports I mentioned. I guess I really should have checked that the comments reflected what was actually in the code.

Quote from: reyalp on 21 / June / 2014, 22:46:33

As far as the original question goes:
100CANON is basically broken on the cameras that use date based naming, so I don't think it should be the default there. IMO, "in dir with jpeg" should probably be the default. Allowing images to be saved somewhere completely outside of the DCIM tree might be a useful option.
If you want to be able to download with (non-chdk) USB, the in folder with jpeg is likely the best choice anyway (assuming it works).

I think the idea of making the "save in dir with jpg" the default option is a good idea. I also like the idea of having the option to mirror the DCIM tree in a RAW tree where the jpg's continue to go in DCIM and the raw/dng goes in RAW. I'm not sure that the other choice (having all the raw/dng file in a single directory) is a lot of use but if I'm going to do a patch, it would not be a big deal to make that the third choice.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14000

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #7 on: 22 / June / 2014, 02:13:13 »

Quote from: waterwingz on 22 / June / 2014, 01:11:07

I ran into this issue testing RAW/DNG on the Powershot N. A little bit of "grepping" took me to the comments on the other ports I mentioned. I guess I really should have checked that the comments reflected what was actually in the code.

Hmm, yes looks like I didn't update that comment when I made the workaround...

FWIW, the "in dir with jpeg" option predates my involvement with CHDK, but my guess is that the hard coded 100CANON option came first, before anyone figured out how to get the proper directory name.

If we canted to do the equivalent for modern cameras, I suppose we could use something like 101___01

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #8 on: 22 / June / 2014, 11:24:56 »

Candidate patch file attached.

Mod's the "RAW File in Dir with JPEG" to "RAW File Folder" and gives three options : A/DCIM , withJPG , A/RAW . Best I could do with the seven characters space limit.

A/DCIM : stores the raw/dng files in either A/DCIM/100Canon or A/DCIM/101___01 (based on #ifdef CAM_DATE_FOLDER_NAMING)
withJPG : stores the raw/dng files in the same folder as the JPG's (this is now the default setting)
A/RAW : stores the raw/dng files in a new main folder and uses the subfolder naming convention used for the camera's A/DCIM jpg storage.

Comments ?

raw_folder_choices.patch (4.82 kB - downloaded 19 times.)

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14000

Re: Time for better folder naming for saving RAW/DNG ?

« Reply #9 on: 12 / July / 2014, 22:20:49 »

I added this to the trunk. I'd like a more descriptive name for the A/DCIM option, but I couldn't come up with one that would fit.

It's a bit confusing that A/RAW uses canon folder numbers like withJPEG, while A/DCIM doesn't.

If we could display 100CANON / 101__01 that might be better, but I'm not sure how you'd get that in the menu.

Logged

Don't forget what the H stands for.