A Few CHDK Questions

BuckSkin

23

RAW Files have a different name ???

« Reply #10 on: 08 / June / 2019, 21:04:42 »

Advertisements

Quote from: waterwingz on 08 / June / 2019, 11:46:09

There is also a utility for your PC that will do the same thing : WASP

From the WASP page: "While Mac OSX and Linux lets you see both partitions simultaneously, Windows will only show the active partition."

If it were not for the aggravation of rebooting back and forth when I have dozens of applications running, I would just switch over to Mint, copy the images to my main Pictures drive, then switch back to Windows and go on with my rat killing.

As it now stands, I have downloaded WASP and within seconds of unzipping the package, I am looking at my test images.

Thank you so much for making me aware of WASP; it is a great help.

I knew this wouldn't be the end of the tunnel; I have ran against another head-scratcher.
I fired up D.I.M. (Digital Image Mover) and set it to work off-copying, renaming, and foldering the files by date; I have never yet had DIM to let me down.
I was highly surprised to find that DIM completely ignored the CR2 RAW images.
I went to over-complicating things, trying to figure out why; and, then it jumped out at me; .....my jpeg images are named IMG_1234 and the accompanying RAW files are named CRW_1234; as far as DIM knows, the CRW_ files are in no way related to the IMG_ files, so it skipped them.

Disregard this bracketed bit as I got it figured out. [[I can skin this cat with Advanced Renamer (or Faststone for that matter), but DIM will do it with one click when the file names match.

Is there some way that I can either make the jpegs have the CRW_ or make the RAWs have the IMG_ ? ]]

I strained my eyes and found an option to use the same IMG_ prefix that the jpegs have, thus eliminating my D.I.M. problem.

« Last Edit: 09 / June / 2019, 00:14:36 by BuckSkin »

Logged

BuckSkin

23

Re: RAW Files have a different name ???

« Reply #11 on: 09 / June / 2019, 01:10:56 »

Quote from: BuckSkin on 08 / June / 2019, 21:04:42

I strained my eyes and found an option to use the same IMG_ prefix that the jpegs have, thus eliminating my D.I.M. problem.

Or so I had thought.......; I found the option to change the RAW prefix to match the jpeg prefix = IMG_
However, after taking several more test shots, all of my RAW files insist on keeping the CRW_ prefix
Maybe I did something wrong.

On EDIT: I put the card back in and checked the CHDK menu and everything I had set was changed back to the original settings.
I did nothing different than before, selecting the IMG_ prefix for the RAWS, made a few test shots, and this time all files had the IMG_ prefix.
Along the same line of thought, several menu changes I have made seem to revert back to default or are completely ignored.

« Last Edit: 09 / June / 2019, 04:50:07 by BuckSkin »

Logged

BuckSkin

23

Differing image sizes ???

« Reply #12 on: 09 / June / 2019, 05:04:12 »

I have found that I am getting three different image dimensions/sizes.

Canon large/fine jpeg is 3648 x 2736

CHDK RAW is 1842 x 1380 = quite a bit smaller = very close to half the size of the jpeg
The better quality may be there; but, the great reduction in size pretty much negates any advantages over the twice-as-large jpeg

CHDK DNG is 3668 x 2756 = 20 pixels larger in both dimensions.

Are these size discrepancies similar to what others are seeing ?

Do I have something set wrong that could be causing these size differences ?

On EDIT: It gets weirder or something is messing with my mind and making an idiot of me; looking at the pixel dimensions in Faststone, a CHDK RAW is 1842 x 1380; the finished jpeg from that RAW (processed in RAWTherapee; the only program I have that will accept them) is 3676 x 2752, nearly twice as large as the RAW; I don't even see how that can be possible.
Taken and processed in the same batch, a CHDK DNG is 3668 x 2756 and it's resultant jpeg is smaller at 3660 x 2748.

Alas, wonderful DxO Optics Pro 9 Elite, the RAW conversion program that made me crave RAW abilities in these compact cameras in the first place, will not recognize either the CHDK RAW, nor the CHDK DNG, refusing to do anything with either of them.

DxO will take the native RAW from my S90 and digest them into some beautiful jpegs without any complaint.

Regarding files that are compatible, try as I may, I have never been able to even come close to the impressive results I get with DxO when I play around with RAWTherapee; but then, RAWTherapee is way over my head; someone who is well versed in RAWTherapee no doubt can produce magnificent results.

« Last Edit: 09 / June / 2019, 10:59:16 by BuckSkin »

Logged

waterwingz

12140

Re: RAW Files have a different name ???

« Reply #13 on: 09 / June / 2019, 11:03:29 »

Quote from: BuckSkin on 09 / June / 2019, 01:10:56

On EDIT: I put the card back in and checked the CHDK menu and everything I had set was changed back to the original settings.
...
Along the same line of thought, several menu changes I have made seem to revert back to default or are completely ignored.

I have not looked at this in a while but changes in CHDK configuration do not get stored to the SD card immediately. I think it happens when you actually exit <ALT> mode (and maybe some on some other events?) so if you make changes and power the camera off while still in <ALT> mode, the changes may not be saved.

Quote from: BuckSkin on 09 / June / 2019, 05:04:12

I have found that I am getting three different image dimensions/sizes.

This is normal. The Canon firmware crops the sensor data a bit when it creates a JPG to eliminate bad/dark pixels at the very edge of the sensor and to correct for lens errors. CHDK RAW and DNG include those sensor pixels and so will be slightly larger (although you can set the crop size of the DNG in the CHDK RAW menu).

Quote

CHDK RAW is 1842 x 1380 = quite a bit smaller = very close to half the size of the jpeg

How did you determine that? CHDK RAW files have no information inside to tell you the pixel height or width of the information stored in the file. You pretty much have to look that up in the CHDK source code (or figure it out from the sensor specs). Also, the full uncropped DNG size and CHDK RAW size are based on the same constants definitions for a particular camera in the CHDK source code. They can't be different!

Quote

Regarding files that are compatible, try as I may, I have never been able to even come close to the impressive results I get with DxO when I play around with RAWTherapee; but then, RAWTherapee is way over my head; someone who is well versed in RAWTherapee no doubt can produce magnificent results.

I'm a little surprize the DxO will not read a DNG file. Did you try DNG versions 1.1 and 1.3?

Also, this : Converting CHDK RAW files to something DXO PhotoLab 2 will read

« Last Edit: 09 / June / 2019, 11:18:18 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

BuckSkin

23

Re: RAW Files have a different name ???

« Reply #14 on: 09 / June / 2019, 14:56:11 »

Quote from: waterwingz on 09 / June / 2019, 11:03:29

Quote from: BuckSkin on 09 / June / 2019, 01:10:56
On EDIT: I put the card back in and checked the CHDK menu and everything I had set was changed back to the original settings.
...
Along the same line of thought, several menu changes I have made seem to revert back to default or are completely ignored.
I have not looked at this in a while but changes in CHDK configuration do not get stored to the SD card immediately. I think it happens when you actually exit <ALT> mode (and maybe some on some other events?) so if you make changes and power the camera off while still in <ALT> mode, the changes may not be saved.

I will make sure I give my changes time to take hold and see if I have better success.

Quote

Quote from: BuckSkin on 09 / June / 2019, 05:04:12
I have found that I am getting three different image dimensions/sizes.
This is normal. The Canon firmware crops the sensor data a bit when it creates a JPG to eliminate bad/dark pixels at the very edge of the sensor and to correct for lens errors. CHDK RAW and DNG include those sensor pixels and so will be slightly larger (although you can set the crop size of the DNG in the CHDK RAW menu).

That makes sense and is something I had not learned before.

Quote

Quote
CHDK RAW is 1842 x 1380 = quite a bit smaller = very close to half the size of the jpeg
How did you determine that? CHDK RAW files have no information inside to tell you the pixel height or width of the information stored in the file. You pretty much have to look that up in the CHDK source code (or figure it out from the sensor specs). Also, the full uncropped DNG size and CHDK RAW size are based on the same constants definitions for a particular camera in the CHDK source code. They can't be different!

When viewing the various files in Faststone is where I am seeing the pixel dimensions that I referred to.

Quote

Quote
Regarding files that are compatible, try as I may, I have never been able to even come close to the impressive results I get with DxO when I play around with RAWTherapee; but then, RAWTherapee is way over my head; someone who is well versed in RAWTherapee no doubt can produce magnificent results.
I'm a little surprize the DxO will not read a DNG file. Did you try DNG versions 1.1 and 1.3?

I think I have the menu set for 1.3; I will try 1.1 and see how it goes.
I think I remember reading somewhere lately that DxO does not accept DNG files; I, too, find that surprising since so many camera brands produce DNG.

Quote

Also, this : Converting CHDK RAW files to something DXO PhotoLab 2 will read

Thanks! I really appreciate your patience with me bombarding you with questions.
I may have already mentioned this; if so, it won't hurt repeating: Every RAW editor I have (and I have several) will handle the native RAW files from my S90, including DxO; I find it surprising that so many refuse the CHDK RAWs.

Another interesting thing that I have discovered: the metadata is stripped from jpegs created from CHDK RAW files converted in RAWTherapee; while, the DNG/RAWTherapee jpegs retain all of the metadata.
I never was a big fan of DNG, but that is good reason to choose DNG over CHDK RAW.
Of course, if the resulting jpegs are better from the CHDK files, using Elements 7, I can open the original camera jpeg with all information intact as a background layer, then drag the CHDK jpeg on top of it; the resulting jpeg will always retain whatever metadata that is in the "background" file.

Logged

reyalp

14110

Re: RAW Files have a different name ???

« Reply #15 on: 09 / June / 2019, 16:30:29 »

Quote from: BuckSkin on 09 / June / 2019, 14:56:11

Another interesting thing that I have discovered: the metadata is stripped from jpegs created from CHDK RAW files converted in RAWTherapee; while, the DNG/RAWTherapee jpegs retain all of the metadata.

CHDK raw contain no metadata. It is just the sensor data and nothing more. This is why many programs don't support it, and why we strongly encourage users to use DNG instead. The data is the effectively the same, so it should always be possible to get equivalent results from DNG. There are very few cases where using CHDK raw has any advantage.

Logged

Don't forget what the H stands for.

waterwingz

12140

Re: RAW Files have a different name ???

« Reply #16 on: 09 / June / 2019, 16:42:37 »

Quote from: BuckSkin on 09 / June / 2019, 14:56:11

When viewing the various files in Faststone is where I am seeing the pixel dimensions that I referred to.

I have not looked at Faststone in a long time but I suspect it's just making stuff up. Some CHDK RAW files are "understood" by RAW software if it is based on the DCRAW libraries. DCRAW just guesses at the image x & y coordinates based on the file size matching a CHDK RAW file size that someone has reported to the author of DCRAW.

Quote

I think I remember reading somewhere lately that DxO does not accept DNG files; I, too, find that surprising since so many camera brands produce DNG.

Link from the DXO website : What DNG files are supported by DxO OpticsPro as input ?. So it looks like CHDK DNG files don't qualify.

Quote

Every RAW editor I have (and I have several) will handle the native RAW files from my S90, including DxO; I find it surprising that so many refuse the CHDK RAWs.

I think your confusion is in the use of the term "RAW". Canon .CR2 RAW files produced natively in the camera have a lot more than just compressed sensor data embedded in the image. You could probably think of them more as JPG files without any compression. The CHDK files - DNG or RAW - just contain the actual pixel values pretty much straight from the sensor.

Quote

Another interesting thing that I have discovered: the metadata is stripped from jpegs created from CHDK RAW files converted in RAWTherapee; while, the DNG/RAWTherapee jpegs retain all of the metadata.

RawTherapee is not stripping anything. There is no metadata in a CHDK RAW file - just a dump of the sensors pixel values. CHDK DNG files do contain metadata.

Quote

Of course, if the resulting jpegs are better from the CHDK files, using Elements 7, I can open the original camera jpeg with all information intact as a background layer, then drag the CHDK jpeg on top of it; the resulting jpeg will always retain whatever metadata that is in the "background" file.

Now that's a different debate. Can you actually get a better final image by post processing RAW/DNG files vs what the camera can produce by itself as the highest resolution / lowest compression JPG's. I have not seen many people offer up images that meet that test - and almost certainly very few people are willing to post process more than a few images that way. YMMV.

Logged

Ported : A1200 SD940 G10 Powershot N G16

BuckSkin

23

Adobe DNG Converter

« Reply #17 on: 09 / June / 2019, 19:40:59 »

Just experimenting around, I found that Adobe DNG Converter version 9.3.0.506 will not recognize CHDK RAW files.

Another fact that I found surprising is when I mistakenly ran a batch of CHDK-produced DNG files through the DNG Converter, it cut the file size by 1/3.
The CHDK DNGs average 14.7 MB; after running these same files through the DNG Converter, they average 9.7 MB.
The DNG-converted DNG images look quite a bit better than the virgin DNG images.
I thought it would be the other way around.

Logged

waterwingz

12140

Re: Adobe DNG Converter

« Reply #18 on: 09 / June / 2019, 19:55:28 »

Quote from: BuckSkin on 09 / June / 2019, 19:40:59

Just experimenting around, I found that Adobe DNG Converter version 9.3.0.506 will not recognize CHDK RAW files.

Ummm .. that's pretty much what we have been trying to tell you. CHDK RAW is not an image file format - it's just a binary file full of unprocessed sensor data. Very few image processing programs can use it - only those based on DCRAW and even then only for some cameras.

Quote

Another fact that I found surprising is when I mistakenly ran a batch of CHDK-produced DNG files through the DNG Converter, it cut the file size by 1/3.
The CHDK DNGs average 14.7 MB; after running these same files through the DNG Converter, they average 9.7 MB. The DNG-converted DNG images look quite a bit better than the virgin DNG images. I thought it would be the other way around.

Was the DNG converter applying any sort of lossless compression?

Logged

Ported : A1200 SD940 G10 Powershot N G16

BuckSkin

23

Re: Adobe DNG Converter

« Reply #19 on: 09 / June / 2019, 20:47:08 »

Quote from: waterwingz on 09 / June / 2019, 19:55:28

Was the DNG converter applying any sort of lossless compression?

I really cannot tell; all of those MBs had to disappear somewhere.
It embeds the original and has a button to extract the original; to my way of thinking, with the original embedded, the resulting files should have been huge.

Just out of curiosity, I opened Chasys Draw IES (FREE and not like anything else out there) and it opened and edited CHDK RAW images as easily as if they were jpegs.

http://www.jpchacha.com/chasysdraw/

Adobe_DNGConverter_ControlBoard.png (67.8 kB, 622x643 - viewed 43 times.)

Logged