Can't use CRW files - page 2 - RAW Shooting and Processing - CHDK Forum  

Can't use CRW files

  • 29 Replies
  • 28431 Views
*

Offline uhoh

  • *
  • 14
Re: Can't use CRW files
« Reply #10 on: 09 / March / 2017, 11:57:23 »
Advertisements
If you can upload an example of one of the original CRW files somewhere (dropbox, google drive etc), we may be able to get a better idea of what's happening.

Note that rawconvert does not demosiac, it just takes the packed 12 bit data and converts it 8 or 16 bit formats that can be loaded into other imaging programs. If you load the resulting pgm in gimp or similar, you will generally a recognizable grayscale image with a checkerboard pattern from the CFA.

OK here's the pgm: https://www.dropbox.com/s/ec1thtgb15efyzl/CRW_2435.CRW.pgm?dl=0

And here's the jpeg:
https://www.dropbox.com/s/p9ydk6tkasau614/IMG_2435.JPG?dl=0

Re not demosaicing, that makes sense. Probably it's time to add that feature, if I can find a decent algorithm.

*

Offline reyalp

  • ******
  • 12864
Re: Can't use CRW files
« Reply #11 on: 09 / March / 2017, 16:17:50 »
OK here's the pgm: https://www.dropbox.com/s/ec1thtgb15efyzl/CRW_2435.CRW.pgm?dl=0

And here's the jpeg:
https://www.dropbox.com/s/p9ydk6tkasau614/IMG_2435.JPG?dl=0
I asked for the original CRW, but from the PGM I would say your CRW only contains random garbage from camera memory, not image data. This happens on some cameras where the raw data isn't available in certain shooting modes (often AUTO), or the raw buffer address is wrong. According the jpeg EXIF, this image was shot in AUTO, so that is likely to be the case.

If any of your CRWs are OK, you should be able to recognize the image in the PGM.

Quote
Re not demosaicing, that makes sense. Probably it's time to add that feature, if I can find a decent algorithm.
As noted in the source, rawconvert is intended as a debugging tool, not an imaging too. Getting good looking images out of raw requires a lot more than just demosiacing, especially for CRW framebuffer dump raw which have no calibration or other metadata.

You could feed the output of rawconvert to something that knows how to demosiac (dcraw, or pambayerp perhaps?) but again, getting nice results requires proper calibration.

DNG4PS2 gets around some of this by using the jpeg for metadata. If some of your raws contain valid images, setting up a VM that can run DNG4PS2 might be a good option. Or maybe the jpegs are good enough ;)
Don't forget what the H stands for.

*

Offline uhoh

  • *
  • 14
Re: Can't use CRW files
« Reply #12 on: 09 / March / 2017, 18:53:19 »
OK here's the pgm: https://www.dropbox.com/s/ec1thtgb15efyzl/CRW_2435.CRW.pgm?dl=0

And here's the jpeg:
https://www.dropbox.com/s/p9ydk6tkasau614/IMG_2435.JPG?dl=0
I asked for the original CRW

It was in the original posting:
https://www.dropbox.com/s/93ew7b8sdc8zkmz/CRW_2435.CRW?dl=0

Anyway I think my best option here is to as you say accept the JPEGs or if possible re-shoot these, storing as DNGs.
« Last Edit: 09 / March / 2017, 18:55:55 by uhoh »

*

Offline reyalp

  • ******
  • 12864
Re: Can't use CRW files
« Reply #13 on: 09 / March / 2017, 22:24:02 »
It was in the original posting:
https://www.dropbox.com/s/93ew7b8sdc8zkmz/CRW_2435.CRW?dl=0
Oh, sorry, I missed that. I can confirm this file doesn't contain valid image data.
Quote
Anyway I think my best option here is to as you say accept the JPEGs or if possible re-shoot these, storing as DNGs.
Yes, and use P mode rather than AUTO.

If you can confirm that all the images you shot in AUTO are bad like the one you posted, I will update the port to disable raw in AUTO. Many cameras of this generation don't seem to have valid raw data in AUTO.

If only some of them are bad, or you also sometimes get the bad images in P mode, there may be some other bug in the port.

When you test, you should try both single shot and continuous shooting for 3 or more shots.
Don't forget what the H stands for.


*

Offline uhoh

  • *
  • 14
Re: Can't use CRW files
« Reply #14 on: 10 / March / 2017, 10:49:07 »

If you can confirm that all the images you shot in AUTO are bad like the one you posted, I will update the port to disable raw in AUTO. Many cameras of this generation don't seem to have valid raw data in AUTO.

OS/X says the bad CRW' JPGs are:
Scene Capture Type: Standard
Exposure Mode: Auto exposure.

Earlier good DNGs' JPGs were
Scene Capture Type: Portrait
Exposure Mode: Auto exposure

I'll try the continuous shooting this weekend.
« Last Edit: 10 / March / 2017, 11:03:37 by uhoh »

*

Offline reyalp

  • ******
  • 12864
Re: Can't use CRW files
« Reply #15 on: 10 / March / 2017, 13:18:19 »
OS/X says the bad CRW' JPGs are:
Scene Capture Type: Standard
Exposure Mode: Auto exposure.
Yeah, sounds like this is one of the cameras that has trouble in auto mode. There were similar reports in the porting thread, but it wasn't really confirmed.

For most normal shooting, I'd suggest using P (Program AE)
Don't forget what the H stands for.

*

Offline uhoh

  • *
  • 14
Re: Can't use CRW files
« Reply #16 on: 12 / March / 2017, 17:37:00 »
When you test, you should try both single shot and continuous shooting for 3 or more shots.

OK, DNGs still work fine. They work in both Portrait mode and in P mode.

All close-up DNGs have a fish-eye appearance whereas their equivalent JPGs do not.

Continuous mode does not initially support DNGs and just produced a series of JPGs, but then I enabled it in the settings and it works fine but it slow.

The High Speed Burst 4MB mode does not produce DNGs.

The monochrome mode produces color DNGs of course.
« Last Edit: 12 / March / 2017, 18:00:36 by uhoh »

*

Offline reyalp

  • ******
  • 12864
Re: Can't use CRW files
« Reply #17 on: 12 / March / 2017, 19:43:27 »
Thanks for testing. I've updated the port to disable raw in auto mode, build 4767 and later.
All close-up DNGs have a fish-eye appearance whereas their equivalent JPGs do not.
This is normal, the camera does jpeg distortion correction in software, the raw is what the sensor actually recorded. You can use a lens profile (there's one for ixus125/elph110 in https://chdk.setepontos.com/index.php?topic=9268.0) or correct the distortion in programs like raw therapee.

Quote
The High Speed Burst 4MB mode does not produce DNGs.
Does not save DNGs at all, or saves bad ones? The code for this port is set to disable raw in "low light", which might also catch this mode since they are implemented the same way. In any case, these reduced resolution modes are not expected to generate usable raw, so not saving in this mode would be correct.
Don't forget what the H stands for.


*

Offline uhoh

  • *
  • 14
Re: Can't use CRW files
« Reply #18 on: 13 / March / 2017, 08:36:27 »

Does not save DNGs at all, or saves bad ones? The code for this port is set to disable raw in "low light", which might also catch this mode since they are implemented the same way. In any case, these reduced resolution modes are not expected to generate usable raw, so not saving in this mode would be correct.

Does not save.

Re not saving in low light, what's the reason?
 I've used this camera to take astronomical photos and I plan to continue, but there have been problems with long exposures sometimes going all white.

*

Offline reyalp

  • ******
  • 12864
Re: Can't use CRW files
« Reply #19 on: 13 / March / 2017, 17:14:00 »
Re not saving in low light, what's the reason?
I meant the special "low light" or "ISO3200" modes, not just shooting in low light. These modes are half resolution like the high speed burst one you mentioned, and all work by binning the raw data to lower resolution before the main CPU sees it. CHDK currently isn't able to save useful raw in these cases, see https://chdk.setepontos.com/index.php?topic=10648.0


Quote
I've used this camera to take astronomical photos and I plan to continue, but there have been problems with long exposures sometimes going all white.
How long? It's not unusual to run into problems when exposures are many minutes long, but in this case, you will probably get better results taking multiple shots and stacking them. This can be done dedicated software like registax or deep sky stacker, or some general imaging software like gmic.
Don't forget what the H stands for.

 

Related Topics