Improving support for cameras with Canon raw - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: [1] 2 3 4   Go Down

Improving support for cameras with Canon raw

  • 31 Replies
  • 2198 Views
*

Offline reyalp

  • ******
  • 12690
Improving support for cameras with Canon raw
« on: 11 / March / 2019, 00:38:29 »
Advertisements
Finally started working on this, which has been on my TODO list for while.

Currently, the only thing that sort of knows about Canon raw is conf.c is_raw_exception. This checks PROPCASE_RESOLUTION == 5, which was RAW on old propset 2 and 3 cams. Later cams uses a separate propcase, and on some the resolution value 5 is re-used for a jpeg size, making this quite broken.

Some other cases where it would be useful for CHDK to be aware of Canon raw:

Remote shoot
The filewrite stuff works with Canon raw files, but for full functionality it needs to know how many files to expect, and getting the extension right would be nice too.

Script
It's nice for scripts to be able to know which formats are enabled, and set them differently from whatever you have set for manual shooting.


Background details:

Native raw cams, with OS and propset
g9      dry23, ps2
g10      dry31, ps2
sx1      dry31, ps2

g11      dry39, ps3
s90      dry39, ps3

s95      dry45, ps4
g12      dry45, ps4
g1x      dry49, ps4
s100   dry49, ps4

s110   dry51, ps5
g15      dry51, ps5
sx50hs   dry51, ps5

g16      dry52, ps6
s120   dry52, ps6? (no port yet)
g7x      dry55p6, ps7
sx60hs   dry55p6, ps8
m3      dry55p6, ps9
m10      dry57, ps10

All supported cams appear to use the CR2 image format. I thought the very earliest might be CRW, but the G9 manual specifically lists CR2. AFAIK none support the "sRAW" or "mRAW" formats found on some DSLRs. We don't yet support anything with the new CR3 format.

propset 2-3 enable raw by setting PROPCASE_RESOLUTION to 5.  I expect just setting the propcase is sufficient, since it is for other resolutions. A separate propcase controls raw+jpeg, but it's unclear whether this can be controlled through propcases alone. (https://chdk.fandom.com/wiki/PropertyCase mentions 280 on propset 2, both 214 or 284 for propset 3)

propset 4 and later cams use PROPCASE_IMAGE_FORMAT (0=raw, 1=jpeg, 2=both). This propcase can be found from the function UIFS_GetStillShotableNumber. It appears in the function regardless of whether the camera supports raw.

Setting PROPCASE_IMAGE_FORMAT is sufficient to change what is saved. On G7 X at least, it is possible to enable Canon raw in some modes where the camera does not normally support it, like portrait, toy camera etc. However, multi-shot modes like HDR crash. Setting it to non-jpeg values on cameras without raw support crashes.

to be continued...
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 12690
Re: Improving support for cameras with Canon raw
« Reply #1 on: 11 / March / 2019, 01:14:41 »
In r5161, I started adding code to support this, and addressed the is_raw_exception issue:

Added camera.h define CAM_HAS_CANON_RAW - set on all the cameras listed previously.
Added PROPCASE_IMAGE_FORMAT for propsets 4 and 5. This is based on UIFS_GetStillShotableNumber, untested by I'm fairly confident it's OK.

Added shooting_get_canon_raw_enabled, which returns 1 if Canon raw enabled on cameras CAM_HAS_CANON_RAW. If PROPCASE_IMAGE_FORMAT is defined, it uses that, otherwise the RESOLUTIOn == 5 check. It might be preferable to return jpeg, raw or raw+jpeg somehow, but I need testing on 2/3 cams to know how to do that.

Added a "raw exception" "Disable Raw @ Canon RAW", default to on i.e. don't save CHDK raw when Canon raw is enabled.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4262
Re: Improving support for cameras with Canon raw
« Reply #2 on: 11 / March / 2019, 15:23:41 »
Quote from: reyalp on 11 / March / 2019, 01:14:41
It might be preferable to return jpeg, raw or raw+jpeg somehow, but I need testing on 2/3 cams to know how to do that.
I have 2 cams with native raw - the m10 (propset 10) and g10 (propset 2). What would I need to test?
Logged
*

Offline reyalp

  • ******
  • 12690
Re: Improving support for cameras with Canon raw
« Reply #3 on: 11 / March / 2019, 16:18:34 »
Quote from: srsa_4c on 11 / March / 2019, 15:23:41
What would I need to test?
On G10, can you check which propcase(s) change when you switch between raw only and raw+jpeg?

On M10, you could check that PROPCASE_IMAGE_FORMAT (221)  0=raw, 1=jpeg, 2=both.

Thanks.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4262
Re: Improving support for cameras with Canon raw
« Reply #4 on: 11 / March / 2019, 17:11:41 »
Quote from: reyalp on 11 / March / 2019, 16:18:34
On G10, can you check which propcase(s) change when you switch between raw only and raw+jpeg?
Like written on the propcase page:
Propcase 280 follows the raw + L jpeg setting in the rec menu. You only get raw if you also select raw in the shooting menu (that's propcase 218).
Propcase 218 and 57 also change like documented (activating raw sets "fine" jpeg quality and removes the quality setting from UI).

Quote
On M10, you could check that PROPCASE_IMAGE_FORMAT (221)  0=raw, 1=jpeg, 2=both.
I can confirm that.
Logged
*

Offline reyalp

  • ******
  • 12690
Re: Improving support for cameras with Canon raw
« Reply #5 on: 18 / March / 2019, 01:34:18 »
Here's a preliminary patch for Canon raw remote shoot support. The chdkptp patch is Lua only (against current svn, not the last snapshot), so doesn't need a rebuild.

I've only tested it on g7x and some of my other cams without native raw.

To remoteshoot canon raw, use -craw. Currently, you must ensure the requested Canon formats are enabled on the camera: If you have raw only enabled, then -jpg will result in an error, but if you have raw+jpeg, then you could use -craw alone (jpeg is discarded) -jpeg (cr2 is discarded) or -jpg -craw.

I intend to make this more friendly, either by defaulting to the enabled formats, or enabling what was requested... or at least a clearer error message  ;)

All can be combined with CHDK -raw or -dng.

The code changes are somewhat complicated because it now needs to know which format filewrite is saving, and needs to know when a file is done even if it's being discarded. Currently the code assumes that Canon raw is always written before the jpeg. It would be possible (and probably better) to use the filename from Open or fwt_data_struct instead.

I renamed some things to no refer to jpeg (because it was confusing me), but I skipped the stuff referenced form platform code to avoid needless noise in the diff.

I'd be particularly interested to know if cameras with CAM_FILEWRITETASK_SEEKS and propset 2 and 3 cams work.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4262
Re: Improving support for cameras with Canon raw
« Reply #6 on: 23 / March / 2019, 13:59:11 »
Quote from: reyalp on 18 / March / 2019, 01:34:18
To remoteshoot canon raw, use -craw. Currently, you must ensure the requested Canon formats are enabled on the camera: If you have raw only enabled, then -jpg will result in an error, but if you have raw+jpeg, then you could use -craw alone (jpeg is discarded) -jpeg (cr2 is discarded) or -jpg -craw.
Testing this on g10, it seems to do what's intended so far. rs fails with init failed when craw or jpeg is requested but the file type is not enabled on Canon UI. The not requested file types are successfully ignored (they don't end up on card).
The port, however, has problems remote shooting craws. Some files arrive truncated, the missing amount varies a lot (up to ~80% can be lost). This isn't related to your patch, happens with vanilla trunk too. Big jpegs don't seem to cause problems, so there's something special going on when writing native raw files.
Can't test the m10 as it's missing filewrite support atm.

edit:
a3400 (CAM_FILEWRITETASK_SEEKS cam) seems to work ok, also with provoked large jpegs (superfine, shooting fine grid on monitor).
« Last Edit: 23 / March / 2019, 15:47:31 by srsa_4c »
Logged
*

Offline reyalp

  • ******
  • 12690
Re: Improving support for cameras with Canon raw
« Reply #7 on: 23 / March / 2019, 17:22:35 »
Thanks again :)

Regarding the propcases:
On G10, can you control raw+jpeg by setting propcase 280?

The wiki says it's read-only for propset 3, but also mentions 214 which sounds like PROPCASE_IMAGE_FORMAT on newer cameras.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4262
Re: Improving support for cameras with Canon raw
« Reply #8 on: 23 / March / 2019, 19:08:18 »
Quote from: reyalp on 23 / March / 2019, 17:22:35
Regarding the propcases:
On G10, can you control raw+jpeg by setting propcase 280?
Yes, and it survives taking a picture. The UI is not updated (but that's not unusual).
I guess ps 3 remains a mystery until someone helps us out.
Logged
*

Offline waterwingz

  • ******
  • 12044
    • Waterwingz
Re: Improving support for cameras with Canon raw
« Reply #9 on: 23 / March / 2019, 19:21:39 »
Quote from: srsa_4c on 23 / March / 2019, 13:59:11
The port, however, has problems remote shooting craws. Some files arrive truncated, the missing amount varies a lot (up to ~80% can be lost). This isn't related to your patch, happens with vanilla trunk too.
I can pretty much guarantee remote shooting craws was never tested by the person who did the port eight year ago. 

Is this likely to be an issue with where the hooks are placed?
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
Pages: [1] 2 3 4   Go Up
« previous next »
 

Related Topics