Improving support for cameras with Canon raw - page 2 - General Discussion and Assistance - CHDK Forum
supplierdeeply
« previous next »
Pages: 1 [2] 3 4   Go Down

Improving support for cameras with Canon raw

  • 31 Replies
  • 10902 Views
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #10 on: 23 / March / 2019, 19:42:23 »
Advertisements
Quote from: waterwingz on 23 / March / 2019, 19:21:39
I can pretty much guarantee remote shooting craws was never tested by the person who did the port eight year ago. 
Well yeah, remoteshoot has never formally supported Canon raw up to now, we (srsa_4c originally, I think) just observed at some point that the same filewrite hook appeared to work.
Quote
Is this likely to be an issue with where the hooks are placed?
A wild guess guess would be the filewrite code takes a code path for Canon raw that isn't hit for jpeg. My next step would probably be debug logging along the lines of what I have for FILEWRITE_DEBUG_LOG in ports like elph180 and g7x and just log what happens shooting raw vs jpeg without remote shoot
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #11 on: 08 / August / 2020, 22:54:26 »
Ping @koshy
I'd like to get some information about propcases on G11 if you still have it and are willing to test. Anyone else with a G11 or S90 is welcome to volunteer too ;)

Goal: Determine which propcases change when raw and raw + jpeg settings are changed

Probable propcases of interest
214: according to wiki,  0 RAW, 1 JPG, 2 RAW+JPG 
220 (PROPCASE_RESOLUTION): Should be 5 when raw is enabled, 0 = large jpeg etc.
284: according to wiki, Rec Raw + L-jpg (0=off, 1=on), writing has no effect

Test:

I'd suggest using chdkptp, change camera to rec, make sure shooting mode allows canon raw (like P).
For each of the following settings, set using Canon menus:
1) Raw disabled, by setting resolution to L
2) Raw enabled, raw + jpeg on
3) Raw enabled, raw + jpeg off
run
Code: [Select]
=return get_prop(214), get_prop(220), get_prop(284)

Note this should be done with any recent CHDK and chdkptp, does not require remoteshoot related patches up thread.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #12 on: 09 / August / 2020, 16:15:38 »
Quote from: srsa_4c on 23 / March / 2019, 13:59:11
The port, however, has problems remote shooting craws. Some files arrive truncated, the missing amount varies a lot (up to ~80% can be lost). This isn't related to your patch, happens with vanilla trunk too. Big jpegs don't seem to cause problems, so there's something special going on when writing native raw files.
I finally got around to looking at this :-[

It seems like MAX_CHUNKS_FOR_JPEG should be 4, rather than 3, based on having 4 cases in the main switch that refer to the write stage call, and the offset of the name field from sub sub_FFA46D70_my. The name field is almost certainly wrong in the current code.

Here's a patch with the define updated and debug logging (for 102a but should be trivial to adapt to to others). I'm guessing 4 chunks is only uses in the raw case.

To use the debug logging with chdkptp, I'd use
Code: [Select]
!require'extras/devutil'.init_cli()
dlstart
shoot
dlgetcam
Even if MAX_CHUNKS_FOR_JPEG fixes the issue, it would be nice to verify the name field is correct because I may use that to identify raw vs jpeg.

edit:
Would probably help if I attached the patch...
Logged
Don't forget what the H stands for.
*

Offline koshy

  • *****
  • 1096
Re: Improving support for cameras with Canon raw
« Reply #13 on: 09 / August / 2020, 16:35:17 »
Quote from: reyalp on 08 / August / 2020, 22:54:26
Ping @koshy
Pong, good you recall this works :)
Quote from: reyalp on 08 / August / 2020, 22:54:26
Test:

I'd suggest using chdkptp, change camera to rec, make sure shooting mode allows canon raw (like P).
For each of the following settings, set using Canon menus:
1) Raw disabled, by setting resolution to L
2) Raw enabled, raw + jpeg on
3) Raw enabled, raw + jpeg off
run
Code: [Select]
=return get_prop(214), get_prop(220), get_prop(284)

In the order suggested above:

Code: [Select]
> =return get_prop(214), get_prop(220), get_prop(284)
3:return:0
3:return:0
3:return:0
> =return get_prop(214), get_prop(220), get_prop(284)
4:return:0
4:return:5
4:return:1
> =return get_prop(214), get_prop(220), get_prop(284)
5:return:0
5:return:5
5:return:0
« Last Edit: 09 / August / 2020, 16:37:06 by koshy »
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #14 on: 09 / August / 2020, 17:22:05 »
Quote from: koshy on 09 / August / 2020, 16:35:17
In the order suggested above:
Thanks. Seems 214 isn't as documented on the wiki.

Can you check if setting 284 allows you to control the raw+jpeg setting?
I'd suggest
Enable RAW and set raw+jpeg off in Canon menus
=set_prop(284,1)
shoot
imls

check if the camera saved both

=set_prop(284,0)
shoot
imls

check if camera saved only raw.
Logged
Don't forget what the H stands for.
*

Offline koshy

  • *****
  • 1096
Re: Improving support for cameras with Canon raw
« Reply #15 on: 09 / August / 2020, 19:41:25 »
Quote from: reyalp on 09 / August / 2020, 17:22:05
Can you check if setting 284 allows you to control the raw+jpeg setting?
Sure, I did, it does. I had JPEG+RAW off on the Canon menu before starting.

Code: [Select]
> =set_prop(284,1)
> shoot
> imls
01/01/09 02:16:56   10497486 A/DCIM/879___01/IMG_0060.CR2
01/01/09 02:16:56    1312655 A/DCIM/879___01/IMG_0060.JPG
> =set_prop(284,0)
> shoot
> imls
01/01/09 02:16:56   10497486 A/DCIM/879___01/IMG_0060.CR2
01/01/09 02:16:56    1312655 A/DCIM/879___01/IMG_0060.JPG
01/01/09 02:17:26   10938890 A/DCIM/879___01/IMG_0061.CR2
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #16 on: 09 / August / 2020, 23:59:22 »
Thanks again. Looks like it behaves exactly like propset 2, which is nice.

With the propset 3 question resolved, here's a patch that adds Lua support for Canon raw control. I plan to check it in, just posting in case anyone has feedback checked in, r5546

get_canon_raw_support()
returns boolean, true if platform has native Canon raw support

get_canon_image_format()
returns current image format as a bitmask, 1 = jpeg, 2 = raw, 3 = both

set_canon_image_format(fmt)
fmt is a bitmask 1 = jpeg, 2 = raw, 3 = both
returns true if the format is valid for the current port, false otherwise

Set has some caveats:
On g7x (and I assume others) changing shooting modes or switching to/from play resets the IMAGE_FORMAT propcase. This makes sense since Canon raw is only available in certain shooting modes, and the actual setting comes from a ui prop / flash param. The reset happens regardless of whether the change is done using physical controls or script, or whether the modes involved support raw.

As discussed earlier in the thread, you can enable Canon raw in some shooting modes the firmware doesn't natively allow it, like monochrome or portrait, but some like HDR crash on shooting. No attempt is made to avoid this. I think this fine, especially since mode switching results in a safe setting.

On propset 2 and 3, raw is controlled by the resolution setting. This means switching from raw to jpeg is not well defined. The code tries deal with this by saving the previous resolution when switching to raw, and restoring it when switching to jpeg. If that's not available, for example because the camera was started in raw mode, L resolution is used. This is pretty hokey, but I think should be OK in practice.
« Last Edit: 12 / August / 2020, 01:52:36 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Improving support for cameras with Canon raw
« Reply #17 on: 12 / August / 2020, 19:03:45 »
Quote from: reyalp on 09 / August / 2020, 16:15:38
It seems like MAX_CHUNKS_FOR_JPEG should be 4, rather than 3, based on having 4 cases in the main switch that refer to the write stage call, and the offset of the name field from sub sub_FFA46D70_my. The name field is almost certainly wrong in the current code.

Here's a patch with the define updated and debug logging (for 102a but should be trivial to adapt to to others). I'm guessing 4 chunks is only uses in the raw case.
That seems to fix cr2 files when remote shooting.
Attached is a log, remoteshooting fine grid on my monitor for bigger filesize. One jpeg, one cr2.
Logged
*

Offline reyalp

  • ******
  • 14080
Re: Improving support for cameras with Canon raw
« Reply #18 on: 12 / August / 2020, 23:53:19 »
Quote from: srsa_4c on 12 / August / 2020, 19:03:45
That seems to fix cr2 files when remote shooting.
Attached is a log, remoteshooting fine grid on my monitor for bigger filesize. One jpeg, one cr2.
Thanks, checked in change to 4 chunks. Log confirms CR2 used 4, and the filename offset is now correct.

I wonder if other cameras are affected. We might be able to check in the sig finder, looking at the number of duplicate entries in the jumptable. I thought about doing the filename offset, but it looked annoying.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Improving support for cameras with Canon raw
« Reply #19 on: 13 / August / 2020, 16:51:16 »
Quote from: reyalp on 12 / August / 2020, 23:53:19
We might be able to check in the sig finder, looking at the number of duplicate entries in the jumptable.
I'll try that in finsig_dryos.

edit:
Code: [Select]
Index: tools/finsig_dryos.c
===================================================================
--- tools/finsig_dryos.c (revision 5547)
+++ tools/finsig_dryos.c (working copy)
@@ -4312,6 +4312,25 @@
     }
 
     find_DebugAssert_argcount(fw);
+
+    k = get_saved_sig(fw,"task_FileWrite");
+    if (k >= 0)
+    {
+        uint32_t fadr = func_names[k].val;
+        k1 = adr2idx(fw, fadr);
+        for (k=1; k<32; k++)
+        {
+            if ((fwval(fw, k1+k) & 0x0fffff00) == 0x008ff100) // add[cond] pc, pc, rx, lsl#2
+            {
+                for (k++;isB(fw,k1+k) && idxFollowBranch(fw,k1+k,1) != idxFollowBranch(fw,k1+k-1,1);k++);
+                int c = 1;
+                for (;isB(fw,k1+k) && idxFollowBranch(fw,k1+k,1) == idxFollowBranch(fw,k1+k-1,1);k++,c++);
+                bprintf("\n// Below goes in 'filewrite.c' or 'platform_camera.h':\n");
+                bprintf("//#define MAX_CHUNKS_FOR_JPEG %d // Found @0x%08x\n",c,idx2adr(fw,k+k1));
+                break;
+            }
+        }
+    }
 }
 
 //------------------------------------------------------------------------------------------------------------
You can make changes to this if you decide to check it in. I have not attempted to compare the sigfinder's constants to those in filewrite.c .
« Last Edit: 13 / August / 2020, 18:33:48 by srsa_4c »
Logged
Pages: 1 [2] 3 4   Go Up
« previous next »
 

Related Topics