CHDK PTP interface

  • 1136 Replies
  • 147265 Views
*

Offline srsa_4c

  • ******
  • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1070 on: 15 / March / 2014, 17:36:51 »
    Advertisements
    Continuing here, since it's a CHDK-side problem.
    First chunk, but last=true?
    This sounds like it could match multiple invocations of filewrite, where the first "session" only has one chunk.
    It seems you're right. Fwt data blocks below, sampled in filewrite_main_hook(), a3200

    Good JPEG
    Code: [Select]
    Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

    00000000  09 00 00 00 00 00 00 00 01 03 00 00 9A BB 24 53  ............š»$S
    00000010  00 00 20 09 00 64 A6 43 00 E4 13 00 00 F4 74 43  .. ..d¦C.ä...ôtC
    00000020  B9 36 1D 00 00 00 00 00 00 00 00 00 00 00 00 00  ą6..............
    00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000050  41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F  A/DCIM/219___03/
    00000060  49 4D 47 5F 30 30 32 38 2E 4A 50 47 00 00 00 00  IMG_0028.JPG....
    Bad JPEG, 4 fwt invocations
    Code: [Select]
    Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

    00000000  09 00 00 00 00 00 00 00 01 83 00 00 A8 BB 24 53  ............¨»$S
    00000010  00 00 20 09 00 F4 74 43 00 AA 22 00 00 00 00 00  .. ..ôtC.Ş".....
    00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000050  41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F  A/DCIM/219___03/
    00000060  49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00  IMG_0029.JPG....

    00000000  09 00 00 00 00 00 00 00 09 80 00 00 A8 BB 24 53  .........€..¨»$S
    00000010  00 00 20 09 00 A0 97 43 00 A8 22 00 00 00 00 00  .. .. —C.¨".....
    00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000050  41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F  A/DCIM/219___03/
    00000060  49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00  IMG_0029.JPG....

    00000000  09 00 00 00 00 00 00 00 09 80 00 00 A8 BB 24 53  .........€..¨»$S
    00000010  00 00 20 09 00 F4 74 43 49 FB 09 00 00 00 00 00  .. ..ôtCIű......
    00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000050  41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F  A/DCIM/219___03/
    00000060  49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00  IMG_0029.JPG....

    00000000  09 00 00 00 00 00 00 00 01 00 00 00 A8 BB 24 53  ............¨»$S
    00000010  00 00 20 09 00 A0 97 43 00 40 00 00 00 00 00 00  .. .. —C.@......
    00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00000040  00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00  ................
    00000050  41 2F 44 43 49 4D 2F 32 31 39 5F 5F 5F 30 33 2F  A/DCIM/219___03/
    00000060  49 4D 47 5F 30 30 32 39 2E 4A 50 47 00 00 00 00  IMG_0029.JPG....
    The trick is that the first chunk starts with garbage, the exif block will be written there - in the final fwt invocation for that file.

    I'm not sure yet how to handle this...

    Right now we don't have a consistent way to tell if a port supports canon raw, or if it is enabled. We'd also need a way of selecting Canon raw for remote shoot, while running through the same code jpeg.
    'Native RAW support' will probably be a define, there are at least 2 propcases that have to do with native raw / raw+jpeg selection.

    *

    Offline srsa_4c

    • ******
    • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1071 on: 16 / March / 2014, 16:25:08 »
    Proposed fix for cameras with multipass JPEG writes. Affected cameras are probably:
    a490, a495, ixus1000, sx40, g12, ixus310, sx220, sx230, s90, sx30, ixus230, a2200, g1x, ixus200, a3200 (the list only includes models which already have filewritetask support).
    The patch only includes support for a3200, a490, a495 at the moment, I'll probably do the rest too.
    For affected cameras, 'fwt_data_struct' will have an additional member, 'oflags'. 'oflags' is the second argument to open (fwt_open), so it isn't hard to recognize.

    Comments on the patch are welcome.

    *

    Offline nafraf

    • *****
    • 1255
  • Publish
    Re: CHDK PTP interface
    « Reply #1072 on: 16 / March / 2014, 20:36:25 »
    fwt_multipass_support patch tested on A495, it works!! Thanks.
    Maybe oflags could be renamed as open_flags?

    *

    Offline reyalp

    • ******
    • 9810
  • Publish
    Re: CHDK PTP interface
    « Reply #1073 on: 16 / March / 2014, 21:47:41 »
    Here's an a490 100f build for jpq5588

    The patch makes sense to me. It seems like it might be possible to clean up the ifdef code a bit so there is less duplicated code. It might also be clearer to have CAM_FILEWRITETASK_SEEKS automatically define multipass, so the common parts only check one ifdef.
    Don't forget what the H stands for.


    *

    Offline srsa_4c

    • ******
    • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1074 on: 17 / March / 2014, 13:16:46 »
    Proposed fix for cameras with multipass JPEG writes.
    I'm still getting partial JPEGs occasionally, it's either an unhandled saving pattern or a bug...  :(

    It seems like it might be possible to clean up the ifdef code a bit so there is less duplicated code. It might also be clearer to have CAM_FILEWRITETASK_SEEKS automatically define multipass, so the common parts only check one ifdef.
    Will do this in the next patch.
    « Last Edit: 17 / March / 2014, 13:22:56 by srsa_4c »

    *

    Offline srsa_4c

    • ******
    • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1075 on: 17 / March / 2014, 18:18:54 »
    Proposed fix for cameras with multipass JPEG writes.
    I'm still getting partial JPEGs occasionally, it's either an unhandled saving pattern or a bug...
    It was a bug. New (still incomplete) patch is attached, along with a new test build for that camera.

    edit:
    Turns out, there's still something wrong. Last chunk is not recognized when the cam is set to continuous mode...


    late edit:
    test build removed
    « Last Edit: 04 / April / 2015, 12:44:28 by srsa_4c »

    *

    Offline nafraf

    • *****
    • 1255
  • Publish
    Re: CHDK PTP interface
    « Reply #1076 on: 20 / March / 2014, 14:30:04 »
    It was a bug. New (still incomplete) patch is attached, along with a new test build for that camera.
    Patch tested on A495, I did not find problems.

    *

    Offline srsa_4c

    • ******
    • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1077 on: 21 / April / 2014, 16:39:58 »
    Turns out, there's still something wrong. Last chunk is not recognized when the cam is set to continuous mode...
    It looks like this can't be solved easily. The code is working when in single shot mode. In continuous mode, there is simply nothing in fwt_data_struct that would indicate whether there will be further fwt invocations for a given file. Therefore, the current code always expects another invocation and causes every rs attempt to fail (and makes the cam lock up afterwards sometimes due to unexpected behaviour in filewritetask)*.
    "Solutions"
    - fail every rs attempt in continuous mode for these cams
    - add code and fw hooks to better deal with the issue (this could end up pretty hacky)
    - rewrite the code to mirror filesystem operations (ouch)

    Any thoughts on how to proceed?

    edit:
    * that's a different issue, the cause is not the CHDK fwt modification
    « Last Edit: 01 / May / 2014, 16:43:39 by srsa_4c »


    *

    Offline reyalp

    • ******
    • 9810
  • Publish
    Re: CHDK PTP interface
    « Reply #1078 on: 21 / April / 2014, 17:06:32 »
    In continuous mode, there is simply nothing in fwt_data_struct that would indicate whether there will be further fwt invocations for a given file.
    :(
    Thanks for working on this.

    Quote
    - fail every rs attempt in continuous mode for these cams
    Better than nothing I guess, but unfortunate, because rs in continuous is actually pretty useful for a some applications.

    Quote
    - add code and fw hooks to better deal with the issue (this could end up pretty hacky)
    Not sure what this entails? An example of what it looks like when it fails might be helpful.

    Quote
    - rewrite the code to mirror filesystem operations (ouch)
    Does that even help if we don't know when it's done? Maybe I'm not understand the the suggestion.

    Other ideas:
    Is the total size available somewhere else in memory so we can check if we are done? Or does seeking mess this up?
    Could we use a timeout to signal the last chunk? If we are waiting for a jpeg, what is the max time between one chunk completing and the next being available? It seems like it could potentially be pretty short.
    Reverse engineer the code that sends the chunks, there must be some loop or variable that knows when it's done.
    Don't forget what the H stands for.

    *

    Offline srsa_4c

    • ******
    • 3104
  • Publish
    Re: CHDK PTP interface
    « Reply #1079 on: 21 / April / 2014, 17:38:45 »
    Quote
    - add code and fw hooks to better deal with the issue (this could end up pretty hacky)
    Not sure what this entails? An example of what it looks like when it fails might be helpful.
    small file
    Code: [Select]
    con 19> rs
    rs_init
    rs_shoot
    get data 1
    rc file IMG_0197.jpg 1
    rc chunk get IMG_0197.jpg 1 0
    rc chunk size:321042 offset:nil last:false
    rc chunk get IMG_0197.jpg 1 1
    RemoteCaptureGetData: unexpected return code 0x2002
    WARNING: capture_get_data error rcgetchunk failed
    script wait time 1.5299
    ERROR: rcgetchunk failed
    large file, fwt (and its related tasks) locked after this
    Code: [Select]
    con 15> rs
    rs_init
    rs_shoot
    get data 1
    rc file IMG_0207.jpg 1
    rc chunk get IMG_0207.jpg 1 0
    rc chunk size:2271744 offset:nil last:false
    rc chunk get IMG_0207.jpg 1 1
    rc chunk size:2271232 offset:nil last:false
    rc chunk get IMG_0207.jpg 1 2
    rc chunk size:1473745 offset:nil last:false
    rc chunk get IMG_0207.jpg 1 3
    rc chunk size:16384 offset:nil last:false
    rc chunk get IMG_0207.jpg 1 4
    RemoteCaptureGetData: unexpected return code 0x2002
    WARNING: capture_get_data error rcgetchunk failed
    script wait time 1.5360
    ERROR: rcgetchunk failed
    etc...
    The code hasn't changed since my last attached patch.
    Quote
    Quote
    - rewrite the code to mirror filesystem operations (ouch)
    Does that even help if we don't know when it's done? Maybe I'm not understand the the suggestion.
    fwt doesn't seek in this camera generation. It opens and closes the file multiple times (either in append or in overwrite mode) when needed.

    Quote
    Other ideas:
    Is the total size available somewhere else in memory so we can check if we are done?
    As far as I know, fwt doesn't have information about the total file size. It may exist somewhere else, didn't try to check yet...
    Quote
    Could we use a timeout to signal the last chunk? If we are waiting for a jpeg, what is the max time between one chunk completing and the next being available? It seems like it could potentially be pretty short.
    A timeout was one of my ideas that I referred to as "hacky". Other "hacks" include signaling from the raw hook.
    I don't yet know how the "fix" for this issue should look.
    Quote
    Reverse engineer the code that sends the chunks, there must be some loop or variable that knows when it's done.
    I gave up on understanding "filescheduletask", it looks much more difficult than the filewritetask code.

     

    Related Topics