CHDK PTP interface - page 99 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 94 95 96 97 98 [99] 100 101 102 103 104 ... 125   Go Down

CHDK PTP interface

  • 1240 Replies
  • 461969 Views
*

Offline reyalp

  • ******
  • 14035
Re: CHDK PTP interface
« Reply #980 on: 09 / June / 2013, 16:08:26 »
Advertisements
Quote from: srsa_4c on 09 / June / 2013, 15:06:44
]I think that too large contiguous blocks of naked asm functions cause this kind of error.
I agree. I suspect older versions of GCC are not smart enough to put pools between functions, while the new  ones are.
Quote
The code in filewrite.c and remotecap.c is still kind of messy. The most irritating part is the huge number of #ifdef's, but I have no idea how to get rid of those.
Yes, agreed. Splitting is only a first step. Somehow the ifdefs and variables need to be cleaned up, but it's not obvious how.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14035
Re: CHDK PTP interface
« Reply #981 on: 09 / June / 2013, 17:00:43 »
My current thinking for handling DNG

- DNG header is an option like raw and jpeg. This is independent from the CHDK menu DNG option, and at the protocol level, independent of whether raw is requested.
- No thumbnail is included, but header is generated normally (as if there were a 128x96 thumb). Client can create a thumb or blank data.
- Raw data is always returned in native byte order, no bad pixel patching etc
- Client is responsible for constructing a valid DNG if desired.
- If a sub-image is requested, the client can adjust the header, or pad the image to full size.

This keeps the options and protocol very simple, and minimizes overhead on the camera side. It requires more work in the client code, but it feels like the right way to do it.

Issues:
- Need to ensure the dng module gets loaded if needed, and preferably unloaded when no longer needed. It seems like the logical place to do this is when the option is set, but loading/unloading the module for very shot is undesirable.
- DNG version of the header? If the DNG option is independent of the menu, it seems like this should be set explicitly as well. It would be tempting to just pick one, but I think we might regret that later.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: CHDK PTP interface
« Reply #982 on: 09 / June / 2013, 18:39:11 »
Quote from: reyalp on 09 / June / 2013, 17:00:43
- DNG header is an option like raw and jpeg. This is independent from the CHDK menu DNG option, and at the protocol level, independent of whether raw is requested.
Makes sense, no separate state variable needed.
Quote
- No thumbnail is included, but header is generated normally (as if there were a 128x96 thumb). Client can create a thumb or blank data.
- Raw data is always returned in native byte order, no bad pixel patching etc
- Client is responsible for constructing a valid DNG if desired.
- If a sub-image is requested, the client can adjust the header, or pad the image to full size.
This will require work on the dng module - create_dng_header() is not suitable for this, unless it gets changed. Making a copy of it would again create maintainability problems.

Quote
This keeps the options and protocol very simple, and minimizes overhead on the camera side. It requires more work in the client code, but it feels like the right way to do it.
An additional downside is that whoever implements a new client, may not feel like messing with this - but at least headerless RAW will always be available.

Quote
Issues:
- Need to ensure the dng module gets loaded if needed, and preferably unloaded when no longer needed. It seems like the logical place to do this is when the option is set, but loading/unloading the module for very shot is undesirable.
The module's unload checker currently bases its decision on the two RAW related config values. What else should it check when rc is involved? Play mode? Timeout? USB presence?
Quote
- DNG version of the header? If the DNG option is independent of the menu, it seems like this should be set explicitly as well. It would be tempting to just pick one, but I think we might regret that later.
Bad pixel treatment* depends on this, so it does matter which version is chosen. I don't think that this(*) task belongs to the client side. You wrote above that "no bad pixel patching". Hmmm. That's DNG 1.3

The other way of completing RAW/DNG could be:
DNG and its version is sent as an option (these bits would need to be treated differently, not as another data type of course). This would require a ptp-local state variable (where should these be kept?). Leave create_dng_for_ptp() almost as it is now (byte reversal and badpixel patching would become optional). In case of partial raw, the client patches the header.
Logged
*

Offline reyalp

  • ******
  • 14035
Re: CHDK PTP interface
« Reply #983 on: 09 / June / 2013, 19:55:36 »
In chdkptp changeset 350 I've moved the handling of chunks into lua. This is in preparation for changing DNG, but the protocol isn't changed yet.

I would like to know if this works on the cameras that require seek in the jpeg chunks.

If someone with one of these cameras can show me the output of
Code: [Select]
set cli_verbose=2
rs -f=3
that would be helpful.

Quote from: srsa_4c on 09 / June / 2013, 18:39:11
This will require work on the dng module - create_dng_header() is not suitable for this, unless it gets changed. Making a copy of it would again create maintainability problems.
I think it will be fairly simple to make create_dng_header() take some options.

Quote
An additional downside is that whoever implements a new client, may not feel like messing with this - but at least headerless RAW will always be available.
and sample code will be available in chdkptp, though likely split between C and Lua which may not be clear to many users.

Quote
-The module's unload checker currently bases its decision on the two RAW related config values. What else should it check when rc is involved? Play mode? Timeout? USB presence?
Yes, this is the question. USB present seems like the most logical, but we really want to know if there is a USB connection, not USB power. A timeout that resets whenever a DNG remote shoot is requested should be fairly simple.

Quote
Bad pixel treatment* depends on this, so it does matter which version is chosen. I don't think that this(*) task belongs to the client side. You wrote above that "no bad pixel patching". Hmmm. That's DNG 1.3
My proposal is they get pure raw data + a DNG header, not DNG data. Since the client is responsible for reversing, it could just as easily patch badpixels too.

That said, I guess if client patches the badpixels, then a 1.3 would load just fine in most programs, and the opcodes would be just be ignored or have no effect.  So maybe just forcing 1.3 is fine.
Quote
The other way of completing RAW/DNG could be:
DNG and its version is sent as an option (these bits would need to be treated differently, not as another data type of course). This would require a ptp-local state variable (where should these be kept?). Leave create_dng_for_ptp() almost as it is now (byte reversal and badpixel patching would become optional). In case of partial raw, the client patches the header.
Yes, I thought about a few variants of this, but they all seemed to end up messy.
Logged
Don't forget what the H stands for.
*

Offline nafraf

  • *****
  • 1308
Re: CHDK PTP interface
« Reply #984 on: 09 / June / 2013, 20:04:39 »
Quote from: reyalp on 09 / June / 2013, 19:55:36
In chdkptp changeset 350 I've moved the handling of chunks into lua. This is in preparation for changing DNG, but the protocol isn't changed yet.

I would like to know if this works on the cameras that require seek in the jpeg chunks.

If someone with one of these cameras can show me the output of
Code: [Select]
set cli_verbose=2
rs -f=3
that would be helpful.
Code: [Select]
connected: Canon PowerShot A810, max packet size 512
con> set cli_verbose=2
con> rs -f=3
ERROR: not in rec mode
con 1> rec
con 2> rs -f=3
got name IMG_0042
rcgetfile IMG_0042.raw 1
rcgetchunk IMG_0042.raw 1
rcgetchunk size:1536 offset:nil last:false
rcgetchunk IMG_0042.raw 1
rcgetchunk size:36864 offset:nil last:false
rcgetchunk IMG_0042.raw 1
rcgetchunk size:24724224 offset:nil last:true
rcgetfile IMG_0042.jpg 0
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1042944 offset:17920 last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:1060864 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:993297 offset:nil last:false
rcgetchunk IMG_0042.jpg 0
rcgetchunk size:17920 offset:0 last:true
With -f=1 jpeg file is fine. 
With -f=2 raw is corrupted
With -f=3 raw is corrupted, jpeg is corrupted (magenta)
RAW was disabled, now it works.
« Last Edit: 09 / June / 2013, 20:27:16 by nafraf »
Logged
*

Offline reyalp

  • ******
  • 14035
Re: CHDK PTP interface
« Reply #985 on: 09 / June / 2013, 20:48:01 »
Thanks nafraf.

It turns out the raw disabled issue happens due to a module loading problem. The dng module will be autoloaded with
Code: [Select]
libdng->capture_data_for_exif();
but
Code: [Select]
int _module_can_unload()
{
    return (running == 0) && ((conf.save_raw == 0) || (conf.dng_raw == 0));
}
will cause it to unload right away if raw isn't on in the menu.

So we need to address this issue anyway, even without my dng proposal.

Alternately, we could just require raw to be enabled, and fail init_remotecap if it isn't, but that seems pretty ugly. I wouldn't want to set it in the config there, since it would change the users saved values unless we also save and restore it...
Logged
Don't forget what the H stands for.
*

Online philmoz

  • *****
  • 3450
    • Photos
Re: CHDK PTP interface
« Reply #986 on: 09 / June / 2013, 21:16:09 »
Quote from: reyalp on 09 / June / 2013, 20:48:01
Thanks nafraf.

It turns out the raw disabled issue happens due to a module loading problem. The dng module will be autoloaded with
Code: [Select]
libdng->capture_data_for_exif();
but
Code: [Select]
int _module_can_unload()
{
    return (running == 0) && ((conf.save_raw == 0) || (conf.dng_raw == 0));
}
will cause it to unload right away if raw isn't on in the menu.

So we need to address this issue anyway, even without my dng proposal.

Alternately, we could just require raw to be enabled, and fail init_remotecap if it isn't, but that seems pretty ugly. I wouldn't want to set it in the config there, since it would change the users saved values unless we also save and restore it...

In the current raw code:
Code: [Select]
    if (conf.save_raw && conf.dng_raw && is_raw_enabled())
    {                             
        libdng->capture_data_for_exif();
    }

So capture_data_for_exif is only called when raw & dng are enabled - so module unloading doesn't happen and isn't an issue.

If you want to use raw & dng without enabling it in the normal manner, you need to provide some other flag to the dng module to stop it unloading.

You also need to take care of the modes where raw/dng doesn't work - currently handled by the is_raw_enabled() call above.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline nafraf

  • *****
  • 1308
Re: CHDK PTP interface
« Reply #987 on: 09 / June / 2013, 21:18:33 »
Two patches to add remote capture support for a1300 and a4000.
- a1300 (tested on A810 100d) - this includes small change to a810 boot.c, to use task_FileWrite name in taskHook()
- a4000 (tested on A4000 101b by rkomar)



Logged
*

Offline reyalp

  • ******
  • 14035
Re: CHDK PTP interface
« Reply #988 on: 09 / June / 2013, 21:57:13 »
Quote from: philmoz on 09 / June / 2013, 21:16:09
So capture_data_for_exif is only called when raw & dng are enabled - so module unloading doesn't happen and isn't an issue.
Right, and the remotecap variant is called in a remotecap.c remotecap_raw_savefile and doesn't do this, which is the problem.

On a semi-related note, the whole business of having the remotecap have it's own butchered variant of raw_savefile called from capt_seq also bothers me. raw_savefile does more than just saving, it's involved in bracketing, shot histogram, curves etc.  Not all of those make sense for remotecap, but it's an ugly duplication.
Quote
You also need to take care of the modes where raw/dng doesn't work - currently handled by the is_raw_enabled() call above.
That's a good point. Which leads to the question of whether remotecap raw should honor or ignore the "raw exceptions" settings. I would say no, but that means that is_raw_enabled() would need to be split into mandatory and preference conditions.

OTOH, if someone uses remotecap for raw and gets garbage, then maybe that's their problem...

edit:
Quote from: nafraf on 09 / June / 2013, 21:18:33
Two patches to add remote capture support for a1300 and a4000.
- a1300 (tested on A810 100d) - this includes small change to a810 boot.c, to use task_FileWrite name in taskHook()
- a4000 (tested on A4000 101b by rkomar)
Thanks. Added in changeset 2853.
« Last Edit: 09 / June / 2013, 22:02:45 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline hwntw

  • ***
  • 159
Re: CHDK PTP interface
« Reply #989 on: 10 / June / 2013, 06:26:41 »
I do my capturing hand held. I also rarely take 2000 RAW shots at a sitting so my savings of microseconds will not be great. Fascinated by these coding exercises, tho'.
Logged
Ixus 300HS S100


Windows 10
Pages: 1 ... 94 95 96 97 98 [99] 100 101 102 103 104 ... 125   Go Up
« previous next »
 

Related Topics