CHDK PTP interface - page 123 - General Discussion and Assistance - CHDK Forum

CHDK PTP interface

  • 1227 Replies
  • 271863 Views
*

Online reyalp

  • ******
  • 12209
Re: CHDK PTP interface
« Reply #1220 on: 08 / August / 2019, 18:37:16 »
Advertisements
send 461 size:53 0x35...ok
send 462 size:52 0x34...I/O error
aborted at send 462 size:52 0x34, communication error
Thanks.
So it fails at exactly 52 (64 - 12 overhead?) and not nearby multiples. Not sure what we could do about this. Could try two reads, like 48 and 4.

Quote
We could, although I'm not sure if it's worth it. The cameras only activate their wireless interface when the user enters the Wifi related dialogs. I find it hard to imagine that anyone (other than some 3-letter agencies and the like) will spend their time exploiting these firmware bugs. And only a very low percentage of cameras run CHDK.
Yeah, this doesn't seem a high risk. OTOH, someone making a stink about "CHDK undoes Canon security fix" could be unfortunate.

Thinking about it more, we could check the USB bit (in physw_status, not the hardware state) and only allow CHDK operations if its set. Since there's no working PTP/IP client for CHDK, having it off when USB is not present shouldn't be a problem (unless there's threading issues that make it drop out momentarily when kbd_task runs  :-[). This could also be an option: CHDK PTP: [USB only, USB+wifi, off]

If there's exposure when the camera is paired with a phone, that could be a bigger risk, but my impression is that's totally separate.

Quote
I posted the links because the descriptions mention the problematic PTP handlers by name. And that we'll get to see the fixes (M3 and M10 are on the list).
That could be interesting.
Don't forget what the H stands for.

*

Online reyalp

  • ******
  • 12209
Re: CHDK PTP interface
« Reply #1221 on: 22 / September / 2019, 16:59:27 »
In trunk 5274, I re-organized the code that uses send_data (camera->host transfers). This shouldn't affect the actual operations. It mostly cleans up the logic around the old global buf_size. Some needless complexity in send_ptp_data goes away, and core_get_free_memory is only called when a buffer is actually needed. This could be extended to use native ptp buffers like recv_data, but there currently doesn't seem to be need.

The main motivation is to simplify the code before trying to address issues with >2gb files reported in https://chdk.setepontos.com/index.php?topic=13896.0
Don't forget what the H stands for.

*

Online reyalp

  • ******
  • 12209
Re: CHDK PTP interface
« Reply #1222 on: 22 / September / 2019, 20:26:59 »
Here's my non-working attempt at a patch to make the CHDK code behave correctly for >2GB files.

Tested on sx710

camtests all pass, so sub 2 GB stuff seems fine.

Used a 2260826276 zip file to test.

Attempting to upload after running camtests resulted in IO Error on the chdkptp side. The camera appears to be running normally, but it is not possible to reconnect, even with reset_device.

I tried this twice. In one instance, powering off the camera seemed normal, but I was unable to power it on without pulling the battery. In another, the camera became unresponsive while still on.

Trying to upload on a fresh boot without running camtests first, I got an immediate "general error"

Copying the file to the SD card with a card reader trying to download in chdkptp, I get
read_data failed 0x2ff
ERROR: I/O error

The file from the failed download is 2145386496 bytes. This is 2^31 - 2 MB. chdkptp uses a 2MB buffer for file transfers, so it seems likely that the read request which hit 2^31 failed.

While it's quite possible there are bugs in my code, my suspicion is the Canon code uses signed values. I haven't tried standard ptp getobject or sendobject with a >2gb file yet.

If we want to support this, it would probably make more sense to add operations to do partial uploads and downloads: Read X bytes from offset, append X bytes or write X bytes at offset.

I'm not  sure this is worth doing. The case of downloading > 2gb files seems useful, but fairly rare.

There are additional issues in CHDK lua, since the stat size is negative for > 2gb files. chdkptp side code can work around this, but it would be on a case by case basis.

It may become more of an issue in the future, @srsa_4c suggested DryOS 59 extended the stat structure to support 64 bit sizes. >4 GB files would require a lot of changes, though probably not too common for CHDK until exFAT boot or multi-partition is supported.
Don't forget what the H stands for.

*

Offline srsa_4c

  • ******
  • 4055
Re: CHDK PTP interface
« Reply #1223 on: 24 / September / 2019, 17:25:13 »
It may become more of an issue in the future, @srsa_4c suggested DryOS 59 extended the stat structure to support 64 bit sizes. >4 GB files would require a lot of changes, though probably not too common for CHDK until exFAT boot or multi-partition is supported.
I don't think there is a PowerShot firmware based camera that records movies exceeding the 4GB barrier - but I can't be sure as this limitation is often omitted from various literature.


*

Online reyalp

  • ******
  • 12209
Re: CHDK PTP interface
« Reply #1224 on: 24 / September / 2019, 21:43:31 »
I don't think there is a PowerShot firmware based camera that records movies exceeding the 4GB barrier - but I can't be sure as this limitation is often omitted from various literature.
Agreed, my impression is that they don't. My thought was extending the stat structure might be a hint that it was coming in future models, though of course it could be for other products that use DryOS, and of course it's possible future models will not be supportable by CHDK anyway.

My current thinking on >2gb files is just to provide a clean error, probably GeneralError in the protocol, something like "file too large" in chdkptp in cases where it would know.
Don't forget what the H stands for.

*

Offline srsa_4c

  • ******
  • 4055
Re: CHDK PTP interface
« Reply #1225 on: 25 / September / 2019, 17:00:45 »
My current thinking on >2gb files is just to provide a clean error, probably GeneralError in the protocol, something like "file too large" in chdkptp in cases where it would know.
It's not like I see 2...4GB files very often, but they can be created on camera. I checked SendObject, it uses Open() rather than one of the fopen() variants. Download of >2GB files does work via libgphoto based software.
How about using open/read instead of fopen/fread when dealing with big files? I agree that files bigger than 4GB don't need to be supported (64bit versions of open etc. may not even exist in firmware).

*

Online reyalp

  • ******
  • 12209
Re: CHDK PTP interface
« Reply #1226 on: 26 / September / 2019, 00:14:29 »
How about using open/read instead of fopen/fread when dealing with big files? I agree that files bigger than 4GB don't need to be supported (64bit versions of open etc. may not even exist in firmware).
My impression was that the problems were in the PTP layer, not file IO, but I didn't dig deeply. If sendobject works, it might be worth investigating further.
Don't forget what the H stands for.

*

Offline srsa_4c

  • ******
  • 4055
Re: CHDK PTP interface
« Reply #1227 on: 28 / September / 2019, 14:34:41 »
My impression was that the problems were in the PTP layer, not file IO, but I didn't dig deeply. If sendobject works, it might be worth investigating further.
It's possible that you're right and the problem is indeed caused by transferring >=2GB of data in one go. I guess this could be addressed by transferring them in multiple pieces, but that would not work with clients not implementing that addition to the protocol (in case that's a problem).


 

Related Topics