CHDK PTP interface - page 119 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 114 115 116 117 118 [119] 120 121 122 123 124 125   Go Down

CHDK PTP interface

  • 1244 Replies
  • 535256 Views
*

Offline srsa_4c

  • ******
  • 4451
Re: CHDK PTP interface
« Reply #1180 on: 26 / April / 2019, 13:43:39 »
Advertisements
Thanks for the recv_ptp_data details.

I found something in disassemblies that might be related to the r31 problem.
There is a cpu cache library of some sort in the firmware. It has approx. 4 routines, the last one is unused in many cameras. Routine #3 (FF810750 in d10) is used in usb-related functions. The first routine is called from the MemoryChecker eventproc (that's one way to find them).
From a not too large sample, I found that an older version of #3 is used in
a470 (r23), g10 (r31). A newer version is used in ixus110 (r31 with problem), d10 (r31 with problem), a3200 (r47).
Now, I did not check whether the old and new #3 are doing exactly the same, but if they differ in operation that might be the reason behind the problem.
In case they do operate the same, this could still be used to identify the possible problem models, I think.
Logged
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1181 on: 26 / April / 2019, 17:10:24 »
Quote from: srsa_4c on 26 / April / 2019, 13:43:39
Now, I did not check whether the old and new #3 are doing exactly the same, but if they differ in operation that might be the reason behind the problem.
In case they do operate the same, this could still be used to identify the possible problem models, I think.
Nice catch. I had sort of assumed it was a hardware difference, but could easily be a difference in the code that handles the transfer. I'll take a closer look at those cache functions later.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1182 on: 27 / April / 2019, 01:12:09 »
Function 3 is something like
cache_control(cachetype,address,length)

If cachetype is 1, it jumps into the middle of the preceding function and flushes the instruction cache

If length is -1 or > 0x2000 (== 8k == size of the caches)
it disables interrupts, cleans and flushes the data caches by iterating index/segment, and then drains the write buffer.

Otherwise, it flushes and cleans the specified range (without disabling interrupts) and drains the write buffer.

In the < 0x2000 case, D10 uses separate instructions for clean and flush:
Code: [Select]
MCR     p15, 0, R1,c7,c10, 1
MCR     p15, 0, R1,c7,c6, 1
while g10 uses the combined instruction:
Code: [Select]
MCR     p15, 0, R1,c7,c14, 1
It seems like this difference should be cosmetic, but maybe there's a subtle difference with interrupts or something.

In the whole cache (length >0x2000 or -1) case, d10  the drains the write buffer for every iteration, before calling the flush and clean function  :blink:
Code: [Select]
MCR     p15, 0, R4,c7,c10, 4
MCR     p15, 0, R2,c7,c14, 2
G10 just does flush and clean. Both drain write buffer at the end. Draining for every iteration seems weird to me.

ELPH130 seems identical to the D10 code, while ELPH180 is quite different. It looks like they got rid of the cache type argument.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1183 on: 27 / April / 2019, 02:29:26 »
It occurred to me that the workaround in https://chdk.setepontos.com/index.php?topic=4338.msg140130#msg140130 is only unsafe at the ends, where it could potentially not be aligned on a cache line boundary.

To avoid this, we could allocate a small uncached buffer just for the ends.
1) Read the first little bit into the temp buffer
2) clean dcache
3) read main buffer + however much was read, up to the last full cache line, uncached,
4) clean and flush dcache
5) memcpy the first bit into the start of the buffer
6) read the last bit, memcpy that in.

This is annoyingly complicated to work around a corner case on a few cameras, but should be safe, negligible impact on performance or available memory, and completely contained in recv_ptp_data without requiring changes in the callers. For small transfers, it could just use the temp buffer and memcpy into whatever was passed to recv_ptp_data.

edit: or I guess transfers smaller than 0x23f4 could not do any cache workarounds, but but I'm somewhat hesitant to trust it.
« Last Edit: 27 / April / 2019, 02:43:55 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1184 on: 28 / April / 2019, 01:27:06 »
I implemented the idea in the previous post. It works great on d10... and fails miserably on a540.

Having platform specific versions would be OK, but it's hard to guess which they should be applied to.

On further investigation, a540 seems to have problems with multiple recv_data calls at certain sizes, and appears to have more problems that I haven't fully figured out when the memory is uncached, even when using umalloc and not doing any of the cache manipulation stuff.

Reading in blocks of 512, connection fails at sizes
1013-1015
1525-1527
etc
It only occurs on the second or later read, so with a 1024 block size, the first error happens at 1525

I tried changing the first read to 500 rather than 512, and the error occurred at the same points.

I haven't verified yet, but I expect the trunk code suffers from this, it just doesn't get hit often because it only affects multiple reads and certain magic sizes. The only time the trunk code does multiple reads is for file uploads that are > 1/2 max free block. On a540, diskboot.bin comes in less than that.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: CHDK PTP interface
« Reply #1185 on: 28 / April / 2019, 15:28:44 »
Quote from: reyalp on 28 / April / 2019, 01:27:06
The only time the trunk code does multiple reads is for file uploads that are > 1/2 max free block.
Perhaps it's time to see how the 100d handler is doing it? I tried uploading files that are larger than the fw's usb buffer, and it worked. No idea whether it suffers from similar problems though.
I think it's possible to remove that handler at runtime and add our own.

Side note: r54 and newer cams use a dedicated fw usb buffer instead of exmem.
Logged
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1186 on: 28 / April / 2019, 18:48:30 »
Quote from: srsa_4c on 28 / April / 2019, 15:28:44
Perhaps it's time to see how the 100d handler is doing it? I tried uploading files that are larger than the fw's usb buffer, and it worked
Yeah, that's definitely something to look at. I didn't try SendObject on A540 when I was playing with it earlier, because it doesn't list FIR files as objects, so it's not obvious what the ObjectInfo should contain. But I didn't dig into it much.

A540 (and likely other vxworks cameras) appear to have another, unrelated problem, present in the current trunk. Rapid fire messages like
Code: [Select]
!m=require'extras/msgtest'
!m.test{size=1,sizeinc=1,count=10000,gc='step'}
occasionally crash (often after 3000-5000 iterations, but variable, sometimes it goes 10k without problems)

I got a couple romlogs for this in memPartInfoGet, (LR = 0xffee1444, ultimately from core_get_free_memory in handle_ptp) which looks related detecting memory corruption. I wondered if memPartInfoGet / malloc / free need to be protected with a semaphore or something, but they don't appear to be in the Canon code, and the VxWorks documentation doesn't obviously suggest that they should be.

Trying to reproduce this today, I got failures
Code: [Select]
send 7445 size:7445 0x1d15...Protocol error: data expected
After one of those, the camera crashed with an "Assert Memory.c line 67" which appears to be from malloc returning 0.

It's unclear whether these are related to PTP, or two tasks rapidly malloc/freeing, or some other memory corruption.

Quote
I think it's possible to remove that handler at runtime and add our own.
Another thing I've been thinking about is hooking the ptp_data functions, either by hooking TrnsCtrlTask which sets them up, or just replacing them on the first ptp call (this does not work). My impression is that the ptp_data structure is set up once per session at most. This would allow logging what the stock operations do.
« Last Edit: 28 / April / 2019, 21:29:00 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1187 on: 28 / April / 2019, 20:43:49 »
Despite not listing FIR files, uploading format BF01 works on A540. Like newer dryos cameras, it asserts if the path includes "A/".

The following successfully creates a 4 MB TEST.DAT in the SD root.
Code: [Select]
!newf={ Filename="TEST.DAT", ObjectFormat=0xBF01, ObjectCompressedSize=1024*1024*4 } return con:ptp_send_object_info(newf)
!return con:ptp_send_object(('x'):rep(newf.ObjectCompressedSize))
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: CHDK PTP interface
« Reply #1188 on: 29 / April / 2019, 00:06:20 »
Quote from: srsa_4c on 28 / April / 2019, 15:28:44
I think it's possible to remove that handler at runtime and add our own.
This works. Attached patch hooks opcode 0x100d (SendObject) for A540.

The stock handlers get installed when when the USB cable is detected (PTPRspnd_StartUpPTPFrameworkClient) and removed when it is disconnected (PTPRspnd_ShutDownPTPFrameworkClient). So you have to replace it later after boot. It this case, I modified the rmem handler so rmem 1 4 installs the hook.

PTP session connect/disconnect and reset don't reset the handlers.

calling ptp_add_handler without removing first returns 0x13, but seems to still replace the handler.

On a540, the SendObject handler appears to have 256K buffer in uncached memory. Every call to recv_data gets the same address (0x11f80000). It does not appear to have the n*512 - 11 problem our code has.

Unrelated to the current problems:
Hooking the handlers (specifically GetObjectHandles) might provide us a way to avoid the black screen / no rec issue that happens when non-CHDK software access the cameras.

On a540 at least, GetObjectInfo and GetObject still work if GetOjectHandles hasn't been called, even in rec mode (the handles appear to just be sequential numbers starting at 2), which suggests the black screen mode isn't deeply tied to the rest of the PTP code.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: CHDK PTP interface
« Reply #1189 on: 29 / April / 2019, 18:32:40 »
Quote from: reyalp on 29 / April / 2019, 00:06:20
On a540, the SendObject handler appears to have 256K buffer in uncached memory. Every call to recv_data gets the same address (0x11f80000).
I experimented a bit on the a3200, and found that there are code paths using cached and uncached memory, so both are "legal" there. The uncached buffer is also 256k. The return value of recv_data is checked (bit0).
Quote
It does not appear to have the n*512 - 11 problem our code has.
How can I test this?
Logged
Pages: 1 ... 114 115 116 117 118 [119] 120 121 122 123 124 125   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal