I spent some more time trying to understand the issues on A540
The crash is "Assert BulkTrns.c line 1544", which happens in sub_FFE4DBDC
This is called from sub_FFE50B50, which comes from some table of BulkTrns related functions at FFE50BAC. The condition being checked in the assert isn't obvious to me.
The crash happens when
1) data->recv_data is called more than once
2) total size is (512*n) - 11 to (512*n) - 1 i.e. 0x...1f5 through 1ff.
Additionally, when using the native buffer (maybe any uncached) a stalled connection happens for something like 512*n to 512*(n+1)-13. Larger sizes appear to be OK, though I haven't checked exhaustively.
These problems happen even using native PTP operations, so
!newfi2={ Filename="BAD.DAT", ObjectFormat=0xbf01, ObjectCompressedSize=1024*256+512-11}
!return con:ptp_send_object_info(newfi2)
!return con:ptp_send_object(('x'):rep(newfi2.ObjectCompressedSize))
Triggers the assert, and size 1024*256+512 triggers the stalled connection, while 1024*257 works fine. I previously thought native operations weren't affected, but my test was incorrect. This is reproducible without CHDK running.
Note as I posted earlier some cameras require A/ on the path and crash with an different assert if it is not present. Others, including A540 (and likely all vxworks cameras) crash if it is present.
Also, although get_ptp_buf_size(4) returns 512K, the camera divides this value by 2 for transfers. I previously verified by hooking data->recv_data and logging the values that it's really doing 256K chunks. Based on the total ComMemMan size and other code, it seems likely the actual buffer is 512K, so why the firmware uses size/2 is unclear.
With CHDK, the problem will only occur if the total transfer is larger than 1/2 free memory and one of the magic sizes. Using a fixed 256K buffer, the following reproduce it
file upload
!fh=io.open('TEST.DAT','wb') fh:write(('x'):rep(1024*255+1525-14)) fh:close()
u test.dat
The -14 accounts for the file name and file name length.
Flushed script message (no script running):
!con:write_msg(('x'):rep(1024*255+1525))
The problem also occurs if the first read isn't a multiple of 512, e.g. if the first recv_data is 4000, it fails at 4085.
tl;dr
1) I'd be interested to see results on other vxworks cameras for the SendObject test.
2) This doesn't seem fixable for A540, and the current trunk style code doesn't seem worse than other options
