At the moment I'm working on remote live preview (or whatever it should be called) and I would like to do it in such a way that it can be added to CHDK permanently. So here's is my proposal (and related questions).
There essentially two extensions. One is to retrieve the various details such as image dimensions and buffer addresses. The other is to obtain a live image but also deal with the fact that buffer addresses are typically rotated.
For the latter I propose to add an flags parameter to the
PTP_CHDK_GetMemory command. The currently only flag (
0x1) will indicate that the first parameter is not a buffer address, but a pointer to function for obtaining the address. So if the flag is set, the camera will essentially first do "
addr = ((ftype) addr)();". This seems a nice minimal adjustment to the PTP code.
For the details (such as the function to use above), I propose to introduce a Lua function. This function (I called it "
get_video_details()") would either return a string containing a C struct with all the details or perhaps make a nice Lua table for it. The type of result probably depends on how often it will be called, but more on that later.
As for the details themselves, we need the following:
- actual live image dimensions (= bitmap dimensions?)
- (function to get) address of relevant viewport buffer
- width and height of viewport image
- width of viewport buffer (height shouldn't matter as the height of the image is all we need)
- x/y offset of viewport image in buffer
- (function to get address) of bitmap buffer
- width and height of bitmap image
- width of bitmap buffer (again no height)
- value or function to determine whether bitmap is shown
- palette (either as a whole or just CAM_BITMAP_PALETTE)
Now one question is how stable is this information and as a consequence how often do we retrieve it? I already assumed we would use a function instead of an address for getting the buffers, but if we get all of this information every time before we get a buffer that seems kind of pointless. The reason I assumed a function is because the other data is relative stable. Only when switching aspect ratio these seem to change. Perhaps it's better to do a less frequent check for such a change?
If details are "unstable" or we just decide to get them every time, perhaps it is a good idea to consider making a function that provides both the details and the buffer(s) together. To keep the PTP code relatively clean we could "export" some functions for sending data that can be used for the big
send_details_and_buffers() function. And in this case the proposed change to GetMemory has to be adapted to just call the function instead of using its result to send data. That or adapt CallFunction such that a flag indicates that the address is in Param2 instead of sent as data (to free up the data phase).
Another question is which actual dimensions one should use in displaying images and how the various components are composed in that image. This is not something that appears in CHDK code, but it is important in the use of it. It seems the bitmap image corresponds to the actual dimensions, but I not 100% sure. Do you just stretch the viewport image to those dimensions or should you add black borders to get the maximum viewport dimensions and stretch that?
If I interpret the comments of image_offset and row_offset correctly the image buffer actually already has black borders where needed and we can just always use the maximum viewport dimensions. This does mean some code needs to be added for some platforms to get this maximum. (Although at the moment this is just relevant for G12 en SX30, right?)
With respect to the palette I was thinking of just sending
CAM_BITMAP_PALETTE and let tools have the (currently) used palettes hard coded in them. This way you don't have to transfer it, but more importantly it seems that this is the only detail that currently isn't readily available.
Some small things:
- In include/platform.h some vid_* functions return long, some int. I don't think there is a reason, but I'd like to be sure.
- In image_offset and row_offset in platform/g12/lib.c I'm a bit confused about the "*3". Shouldn't it be 6 bytes per 4 pixels instead of 3 per 1?
- Also, shouldn't the version of row_offset in generic be buffer_width-width to be safe?
- I was also wondering where viewport_fb exactly fits in. It is used differently in various places: as default for record mode, as "slow" fallback in record mode, as fallback in any mode, in case of "KEY_SHOOT_HALF"...
- Looking at ewavr's old code I noticed that he introduced special _for_ptp functions (e.g. vid_get_bitmap_fb_for_ptp) that differ from their "non-ptp" variant. Not sure what's going on there. I would not expect this code to depend on using PTP or not.
Please let me know what you think about all this. And thanks for reading it all. You did read it all, didn't you?
To really conclude some side notes:
Proper serialisation in Lua looks nice, but also complicated/expensive. And it would indeed bump up the PTP version. Perhaps an option is to leave implicit serialisation as it is now and just provide a proper serialise function for Lua programmers to use if they want to communicate more complex things (as is the case in most languages I believe).
I would also like to note that my experience with installing and figuring out how to draw an image with IUP has been quite frustrating. In the end I just piped the data to an external program. It definitely has resulted in my not being a fan of it. Just wanted to get that of my chest.
On a more positive note, I've found a nice project:
LibUsbDotNet This provides a cross-platform .NET interface to libusb (and something similar(?) called WinUsb). I haven't really tested in on Linux yet, but it seems to work. I'm using C# to make a (CHDK-)PTP layer on top of it.
