CHDK PTP interface

reyalp

14118

Re: CHDK PTP interface

« Reply #480 on: 20 / April / 2011, 12:53:15 »

Advertisements

mweerden, nice to see you working on this again.

Some comments below, I'll try to give the other issues a more detailed response when I have a bit more time.

Quote

actual live image dimensions (= bitmap dimensions?)

There is no simple, general relation between bitmap and live dimensions. The CHDK code is currently very ugly in the way it deals with live view dimensions. The best thing to do would be to clean this up before trying to build a PTP interface for it, but this would be a big job.

I wonder if you'd be better off with a new ptp function that's something like "get video frames" which takes flags specifying which items you want (bitmap, viewport, bitmap palette if we can get it) and returns a structured result. The camera side could notice when aspect and possibly palette changed, and automatically send updated information. edit: batching them all together should be more efficient.

Some discussion of viewport layout is in http://chdk.setepontos.com/index.php?topic=5415.0 and http://chdk.wikia.com/wiki/Frame_buffers

Quote

palette (either as a whole or just CAM_BITMAP_PALETTE)

The actual camera palette would be best, but requires find the relevant function for all cameras. Also, it changes at unpredictable times, so it wouldn't be clear when you'd need to update it. Doing it every frame would be wasteful, but as above maybe the camera could notice and only send it when requested or it changes.

Quote

I would also like to note that my experience with installing and figuring out how to draw an image with IUP has been quite frustrating. In the end I just piped the data to an external program. It definitely has resulted in my not being a fan of it. Just wanted to get that of my chest.

IUP documentation isn't great. My understanding is that it doesn't really support drawing directly to a window. To do this you should either use CD http://www.tecgraf.puc-rio.br/cd/ or the GL canvas, or roll your own function in C that draws directly the the hwnd or equivalent.

That said, I think the ability to easily receive lua values from the camera and use them directly is very high, and once you get used to it's oddities, it's pretty convenient for general gui stuff.

Quote

In include/platform.h some vid_* functions return long, some int. I don't think there is a reason, but I'd like to be sure

No point, long and int are identical. I would really like to get rid of all the longs from everything except possibly functions that are defined as 'long' in some external standard.

Quote

Proper serialisation in Lua looks nice, but also complicated/expensive. And it would indeed bump up the PTP version. Perhaps an option is to leave implicit serialisation as it is now and just provide a proper serialise function for Lua programmers to use if they want to communicate more complex things (as is the case in most languages I believe).

My current thinking is to add an option to lua exec, which uses serialize for everything (except maybe strings, because escaping them does add quite a bit of overhead for not much value.) The serialize function really isn't very complicated (currently 100 lines in chdku), and the overhead for simple values should be negligible compared to ptp overhead. It doesn't handle cyclic tables or code, but there's not much need for those. Handling cyclic tables wouldn't be that hard if you needed to, and you can just override the serialize function with a smarter one.

Sending the values as lua leaves them quite human readable.

Logged

Don't forget what the H stands for.

mweerden

194

Re: CHDK PTP interface

« Reply #481 on: 21 / April / 2011, 06:25:26 »

Quote from: reyalp on 20 / April / 2011, 12:53:15

The CHDK code is currently very ugly in the way it deals with live view dimensions. The best thing to do would be to clean this up before trying to build a PTP interface for it, but this would be a big job.

And that's why it's not what we should do.

In my opinion having remote live preview far outweighs it being optimal (or perhaps even working) on each camera. We basically have it ready on a shelf waiting and it's a shame not to use it. I think in the current situation we can make something that works well in almost all cases. If we later find a much better but incompatible method, we'll just have to do a major++ and that's that.

Quote

Quote
actual live image dimensions (= bitmap dimensions?)
There is no simple, general relation between bitmap and live dimensions.

Reading the post you referred to I feel the need to clarify that with "actual live image" I mean the "display area" or the image as it should be shown to the user. Perhaps presentation image/dimensions is a clearer term.

That said, from this statement (and the other post) I get that in general the bitmap dimensions need not be the same as the presentation dimensions. So at this moment there is no way to actually get the latter dimensions.

As far as I can tell, however, it seems that for most cameras we can use the bitmap dimensions. Or perhaps even just default to 360x240. We can add vid_get_presentation_width/height to generic/wrappers.c with either of these choices such that one can easily override it whenever someone figures out it doesn't work for them.

Side question: are there cases where it is not 360x240? And even not 3:2? (As long as it is 3:2 it will just show up a bit smaller or bigger than it should, which isn't the end of the world.)

Quote

I wonder if you'd be better off with a new ptp function that's something like "get video frames" which takes flags specifying which items you want (bitmap, viewport, bitmap palette if we can get it) and returns a structured result. The camera side could notice when aspect and possibly palette changed, and automatically send updated information. edit: batching them all together should be more efficient.

Yeah, this is pretty much the "big send_details_and_buffers()" function I mentioned. Except I would not make it part of CHDK PTP on account of separation of concerns. (Although that means you'll have to either get the function pointer first or send the data as script result.)

Quote

Some discussion of viewport layout is in http://chdk.setepontos.com/index.php?topic=5415.0 and http://chdk.wikia.com/wiki/Frame_buffers

Thanks for that. I searched using the function names and didn't come up with anything like this.

As for the remark about the mismatch of dimensions w.r.t. the S90 and G11: I currently have a DISKBOOT.BIN where viewport_width is 360 and a PS.FI2 where it is 720. No noticeable difference with edge overlay. Very robust code I guess...

Quote

The actual camera palette would be best, but requires find the relevant function for all cameras. Also, it changes at unpredictable times, so it wouldn't be clear when you'd need to update it.

It changes? (Wouldn't expect that at all looking at the code.)

Quote

My current thinking is to add an option to lua exec, which uses serialize for everything (except maybe strings, because escaping them does add quite a bit of overhead for not much value.)

That's even better.

Logged

reyalp

14118

Re: CHDK PTP interface

« Reply #482 on: 21 / April / 2011, 12:55:03 »

Quote from: mweerden on 21 / April / 2011, 06:25:26

It changes? (Wouldn't expect that at all looking at the code.)

The CHDK palette doesn't change. The canon one does (switching between play and rec for example, or opening the menu.) The CHDK palette should be selected such that it is readable in either canon palette, but the exact colors vary, and it doesn't cover anywhere close to the entire canon palette.

Note that ewavrs old live view code did get the canon palette, e.g. from a540:

Code: [Select]

unsigned int *vid_get_palette() 
{
    static unsigned pal[16];         
    unsigned int*syspal= (unsigned int*)0x634E0; // GetPaletteFromPhysicalScreen
    int i;
    for (i=0;i<16; i++) {  // big-endian to little-endian
    pal[i] = (syspal[i]>>24) | 
             ((syspal[i]<<8) & 0x00FF0000) |
             ((syspal[i]>>8) & 0x0000FF00) |
             (syspal[i]<<24);
    }
    return pal;
}

As for the viewport dimensions:
1) The CHDK code is confusing, because the actual buffer is 4 Y units for each U and V. CHDK splits the difference so "360x240" in lib.c is actually 720 in Y. 360x240 gives square pixels.
2) On some cameras (e.g. SD990, D10, SD1200...) the bitmap pixels aren't square either. All of these have 720x240 (non-square pixel) bitmap displays, but the lib.c code still says 360x240, and correction is done elsewhere. Not all of them are a simple 2x, see the CAM_ASPECT_CORRECTION nonsense in */platform_camera.h but beware the comments have been copy pasted!
3) On some cameras, the height of the live view varies slightly between record and play mode (e.g. a570)
4) some cameras have buffers that are bigger than the display area. Can also apply to bitmap buffer (e.g. tx1)
5) some cameras are switchable between 16:9 and 4:3. The display on the SD4000 is actually 16:9, while some others (e.g. G12) are 4:3.

The great difficulty with all this is that without the particular camera, it's quite hard to know what it's actually doing.

edit to add:
In spite of all the above, all the necessary information should be available, since CHDK can display and do edge overlay correctly. It's just a matter of gathering all the information in a way that doesn't drive you insane...

Quote

As for the remark about the mismatch of dimensions w.r.t. the S90 and G11: I currently have a DISKBOOT.BIN where viewport_width is 360 and a PS.FI2 where it is 720. No noticeable difference with edge overlay. Very robust code I guess...

~~Do histogram, zebra and edge overlay work correctly on both ?~~
Oops, you say edge overlay works. I'm guessing they are just using a different value there, but like I said, it's a mess.

« Last Edit: 21 / April / 2011, 13:05:58 by reyalp »

Logged

Don't forget what the H stands for.

mweerden

194

Re: CHDK PTP interface

« Reply #483 on: 21 / April / 2011, 17:26:05 »

Hmm... It seems that, just as in life, the more you know, the more it sucks.

Quote from: reyalp on 21 / April / 2011, 12:55:03

... so "360x240" in lib.c is actually 720 in Y. 360x240 gives square pixels.

Just to make sure I absolutely understand correctly: the value vid_get_viewport_width() returns is not the actual width of the image in the buffer (or the number of Y's, if you want)?

I think it is key to first figure out what information we precisely need, without looking at what is available (in CHDK). For every thing we don't have (or just not reliable or easy enough) we should just introduce new functions (e.g. vid_get_viewport_width_proper()) with some default implementation. Getting the palette? Just return a palette with completely transparent colours. But, of course, preferably default implementations that works for most cameras.

One thing I keep wondering about is what happens with these changing viewport dimensions. The bitmap dimensions seem to be constant, right? So that seems to suggest that the presentation(/display area) dimensions are constant as well. If so, and if the unused pixels in the viewport buffer are set to black by the camera (which I'm hoping), doesn't that mean that we don't have to concern ourselves with the whole changing dimensions and aspect ratio business? Can't we just use the buffer as if the viewport dimensions are maximal?

Of course, there might be interest in having the live image without overlay with the border stripped. However, it does become optional.

Logged

reyalp

14118

Re: CHDK PTP interface

« Reply #484 on: 22 / April / 2011, 00:42:39 »

Quote from: mweerden on 21 / April / 2011, 17:26:05

Hmm... It seems that, just as in life, the more you know, the more it sucks.

Quote from: reyalp on 21 / April / 2011, 12:55:03
... so "360x240" in lib.c is actually 720 in Y. 360x240 gives square pixels.
Just to make sure I absolutely understand correctly: the value vid_get_viewport_width() returns is not the actual width of the image in the buffer (or the number of Y's, if you want)?

Right, on the classic cameras where the viewport is "360x240" there are actually 720 y values. Histogram, zebra, MD etc, just ignore half of them. This made things easy on these cameras, because the bitmap was actually 360x240.

Quote

One thing I keep wondering about is what happens with these changing viewport dimensions. The bitmap dimensions seem to be constant, right?

I'm nut sure what the bitmap does on cameras with widescreen modes... I don't own one

Quote

I think it is key to first figure out what information we precisely need, without looking at what is available (in CHDK).

Agreed. Looking at ewars CHDKCAM and corresponding CHDK patches may also be useful. He addressed a lot of these issues (by adding new functions) but the cameras permutations have also become more complicated.

I would say
bitmap dismensions and aspect ratio (cameras like D10 do not have square pixels).
palette - you'll want to get it from the cameras eventually, but you can just use a default one when it's not available. You could use CAM_BITMAP_PALETTE (probably pulled out of the source code, not returned by the camera) to populate some of it.

For the viewport, I'm less sure because I'm not sure I understand how all permutations actually work. But you can start with whatever camera you have, and make the owners of the funny ones implement their own

Quote

So that seems to suggest that the presentation(/display area) dimensions are constant as well. If so, and if the unused pixels in the viewport buffer are set to black by the camera (which I'm hoping),

Worth experimenting, but I wouldn't count on it. My guess is the canon video hardware is doing the overlay stuff much later.

Logged

Don't forget what the H stands for.

mweerden

194

Re: CHDK PTP interface

« Reply #485 on: 22 / April / 2011, 06:12:02 »

Quote from: reyalp on 22 / April / 2011, 00:42:39

Looking at ewars CHDKCAM and corresponding CHDK patches may also be useful. He addressed a lot of these issues (by adding new functions) ...

Yeah, but until our discussion I had no idea what these "_for_ptp()" functions were about. Looking at it now it has become a lot clearer.

Thanks very much for your detailed information, reyalp. I'm pretty confident that I can now make a nice patch.

Logged

reyalp

14118

Re: CHDK PTP interface

« Reply #486 on: 24 / April / 2011, 17:35:53 »

protocol update: http://chdk.setepontos.com/index.php?topic=2509.msg65421#msg65421

I've decided to hold off changing the format of returned tables, since overriding it in lua is quite convenient for development anyway (if you have code to send additional lua with every call, like chdkptp does).

Logged

Don't forget what the H stands for.

Bernd R

430

Re: CHDK PTP interface

« Reply #487 on: 28 / April / 2011, 15:07:17 »

I have test Ix 1000 HS with 1154 build and chdkptp.

I guess when i click on files tab and i call refresh, the directory should show.But camera show only in log short some "finished ***" messages and nothing happen

console output is this.i test several times

"""""
tree right click: fullpath A/
old state COLLAPSED
populate branch 0 A/
tree right click: fullpath A/
old state COLLAPSED
populate branch 0 A/
tree right click: fullpath A/
old state COLLAPSED
populate branch 0 A/
""""""

reboot button work, but it do not start chdk when sd card is not write protect.

shutdown button do not shutdown.there is only "finished ***" on cam log show.when i remove USB cable, i am not able to switch off camera.i need remove battery.

same happen when i do after a connect and upload file or press play.camera is not dead, show finish on every action, but it can not switch off with the buttons.when remove USB cable canon keys work all,but i am not able to switch to record mode, even if no USB cable is in.

do a disconnect before remove of USB cable too not help to switch camera off.

more tests i do:

when i click on files tab upload of file work.

the switch to record mode crash this camera as before.lens move out and after that, camera crash.seem this camera is not able to switch to record mode

« Last Edit: 28 / April / 2011, 15:09:22 by Bernd R »

Logged

Ixus 1000 HS

reyalp

14118

Re: CHDK PTP interface

« Reply #488 on: 28 / April / 2011, 15:59:32 »

Thanks for testing CHDKPTP.

This belongs in the chdkptp thread: http://chdk.setepontos.com/index.php?topic=6231.0 if any mods want to move it.

Quote from: Bernd R on 28 / April / 2011, 15:07:17

I have test Ix 1000 HS with 1154 build and chdkptp.

I guess when i click on files tab and i call refresh, the directory should show.But camera show only in log short some "finished ***" messages and nothing happen

You need to double click to expand the tree. This automatically re-freshes. When I said the file browser was very alpha, I wasn't joking

Quote

reboot button work, but it do not start chdk when sd card is not write protect.

What exactly did you expect to happen ?

With script you can boot to a specific file (PS.FI2 or *unencoded* disboot.bin) but I'm not sure how you'd expect that to be done automatically. Reboot without a file just reboots the camera, which obviously goes through the normal canon boot process. Adding a file selection option (boot to...) is a good idea, I'll add it to the planner.

Quote

shutdown button do not shutdown.there is only "finished ***" on cam log show.when i remove USB cable, i am not able to switch off camera.i need remove battery.

Sounds like shutdown script function is broken in your port.

Quote

same happen when i do after a connect and upload file or press play.camera is not dead, show finish on every action, but it can not switch off with the buttons.when remove USB cable canon keys work all,but i am not able to switch to record mode, even if no USB cable is in.

do a disconnect before remove of USB cable too not help to switch camera off.

Sounds like: http://chdk.setepontos.com/index.php?topic=4338.msg64324#msg64324

I have not yet implemented this workaround in CHDKPTP. I think it should be a function e.g. "unlock_keys_for_usb()" with the default implementation sending that message, not hardcoded in PTP client.

Quote

the switch to record mode crash this camera as before.lens move out and after that, camera crash.seem this camera is not able to switch to record mode

Sounds like your ports implementation of mode switching is broken.

Logged

Don't forget what the H stands for.

mweerden

194

Re: CHDK PTP interface

« Reply #489 on: 04 / May / 2011, 12:19:32 »

At GitHub you can now find my patch for remote live preview as well as a CHDK PTP .NET library (+test app). (Direct link to the patch: chdk.patch)

For the .NET library you'll need the LibUsbDotNet DLL. With VS2010 you can just put in the "root" (i.e. where the .sln file is). Currently the library is for .NET 4.0, but it is pretty easy to make it work with 3.5. The library consist of a three layers: PTP, CHDK PTP and CHDKPTPRemote. The latter provides an easy interface with the basic functionality. Note that I haven't implemented all CHDK PTP functionality (and definitely not all PTP functionality).

As for the patch: I've adapted the CallFunction command such that you can call functions that return data via PTP and added a function that can send a combination of the buffers and additional dynamic info. The address of the function together with static information can be obtained with Lua using the get_video_details() function.

Regarding the viewport buffer and different aspect ratios: I'm told that on the G12 the buffer simply contains black borders. On my IXUS870IS I can select a "wide" format, but that doesn't affect the viewport buffer. The black borders are added by drawing them in the bitmap.

Edit: I've made a separate branch for the remote live preview stuff. The master branch should be compatible with the current CHDK trunk (i.e. CHDK PTP v2.0).

« Last Edit: 06 / May / 2011, 10:29:02 by mweerden »

Logged