CHDKPTP: for some - a sophisticated work - page 2 - General Discussion and Assistance

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #10 on: 13 / February / 2019, 09:37:40 »

Advertisements

In a first attempt to work, end-to-end, through a ptp command it's a little tricky for me "C" wise (especially having just briefly glimpsed the ioctl stuff wondering how that interacts with the usb host controller).

A couple of wonders at the very start:

1.
I can't see where/how (741) ptp.c (https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.c) is aware of (741) https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdk_headers/core/ptp.h ?

2.
We declare (741) https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdk_headers/core/ptp.h#ln32

Code: [Select]

enum ptp_chdk_command {
PTP_CHDK_Version = 0,     // return param1 is major version number
                                       // return param2 is minor version number
.
.
};

Having looked at a MS C Enumeration Declarations article, my imagination was that the related assignment would include the enum keyword, something like:

Code: [Select]

enum ptp_chdk_command ptp.Param1;
ptp.Param1 = PTP_CHDK_Version;

Instead we have (741) https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.c#ln2268:

Code: [Select]

ptp.Param1=PTP_CHDK_Version;
Is there an easy way to understand how that works without use of the enum keyword?

(struct member Param1 is originally defined here (741) https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.h#ln42 as unit32_t)

« Last Edit: 13 / February / 2019, 13:19:33 by andrew.stephens.754365 »

Logged

reyalp

11998

Re: CHDKPTP: for some - a sophisticated work

« Reply #11 on: 14 / February / 2019, 00:14:47 »

Quote from: andrew.stephens.754365 on 13 / February / 2019, 09:37:40

In a first attempt to work, end-to-end, through a ptp command it's a little tricky for me "C" wise (especially having just briefly glimpsed the ioctl stuff wondering how that interacts with the usb host controller).

A couple of wonders at the very start:

1.
I can't see where/how (741) ptp.c (https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.c) is aware of (741) https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdk_headers/core/ptp.h ?

chdkptp/ptp.h includes core/ptp.h, and chdk_headers is added to the include path by default. The reason it's set up this way is that for protocol development, it's convenient to use *one* ptp.h for both CHDK and chdkptp, which can be accomplished by putting the CHDK source root on the include path instead. Putting it under core/ distinguishes it from the unrelated chdkptp/ptp.h

Quote

Is there an easy way to understand how that works without use of the enum keyword?

In C, enums are some kind of integer under the hood, and can be assigned to integers or passed to functions that expect integers without complaint.

Logged

Don't forget what the H stands for.

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #12 on: 14 / February / 2019, 04:24:58 »

Thanks. I suspect there will be more of those as this thread develops but will not write them all down in order to improve readability.

Logged

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #13 on: 04 / March / 2019, 07:14:56 »

The simple as possible command chosen for this end-to-end walkthrough is ptp_chdk_get_version(). It is definitely not simple, for me.

Quote from: reyalp on 17 / January / 2014, 17:02:01

It maintains a separate PTPParams and PTP_CON_STATE (equivalent to PTP_USB in the ptpcam code) for each connection.

A:
~~I can easily see the PTP_USB typedef in ptpcam.h but am (a bit embarrassed) not able to find the equivalent PTP_CON_STATE struct definition in the source. A link to that would be appreciated.~~

Oops, PTP_CON_STATE found at https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptpcam.h#ln74 (448)

B:
In https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdkptp.c#ln247

Code: [Select]

result=USB_BULK_WRITE(ptp_cs->usb.handle,ptp_cs->usb.outep,(char *)bytes,size,ptp_cs->timeout);
I think usb.handle here contains the address of a usb_dev_handle struct
(Edit: further to above discovery at point A. https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptpcam.h#ln40

Code: [Select]

usb_dev_handle* handle;)
and, for purposes of attempting to further read the code, I have assumed that struct to be equivalent to the description here https://alioth-lists.debian.net/pipermail/libhid-discuss/2007-March/000114.html i.e.

Code: [Select]

-/* NOTE: included from libusb/usbi.h. UGLY, i know, but so is libusb! */
-struct usb_dev_handle {
-  int fd;
-  struct usb_bus *bus;
-  struct usb_device *device;
-  int config;
-  int interface;
-  int altsetting;
-  void *impl_info;
-};
+struct usb_dev_handle;

but I can't find that directly in the libusb source. A link to that definition would be appreciated.

C:
In libusb https://sourceforge.net/p/libusb-win32/code/HEAD/tree/trunk/libusb/src/windows.c#l373

Code: [Select]

usb_context_t **c = (usb_context_t **)context;
I can't find the source of the usb_context_t struct. A link to that definition could also be useful.

« Last Edit: 04 / March / 2019, 13:51:37 by andrew.stephens.754365 »

Logged

reyalp

11998

Re: CHDKPTP: for some - a sophisticated work

« Reply #14 on: 04 / March / 2019, 13:26:25 »

Quote from: andrew.stephens.754365 on 04 / March / 2019, 07:14:56

A:
I can easily see the PTP_USB typedef in ptpcam.h but am (a bit embarrassed) not able to find the equivalent PTP_CON_STATE struct definition in the source. A link to that would be appreciated.

PTP_CON_STATE starts at line 61, just below PTP_USB and PTP_TCP.

I highly recommend using a grep tool to find things like this. I frequently use https://tools.stefankueng.com/grepWin.html

Quote

I think usb.handle here contains the address of a usb_dev_handle struct and, for purposes of attempting to further read the code, I have assumed that struct to be equivalent to the description here
...
but I can't find that directly in the libusb source. A link to that would be appreciated.

A handle is typically intended to be opaque. While it might (or might not) actually be a pointer to a struct under the hood, that shouldn't matter to applications that use it.

Quote

C:
In libusb https://sourceforge.net/p/libusb-win32/code/HEAD/tree/trunk/libusb/src/windows.c#l373
Code: [Select]
usb_context_t **c = (usb_context_t **)context;
I can't find the source of the usb_context_t struct. A link to that could also be useful.

FWIW, I'm not clear why you are digging around in the guts of libusb. This shouldn't be necessary to understand how the CHDK protocol works or to write libusb applications. You should just need the API documentation. Without going digging, I don't have any idea where these things are, because I've never had a need to look at them.

If you do want to learn about that, I'd suggest downloading a copy of the libusb source and grepping. That's what I'd have to do to answer these...

edit:
I should add if you are doing this just for your own interest, that's totally fine of course. I'm only suggesting it's unnecessary if you are trying to do something specific with chdkptp.

« Last Edit: 04 / March / 2019, 14:39:10 by reyalp »

Logged

Don't forget what the H stands for.

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #15 on: 04 / March / 2019, 14:37:11 »

Quote from: reyalp on 04 / March / 2019, 13:26:25

Without going digging, I don't have any idea where these things are, because I've never had a need to look at them.

Understood.

I'm attempting to get a comfort feeling, from as many angles as possible, with USB/PTP operation including e.g. how the data queues are formed in and managed by the host controller.

If possible, having some sort of basis to approximate the various parts of how things work (tens, hundreds, thousands of usec) is desired to, perhaps, better address things like https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/lua/multicam.lua#ln613 (841):

Code: [Select]

-- msend average time to complete a send, accounts for a portion of latency
-- not clear what this includes, or how much is spent in each direction

So, if I still think you can comment, at a conceptual level, against some of the libusb stuff i'll post that in case you might have a useful insight.

Thanks for the other comments.

Logged

reyalp

11998

Re: CHDKPTP: for some - a sophisticated work

« Reply #16 on: 09 / March / 2019, 21:06:15 »

Quote from: andrew.stephens.754365 on 04 / March / 2019, 14:37:11

Code: [Select]
-- msend average time to complete a send, accounts for a portion of latency -- not clear what this includes, or how much is spent in each direction
So, if I still think you can comment, at a conceptual level, against some of the libusb stuff i'll post that in case you might have a useful insight.

Thanks for the other comments.

FWIW, given the 10s of ms uncertainty inherent in the CHDK side, I'm not convinced there's much value chasing usec elsewhere.

When you run init_sync it shows the time to send the tick message, which for my PC with modern (digic 4 and later) cams is ~1ms. This represents a full round trip at the USB level, since the transaction doesn't complete until it receives status back over the wire from the camera.

I think we've previously discussed the big sources of uncertainty on the CHDK / script side.

Logged

Don't forget what the H stands for.

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #17 on: 17 / March / 2019, 11:06:29 »

Quote from: reyalp on 09 / March / 2019, 21:06:15

I'm not convinced there's much value chasing usec elsewhere.

Very probably.

Still pretty challenged by C while trying to get a basic understanding of the no data phase ptp_chdk_get_version()

In https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdkptp.c#ln1293

Code: [Select]

static int chdk_camera_api_version(lua_State *L) {
     CHDK_CONNECTION_METHOD;
     CHDK_ENSURE_CONNECTED;
     int major,minor;

     api_check_ptp_throw(L,ptp_chdk_get_version(params,&major,&minor));

     lua_pushnumber(L,major);
     lua_pushnumber(L,minor);
     return 2;
}

where does the "params" argument of api_check_ptp_throw come from?

In https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdkptp.c#ln811

Code: [Select]

/*
get pointers out of user data in given arg
*/

static void get_connection_data(lua_State *L,int narg, PTPParams **params,PTP_CON_STATE **ptp_cs) {

    *params = (PTPParams *)luaL_checkudata(L,narg,CHDK_CONNECTION_META);
    *ptp_cs = (PTP_CON_STATE *)((*params)->data);
}

Is the following a correct interpretation?

ptp_cs is defined, in the parameter list, as a pointer to a pointer to a PTP_CON_STATE structure
* 'dereferences' ptp_cs to get the pointer to the PTP_CON_STATE structure.

"data" is an element of the PTPParams struct but it is an element that is declared as a pointer (i.e. https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.h#ln819

Code: [Select]

void *data;)

So, this:

Code: [Select]

((*params)->data)
is a pointer to a pointer to a PTP_CON_STATE struct.

Now, the "*" of:

Code: [Select]

(PTP_CON_STATE *)
acts to dereference the pointer to the pointer to the PTP_CON_STATE struct with result that:

Code: [Select]

(PTP_CON_STATE *)((*params)->data);
is a pointer to a PTP_CON_STATE struct

?

FMI:

ptp_chdk_get_version() (returns '1' if success,otherwise '0')
ptp_transaction() (returns 0x2001 if success)
ptp_usb_sendreq() (if fail return I/O error code otherwise code returned by ptp_write_func())
ptp_write_func() (returns PTP_RC_OK or PTP_ERROR_IO code)
usb_bulk_write() (return value returned by _usb_transfer_sync())
_usb_transfer_sync() (returns _usb_setup_async() error number or bytes
transferred)
_usb_setup_async() (returns error number or '0' on success)

From _usb_transfer_sync() , usb_submit_async() repeatedly called until all
bytes sent :-

usb_submit_async() (returns '0' on success or Windows error number
converted to libusb's own error number)
usb_reap_async() (returns bytes transferred or Windows error number
converted to libusb's own error number)

Quote from: reyalp on 24 / March / 2019, 17:59:10

I don't see anything obvious in chdkptp that would depend on whether the size was >= 18*512. The chdkptp code does *not* break writes in to 512 byte chunks. It does send the null packet when the total is a multiples of 512. This logic is unchanged from ptpcam.

« Last Edit: 26 / March / 2019, 17:57:17 by andrew.stephens.754365 »

Logged

reyalp

11998

Re: CHDKPTP: for some - a sophisticated work

« Reply #18 on: 17 / March / 2019, 15:26:15 »

Quote from: andrew.stephens.754365 on 17 / March / 2019, 11:06:29

Quote from: reyalp on 09 / March / 2019, 21:06:15
I'm not convinced there's much value chasing usec elsewhere.

Very probably.

Still pretty challenged by C while trying to get a basic understanding of the no data phase ptp_chdk_get_version()

In https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdkptp.c#ln1293
Code: [Select]
static int chdk_camera_api_version(lua_State *L) { CHDK_CONNECTION_METHOD; CHDK_ENSURE_CONNECTED; int major,minor; api_check_ptp_throw(L,ptp_chdk_get_version(params,&major,&minor)); lua_pushnumber(L,major); lua_pushnumber(L,minor); return 2; }
where does the "params" argument of api_check_ptp_throw come from?

Evil macros at the top of chdkptp.c

Code: [Select]

// TODO this is lame
#define CHDK_CONNECTION_METHOD PTPParams *params; PTP_CON_STATE *ptp_cs; get_connection_data(L,1,&params,&ptp_cs);

// so is this
#define CHDK_ENSURE_CONNECTED if(!ptp_cs->connected) {push_api_error_ptp(L, PTP_ERROR_NOT_CONNECTED); return lua_error(L);}

I did this because the above is repetitive and required in all the normal connection methods. It's not a function, because each would still need to be wrapped in an if/return.

Quote

Is the following a correct interpretation?

ptp_cs is defined, in the parameter list, as a pointer to a pointer to a PTP_CON_STATE structure
* 'dereferences' ptp_cs to get the pointer to the PTP_CON_STATE structure.

"data" is an element of the PTPParams struct but it is an element that is declared as a pointer (i.e. https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/ptp.h#ln819
Code: [Select]
void *data;)

So, this:
Code: [Select]
((*params)->data)
is a pointer to a pointer to a PTP_CON_STATE struct.

Now, the "*" of:
Code: [Select]
(PTP_CON_STATE *)
acts to dereference the pointer to the pointer to the PTP_CON_STATE struct with result that:

Code: [Select]
(PTP_CON_STATE *)((*params)->data);
is a pointer to a PTP_CON_STATE struct

?

I think you are misinterpreting the casts in the above code.

The point of get_connection_data is to get the information needed by connection functions out of a lua user data.
params and ptp_cs are output parameters. In C, the way you do output parameters is to pass a pointer where the thing you want back should go. Since the things being passed back are themselves pointers, they are **.

So:

Code: [Select]

 *params = (PTPParams *)luaL_checkudata(L,narg,CHDK_CONNECTION_META);

Looks for a Lua user data at stack position narg of type CHDK_CONNECTION_META, erroring if not found.
The Lua userdata is a void pointer (pointer to anything), which is cast to PTPParams pointer, and stored in params (which is a pointer to a PTPParams pointer, because it's an output parameter)

Code: [Select]

 *ptp_cs = (PTP_CON_STATE *)((*params)->data);

gets the data member of the PTPParams pointed to by params (which again is a void pointer), casts it to a PTP_CON_STATE *, and stores it in ptp_cs (which is a pointer to a PTP_CON_STATE pointer)

Hope that helps.

Logged

Don't forget what the H stands for.

andrew.stephens.754365

Re: CHDKPTP: for some - a sophisticated work

« Reply #19 on: 26 / March / 2019, 07:43:41 »

Sorry this is taking so long, but I need to make sure i've got it, mostly at least, right before starting to look at the ptp_chdk_write_script_msg & ptp_chdk_read_script_msg commands in parallel with how you conceived of chdkptp here https://chdk.setepontos.com/index.php?topic=4338.msg58684#msg58684.

Quote from: reyalp on 17 / March / 2019, 15:26:15

So:
Code: [Select]
*params = (PTPParams *)luaL_checkudata(L,narg,CHDK_CONNECTION_META);Looks for a Lua user data at stack position narg of type CHDK_CONNECTION_META, erroring if not found.

I haven't studied that part of the code yet but assume that is accomplished by

Code: [Select]

luaL_checkudata(L,narg,CHDK_CONNECTION_META)

Quote from: reyalp on 17 / March / 2019, 15:26:15

The Lua userdata is a void pointer (pointer to anything), which is cast to PTPParams pointer

Ok (thanks for the link), that's this part:

Code: [Select]

(PTPParams *)

Quote from: reyalp on 17 / March / 2019, 15:26:15

and stored in params (which is a pointer to a PTPParams pointer, because it's an output parameter)

That's this part:

Code: [Select]

*params =
So, via PTPParams **params, we passed a pointer to a pointer to a PTPParams struct into function get_connection_data() and name it as "params" within that.

But,

surely the "*" of:

Code: [Select]

*params =
signifies a single dereference of params (our pointer to pointer to a PTPParams) with result that the storage you refer to is not to "params" (i.e. our pointer to pointer to a PTPParams) but rather to the same address as the single pointer to PTPParams (so happens to be named "params") within the calling macro here https://app.assembla.com/spaces/chdkptp/subversion/source/HEAD/trunk/chdkptp.c#ln168:

Code: [Select]

#define CHDK_CONNECTION_METHOD PTPParams *params; PTP_CON_STATE *ptp_cs; get_connection_data(L,1,&params,&ptp_cs);
In summary:

assigning the address to the single dereference of the double pointer within the function has the desired effect which is to correctly set the base address of the PTPParams structure within the single pointer outside of the function?

FMI
pointer-to-pointer

Quote

It's simple. If you need to modify a pointer (a variable of type type*) in another function then you must pass the variable by pointer, so if the type is type* (pointer), then the new type you would need the function to take is type**

« Last Edit: 26 / March / 2019, 14:27:41 by andrew.stephens.754365 »

Logged