CHIMP - Canon Hack Installation and Management Platform - page 9 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 4 5 6 7 8 [9] 10 11 12 13 14   Go Down

CHIMP - Canon Hack Installation and Management Platform

  • 136 Replies
  • 67951 Views
*

Offline reyalp

  • ******
  • 14000
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #80 on: 24 / June / 2017, 16:20:04 »
Advertisements
Quote from: dmitrys on 24 / June / 2017, 11:47:52
Added table support.

Source.
Note that for historical reasons, the default table format is a human readable format which doesn't unambiguously serialize arbitrary Lua tables. You can override this by defining your own  usb_msg_table_to_string in the code you send.

chdkptp uses a function which serializes them into Lua table definitions:
https://app.assembla.com/spaces/chdkptp/subversion/source/744/trunk/lua/rlibs.lua#ln130

Generating JSON or something like that instead should be relatively easy.

Logged
Don't forget what the H stands for.
*

Offline dmitrys

  • ****
  • 267
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #81 on: 25 / June / 2017, 09:04:02 »
Quote from: reyalp on 24 / June / 2017, 16:08:40
If you mean run a pure Lua zip library on the camera, in theory, yes. In practice, beware that cameras may not have enough free RAM to hold a CHDK zip in memory. Also CHDK Lua has some quirks and differences from the standard implementation: http://chdk.wikia.com/wiki/Lua#Lua_standard_libraries:_io.2C_os.2C_string.2C_and_math

I suppose it would be simpler (and probably faster) to just upload each file separately. If there were a way of creating a "universal executable" and having the camera reboot into it, that would be great (not only for CHIMP), but I guess that's not really feasible.

Quote
If the Lua module relies on binary modules, then it wouldn't work. Currently, CHDK Lua does not support loading flt modules like standard Lua supports dll/so modules. This is something I'd like to add some day.

Is that because there can be at most one library loaded and Lua occupies that spot?

How about adding a "load module" command directly to PTP?

Quote
You can get the camera model name from the USB layer, see https://app.assembla.com/spaces/chdkptp/subversion/source/744/trunk/chdkptp.c#ln1805

That's what CHDKPTPRemote uses for the Name property. It's always set to "Canon Digital Camera".

Edit: Checked DeviceInfo.ProductString.
« Last Edit: 25 / June / 2017, 10:19:23 by dmitrys »
Logged
Author of CHIMP, Canon Hack Installation and Management Platform
*

Offline dmitrys

  • ****
  • 267
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #82 on: 25 / June / 2017, 09:13:03 »
Quote from: reyalp on 24 / June / 2017, 16:20:04
Note that for historical reasons, the default table format is a human readable format which doesn't unambiguously serialize arbitrary Lua tables. You can override this by defining your own  usb_msg_table_to_string in the code you send.

Right now I have two tables that need deserializing, namely those returned by get_buildinfo() and get_partitionInfo(), and the existing format works just fine for these.

However, I'm having trouble with the values returned by the latter.

Regardless of the active partition, the partition type remains 1. Is that a bug or does "type" stand for something other than the MBR partition type?

Quote
Generating JSON or something like that instead should be relatively easy.

You mean in the client/server protocol? That would be nice (and I just implemented a JSON writer), but what about backward compatibility? Maybe add an optional "format" parameter?
Logged
Author of CHIMP, Canon Hack Installation and Management Platform
*

Offline reyalp

  • ******
  • 14000
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #83 on: 25 / June / 2017, 14:43:53 »
Quote from: dmitrys on 25 / June / 2017, 09:04:02
I suppose it would be simpler (and probably faster) to just upload each file separately. If there were a way of creating a "universal executable" and having the camera reboot into it, that would be great (not only for CHIMP), but I guess that's not really feasible.
I not sure what you mean here.

Quote
Quote
Currently, CHDK Lua does not support loading flt modules like standard Lua supports dll/so modules. This is something I'd like to add some day.
Is that because there can be at most one library loaded and Lua occupies that spot?
No, flt modules are relocated, it's just a matter of the relevant code not being implemented. The problem is that Lua C module code is designed around windows and *nix shared library APIs, and CHDK flt modules don't behave the same way. For example, any useful Lua C module needs to be able to call the Lua API, which resides in lua.flt, but the flt system doesn't provide a generic mechanism for module to module exports.


Quote
How about adding a "load module" command directly to PTP?
I don't understand what the purpose would be. In general, I'd like to keep the PTP API as simple as possible.

Quote
That's what CHDKPTPRemote uses for the Name property. It's always set to "Canon Digital Camera".
The name in chdkptp comes from PTP_OC_GetDeviceInfo and contains the actual name from the camera.

Quote
Regardless of the active partition, the partition type remains 1. Is that a bug or does "type" stand for something other than the MBR partition type?
When I want to know something like this, I find the call luascript.c, grep the source of the underlying function, and google/debug as needed. Not trying to be snarky, CHDK is big enough that I don't know everything off the top of my head, so this is what I end up doing when someone asks a question like this.

In this case, this leads to
platform/generic/wrappers.c get_part_type
Code: [Select]
    partType=mbr_buf[0x1C2+(get_active_partition()-1)*16];
https://en.wikipedia.org/wiki/Master_boot_record#Sector_layout
says the partition table is at 0x1be and the above code looks at 0x1c2 = 0x1be+4 (+ the offset for the partition number. Per https://en.wikipedia.org/wiki/Master_boot_record#PTE this is the offset of the partition type https://en.wikipedia.org/wiki/Partition_type

Quote
You mean in the client/server protocol? That would be nice (and I just implemented a JSON writer), but what about backward compatibility?
No, like chdkptp does. It overrides usb_msg_table_to_string on every call.
Logged
Don't forget what the H stands for.
*

Offline dmitrys

  • ****
  • 267
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #84 on: 25 / June / 2017, 15:24:57 »
Quote from: reyalp on 25 / June / 2017, 14:43:53
Quote from: dmitrys on 25 / June / 2017, 09:04:02
I suppose it would be simpler (and probably faster) to just upload each file separately. If there were a way of creating a "universal executable" and having the camera reboot into it, that would be great (not only for CHIMP), but I guess that's not really feasible.
I not sure what you mean here.

I'm thinking of something along the lines of the "unified installer" that's implemented in ML. Although it does other things, it would be really useful to have a very small executable that would extract the ZIP, optionally delete it, and reboot. It would only require the most basic I/O functions, namely:

fopen
fread
fseek
fwrite
fclose
mkdir
remove (not strictly required)
reboot

If there were such an unencoded main.bin, CHIMP could easily encode it for the target platform (if needed).

Quote
Quote
How about adding a "load module" command directly to PTP?
I don't understand what the purpose would be. In general, I'd like to keep the PTP API as simple as possible.

In the update scenario that would be invoking an unzip.flt.

Quote
Quote
That's what CHDKPTPRemote uses for the Name property. It's always set to "Canon Digital Camera".
The name in chdkptp comes from PTP_OC_GetDeviceInfo and contains the actual name from the camera.

Maybe I'm being snarky about this, but a search for PTP_OC_GetDeviceInfo in CHDK produces 0 matches.

Quote
https://en.wikipedia.org/wiki/Master_boot_record#Sector_layout
says the partition table is at 0x1be and the above code looks at 0x1c2 = 0x1be+4 (+ the offset for the partition number. Per https://en.wikipedia.org/wiki/Master_boot_record#PTE this is the offset of the partition type https://en.wikipedia.org/wiki/Partition_type

Right. The problem is both partitions show as type 1, so there is no way for me to know which partition is which, unless it's safe to assume that 1 is always the small one. Is it?

P.S. I believe there lies the problem:

Quote
Code: [Select]
    partType=mbr_buf[0x1C2+(get_active_partition()-1)*16];

swap_partitions(2) makes the second partition first, so get_part_type() returns the wrong data.
« Last Edit: 25 / June / 2017, 15:36:43 by dmitrys »
Logged
Author of CHIMP, Canon Hack Installation and Management Platform
*

Offline reyalp

  • ******
  • 14000
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #85 on: 25 / June / 2017, 16:40:16 »
Quote from: dmitrys on 25 / June / 2017, 15:24:57
I'm thinking of something along the lines of the "unified installer" that's implemented in ML. Although it does other things, it would be really useful to have a very small executable that would extract the ZIP, optionally delete it, and reboot. It would only require the most basic I/O functions, namely:
...
If there were such an unencoded main.bin, CHIMP could easily encode it for the target platform (if needed).
If the idea is for this to be something that can run without CHDK, I don't see how it could be implemented. The OS needs to be running before you can use those functions, and that requires significant camera specific code. The addresses of the functions are also platform specific. As I explained before, on P&S cameras, all the mechanisms we know to load an executable trash the existing kernel data.

(Technically you can load/run machine code using Canon Basic, but this requires card setup similar to making the card bootable, and no one has found a way to start Canon Basic without a physical key press)
Quote
In the update scenario that would be invoking an unzip.flt.
I still don't get the point. What's wrong with just uploading the files? Why add to a totally new mechanism to solve a problem that isn't a real problem?
Quote
Maybe I'm being snarky about this, but a search for PTP_OC_GetDeviceInfo in CHDK produces 0 matches.
It's a standard PTP operation, implemented by the Canon firmware not CHDK. It does appear in the chdkptp client code.

Quote
swap_partitions(2) makes the second partition first, so get_part_type() returns the wrong data.
Yeah, sounds like a bug, either in the active partition logic or swapping code. I'm not totally sure it was originally intended to reflect updates before reboot, but swap does update mbr_buf in RAM so it seems like it should.
Logged
Don't forget what the H stands for.
*

Offline dmitrys

  • ****
  • 267
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #86 on: 26 / June / 2017, 13:59:00 »
Quote from: reyalp on 25 / June / 2017, 16:40:16
I still don't get the point. What's wrong with just uploading the files? Why add to a totally new mechanism to solve a problem that isn't a real problem?

Well, the UX, for one. The status notification blinks on every file being uploaded (at least with the chdkptp client).

Another possible use for this would be in a desktop user menu editor, since the main menu contents change per camera model. I suppose you could expose the entire menu tree to Lua, but IMHO this is best done natively.

Quote
It's a standard PTP operation, implemented by the Canon firmware not CHDK. It does appear in the chdkptp client code.

Thanks. Too bad mweerden's library only implements OpenConnection() and CloseConnection()...

Quote
Yeah, sounds like a bug, either in the active partition logic or swapping code. I'm not totally sure it was originally intended to reflect updates before reboot, but swap does update mbr_buf in RAM so it seems like it should.

Continued here.
Logged
Author of CHIMP, Canon Hack Installation and Management Platform
*

Offline philmoz

  • *****
  • 3449
    • Photos
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #87 on: 26 / June / 2017, 17:49:09 »
Quote from: dmitrys on 25 / June / 2017, 09:13:03
Quote from: reyalp on 24 / June / 2017, 16:20:04
Note that for historical reasons, the default table format is a human readable format which doesn't unambiguously serialize arbitrary Lua tables. You can override this by defining your own  usb_msg_table_to_string in the code you send.

Right now I have two tables that need deserializing, namely those returned by get_buildinfo() and get_partitionInfo(), and the existing format works just fine for these.

However, I'm having trouble with the values returned by the latter.

Regardless of the active partition, the partition type remains 1. Is that a bug or does "type" stand for something other than the MBR partition type?


For a card with 1 partition 'get_active_partition' always returns 1, and 'get_part_type' returns the type for this partition.


For cards with 2 partitions 'get_active_partition' returns the partition number of the smaller (bootable) partition - NOT the larger partition where photos are saved - 'get_part_type' returns the type of this 'bootable' partition so will always return the same value.


In normal use for a 2 partition card, 'get_active_partition' will still return 1. However, if the user has used the 'swap partitions' function in CHDK the two partitions are physically swapped on the card, and 'get_active_partition' will return 2. This is used by the partition swapping code.


These are working correctly, and the values are exposed via Lua functions - since you can't predict every possible usage of the Lua function, you can't change the behaviour just because it's not what you think it should be.


Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline reyalp

  • ******
  • 14000
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #88 on: 27 / June / 2017, 00:00:47 »
Quote from: dmitrys on 26 / June / 2017, 13:59:00
Well, the UX, for one. The status notification blinks on every file being uploaded (at least with the chdkptp client).
I don't know what status notification you are referring to, but that really doesn't seem like a good reason to make changes to the protocol or CHDK side code.

Quote
Another possible use for this would be in a desktop user menu editor, since the main menu contents change per camera model.
I don't follow, but I'm not clear how the (edit: your PC side) user menu editor works.

Quote
Thanks. Too bad mweerden's library only implements OpenConnection() and CloseConnection()...
Without knowing anything about the code, I'd expect implementing GetDeviceInfo to be pretty easy implement. It operates in the same way as other PTP commands, so it should just be a matter of adjusting the parameters. The chdkptp implementation is https://app.assembla.com/spaces/chdkptp/subversion/source/744/trunk/ptp.c#ln874
« Last Edit: 27 / June / 2017, 00:13:41 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline dmitrys

  • ****
  • 267
Re: CHIMP - Canon Hack Installation and Management Platform
« Reply #89 on: 27 / June / 2017, 16:25:50 »
Quote from: reyalp on 27 / June / 2017, 00:00:47
I don't know what status notification you are referring to, but that really doesn't seem like a good reason to make changes to the protocol or CHDK side code.

I'm referring to the text at the bottom of the camera display consisting of the script title and "<ALT>", the latter against a screaming red background.

Quote
I don't follow, but I'm not clear how the (edit: your PC side) user menu editor works.

It doesn't ATM, since I didn't get to hacking the UMENU.CFG format just yet. Once it does, it will need to somehow obtain a full list of the available menu items. Providing it is the sole purpose of menumeta.flt.

Quote
Without knowing anything about the code, I'd expect implementing GetDeviceInfo to be pretty easy implement. It operates in the same way as other PTP commands, so it should just be a matter of adjusting the parameters. The chdkptp implementation is https://app.assembla.com/spaces/chdkptp/subversion/source/744/trunk/ptp.c#ln874

Yes, the wrapper function is pretty easy indeed. Most of the complexity is inside ptp_unpack_DI(), which, as it turns out, is a real pain to implement in pure managed code.

As if that wasn't enough, I'm using S3 IS for testing, and it's been giving me hell. I'm getting Response Code 0x1001, which AFAIK isn't even a valid RC. Incidentally, chdkptp blatantly refuses to display the camera in the device list. Could the reason lie with the vxWorks PTP port having some serious quirks?
« Last Edit: 27 / June / 2017, 16:29:02 by dmitrys »
Logged
Author of CHIMP, Canon Hack Installation and Management Platform
Pages: 1 ... 4 5 6 7 8 [9] 10 11 12 13 14   Go Up
« previous next »
 

Related Topics