Wireless investigations - General Discussion and Assistance - CHDK Forum supplierdeeply

Wireless investigations

  • 72 Replies
  • 34703 Views
*

Offline reyalp

  • ******
  • 14079
Wireless investigations
« on: 22 / September / 2013, 19:45:54 »
Advertisements
I've spent some time investigating the elph130 native wifi capabilities. This should be applicable to all recent wifi cams. The original ixus wireless / sd430 from 2005 is likely somewhat different, although I think it uses the same basic technology.

Brain dump follows:

These cameras use UPnP for discovery and PTP/IP for image transfer when connecting to a PC. I haven't investigated the non-PC options.

The spec for PTP/IP is not publicly available (edit: a draft is http://www.cipa.jp/hyoujunka/kikaku/pdf/DC-X005.pdf ), but as the name suggests it's basically PTP over TCP/IP instead of USB. It turns out that PTP commands on the camera are handled by the same code, regardless of the source. This means CHDK PTP extension already supports wireless, it just needs a client that supports IP transport.. I verified this by hacking together a client that sends a CHDK PTP version request, and got the expected results back. Note that this has fairly serious security implications...

Some documentation of PTP/IP is available at:
http://www.gphoto.org/doc/ptpip.php

The current development builds of wireshark support a PTP/IP dissector. These are available from http://www.wireshark.org/download/automated/

Aside from writing proper IP transport code, there are some hurdles to making this really usable. Normally, the camera only turns on wifi if you go through the menu and select a connection. It drops out of this mode and shuts down wifi as soon as you close the PTP socket.

Getting to a point where you can actually make a connection also takes some work.

To make a connection, your PC has to be "added" on the camera. You don't have to install the canon software, just go to the Canon "add" screen on the camera, and (on windows 7) go to "devices and printers", select add device etc. This does a whole bunch of fun UPnP stuff with XML and HTTP over UDP etc  :-[ You need UPnP and device discovery enabled and not blocked by your firewall.

Once the device is added, it shows up in the normal windows browser similar to the way it shows up under USB. Like USB, connecting this way will make the screen go black. There is no way to "eject" the camera, you have to shut it down to end the connection. I have found no way to open another PTP connection to the camera when this is going on.

However, after setting up, you can remove the camera in "devices and printers". It still exists on the camera, so you can start the connection process on the camera, and then when it's waiting for the computer, you can run your own PTP/IP software and connect. Connecting from a different host doesn't seem to work. The connection process contains a GUID and name, but these don't appear to need to be the same as what windows used when the connection was set up.

If you had control of your own UPnP "control point", this could probably be simplified.

Some other random notes:
The camera will show you it's MAC in the "wifi settings" menu item. This is useful for sniffing with wireshark, or setting up firewall rules to ensure it doesn't "phone home" (as far as I can tell, it doesn't). I wasn't able to find the MAC in ROM, not sure if I missed it or it's stored somewhere on the wireless module.

If you have USB connected and try to make a wifi connection (only possible if you've unlocked the keyboard with post_levent_to_ui(4484)) the camera will shut down with an assert. If you plug in the USB cable after a wifi connection is established, it is ignored.

When you are adding a connection in the camera network screen or connecting to an existing connection, there is a web server listening on port 49152, which will serve /upnp/CameraDevDesc.xml and  /upnp/CameraSvcDesc.xml as well as various HTTP error codes if you request something else. /upnp/CameraDevDesc.xml  is announced in SSDP multicasts when you start the connection process.

Cross post of a comment I posted from dpreivew http://www.dpreview.com/forums/post/52177988

Quote
I found a function in the elph130 firmware (at FF022068) which based on error messages allowed me to tentatively identify functions like socket(), setsockopt(), bind(), select() etc.

This appears to be related to some kind of networking test code, task wprd, appears to be associated with dryshell command 'WiFi Test Tool for Product". I haven't played with these yet, because PTP/IP seems like the quickest path to useful functionality. However, this sort of lower level approach could be very useful.

One thing that would be really useful is to be able to tell the camera to start wifi and listen for ptp connections without going through the camera UI.
« Last Edit: 29 / September / 2013, 04:03:17 by reyalp »
Don't forget what the H stands for.

*

Offline ahull

  • *****
  • 634
Re: Wireless investigations
« Reply #1 on: 23 / September / 2013, 07:55:38 »
I've spent some time investigating the elph130 native wifi capabilities. This should be applicable to all recent wifi cams. The original ixus wireless / sd430 from 2005 is likely somewhat different, although I think it uses the same basic technology.

I have a working Ixus Wireless (SD430) somewhere, so if there is anything you would like to test with this, I can dig it out and dust it down.

*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #2 on: 23 / September / 2013, 13:13:20 »
I have a working Ixus Wireless (SD430) somewhere, so if there is anything you would like to test with this, I can dig it out and dust it down.
Since there is currently no CHDK port for this camera, I don't think we need to test. If you want to make a port, that would make our support of the old vxworks models almost complete ;)

I had a quick look at the firmware dump I have for that one, and it does indeed use UPnP and PTP/IP, and based on strings in the firmware seems relatively similar to modern wireless cams. The dump I have isn't a full dump, if you want to run the canon basic dumper on it, that could be helpful if it gets ported in the future. Beware though that the elph130 stores some information about networks you have connected to in flash. It's not clear if this includes wireless keys, but I would assume it does since you don't have to re-enter them after registering an AP. Comparing dumps from before I used wireless and after, I found changes at:
FFF00000 - FFF0414C looks something network related, has AP names and IP addresss
FFF60AF0 - FFF72050 params, includes camera nick name in param 138
FFFA0000 - FFFA0BD0 romlog
FFFC0000 - FFFC01F0 something else, starts with a date/time stamp
Note the end addresses above are probably aren't the real end of a given area, it's just around where I started seeing unused (filled with FFFF) values.

FWIW, I checked my test code into the chdkptp svn (ptpip.c). Note that this is just test code in case someone wants to play with it, it's not a working client and doesn't do anything useful. I have also started re-working the chdkptp code to support ptp/ip but this will take some work.
Don't forget what the H stands for.

*

Offline ahull

  • *****
  • 634
Re: Wireless investigations
« Reply #3 on: 23 / September / 2013, 15:03:13 »
I took a look at dumping the roms from the Ixus Wireless a while back (at least a year), but got caught up with other things, I just found a backup of  this lot for the Ixus Wireless f/w 1.00f and 1.10a

1.00f/
1.00f/PRIMARY.BIN.labels
1.00f/PRIMARY.BIN.strings
1.00f/PRIMARY.BIN.dis
1.00f/PRIMARY.BIN
1.00f/PRIMARY.BIN.elf
1.10a/
1.10a/PRIMARY.BIN.labels
1.10a/PRIMARY.BIN.strings
1.10a/PRIMARY.BIN.dis
1.10a/PRIMARY.BIN
1.10a/PRIMARY.BIN.elf

I presume they are complete, I can tar them up and send to you if you want.
There are two ROM dumps because I dumped the original and then upgraded to the latest version on the web.
There will be no personal data in them because the Camera was defaulted to the factory settings prior to dumping.

EDIT: I was sure I already passed the rom dumps  on, and it seems I  did.. at least for the 1.00f dump.  Dumping thread... http://chdk.setepontos.com/index.php?topic=288.msg83401#msg83401
« Last Edit: 23 / September / 2013, 15:10:34 by ahull »


*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #4 on: 23 / September / 2013, 15:59:39 »
EDIT: I was sure I already passed the rom dumps  on, and it seems I  did.. at least for the 1.00f dump.  Dumping thread... http://chdk.setepontos.com/index.php?topic=288.msg83401#msg83401
Oops, sorry for the run around, I just assumed the one in my personal collection was the only one we had.

The 110a in the archive looks like it's a ~4mb dump from 2007.
Don't forget what the H stands for.

*

Offline ahull

  • *****
  • 634
Re: Wireless investigations
« Reply #5 on: 24 / September / 2013, 03:31:46 »
If you want my version of either of these dumps, let me know and I will email them or post them on googledocs, dropbox or wherever.

EDIT: Since the Ixus Wireless port is actually off topic, I have opened a separate porting thread for it here.

EDIT: The porting of the Ixus Wireless SD430 is now complete.
 
« Last Edit: 25 / November / 2013, 12:24:33 by ahull »

*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #6 on: 29 / September / 2013, 03:17:58 »
Some more notes:

You can get a draft of the PTP/IP spec from CIPA http://www.cipa.jp/hyoujunka/kikaku/pdf/DC-X005.pdf

Not clear how close to final it is. One lovely thing is that none of the actual values of constants are given.

The wireshark dissector (as of wireshark svn r52161 at least) does not handle cases where multiple PTP/IP packets are present in a single IP packet. So if the camera sends the Start_Data_Packet and Data_Packet in one IP packet, you will only see the Start_Data_Packet, with the PTP/IP length field shorter than than the IP packet data.

Switching to "rec" mode with either set_record or switch_mode_usb() seems to kill the network connection.

The keyboard isn't locked, but when you are in the network screen on the camera, the only keys that will do anything end the connection.

edit:
The camera also seems to drop connection after some fairly short period of time.

In chdkptp r448 I added enough code to make most of the ptp functions accessible. File download doesn't work (because my hack to download directly to disk uses usb specific stuff) and some of the lua code depends on usb stuff that doesn't exist on a ptp/ip connection. The network code also probably definitely has issues.


To make a connection, connect to your PC on the camera, and then on the PC use
!con=chdku.connection{host='host'}
!con:connect{raw=true}
where host is the hostname or IP address of the camera.


edit:
I fixed the lua code so you can now use
c -h=<host>

edit:
as of chdkptp r453, all the basic ptp protocol command should work http://chdk.setepontos.com/index.php?topic=6231.msg106005#msg106005
« Last Edit: 06 / October / 2013, 19:15:40 by reyalp »
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #7 on: 07 / October / 2013, 02:17:33 »
A few more bits

RegComEventProc registers some network related eventprocs.

PairingDataOutput will output data about connections used to a file called PAIRIN~1.XLS (would be PairingData.xls) which is actually a tab separated value file. It shows information about the computers and access points that you've registered. The AP data includes your wireless password (WPA-PSK in my case) in clear text.

There are functions to connect to a network, but they require terminal input.

I'm assume there's a function somewhere that will let you connect to one of the saved connections by slot number in the pairing data, but I haven't found it yet.

I investigated the issue where the network connection will close after a short time if you don't have device enabled in windows.

The timeout seems to be about 1 minute, regardless of whether you are doing anything on the connection. From sniffing the network traffic in both cases, I don't think this is controlled by any of the non-ptp (UPnP/SSDP) stuff. My guess is that the timeout is because the camera is still in the "connecting" screen, and that what ever puts it in the black screen PTP mode also takes care of this. If it's like USB PTP, the black screen mode is triggered by some normal PTP commands.

Examining the PTP traffic shows that the camera supports MTP (http://en.wikipedia.org/wiki/Media_Transfer_Protocol) commands. I haven't investigated which commands it supports, but windows sends some on connect.
Don't forget what the H stands for.


*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #8 on: 13 / October / 2013, 17:23:24 »
Another brain dump, to help me organize what I know so far...

Most of the actual connection management is done by a task called ComWireless

Like many other tasks, it sits in a loop waiting for messages with ReceiveMessageQueue and performs various actions in response. The message format is 3 words: a message number, and two parameters which may be pointers or numbers. The meaning varies depending on the message. Pointers may be input or output parameters, with the second one frequently being the output.

ComWireless operates as a state machine, where the message handling function explicitly check a state field and generate error messages (e.g. "StartPairing State Error") and set an error state if the requested action isn't valid in the current state.

Structures:
The main ComWireless data structure seems to be at 0x65B0 (found at the top of the task_ComWireless)
0x65B0 + 8 is a pointer to another large structure
Offset 0 in this structure is the state machine state. Other members include the message queue (+0xBC) and an event flag (+0xC0). The structure extend to at least 0xF0; Based on the getCWSStatus eventproc states 0 and 1 are considered "idle". The underlying function used by this eventproc can probably be used in CHDK to see if the network is on.

Messages are sent to ComWireless using a function I've called post_to_comwireless (sub_FF17C4D0). It takes the 3 message fields as parameters. This helps find out what the messages do, since you can trace back to things like the eventprocs mentioned earlier (but I think the normal canon UI goes through a "controller" which uses function pointers that aren't easy to trace back to a particular UI action)

ComWireless frequently posts to itself, so a message from an external task may trigger multiple messages to be processed by ComWireless.

The attached patch has code to log the messages and state transitions. The attached log shows the following sequence:
1) connect with USB, start uart redirection, show camera log
2) disconnect usb, unplug USB cable
3) press wireless button (up) wait 5 seconds
4) press set to select connecting to my existing PC connection
5) camera connects to network, gets an IP address, goes into PTP/IP ready state
6) connect via PTP/IP, show camera log
7) Disconnect PTP
8 ) Press "menu" to exit the "connection closed" screen on the camera. This brings up the AP list
9) Press "menu" again to return to the device selection list, and again to exit the network screen
10) Connect USB, show camera log

Ok, that's all fun, but what can we do with it? I made post_to_comwireless available to lua, and started trying to re-create the connect sequence without going through the UI.

1) Some sequences of messages cause the camera to make a disturbing (mechanical I think, but possibly speaker?) clicking.  :blink:
2) The camera being connected to USB PTP affects the behavior. No surprise since I already knew having USB connected and trying to start a network connection through the UI triggers an assert, but...
3) With USB connected:
If I post (8,0,0), (0,0,0) comwireless state goes 5, wifi is turned on, the camera connects to my AP, and I can ping the camera. Wireshark shows that the second call triggers a DHCP request from the camera. If I attempt to make a PTP/IP connection (after unplugging USB), the TCP connection is refused. However, it seems likely that at this point you could use socket functions on the camera. The camera remains pingable if I switch to rec mode. If I re-plug the USB cable, a USB/PTP connection cannot be established (I believe the PTP code checks the status function mentioned earlier)

Adding a post (0x3,0,0) with a sleep that allows me to disconnect the USB cable results in a PTP/IP connection rejection on the command channel.

Adding
.sleep(5000) post_to_comwireless(0x3,0,0) sleep(1000) print('state',get_comwireless_state()) sleep(1000) post_to_comwireless(0xa,0,0) print('state',get_comwireless_state()) sleep(100) call_event_proc('ShowCameraLog')

Allows me to connect with PTP/IP, but the camera asserts on the OpenSession command, with the same  SelectFuncCo0 assert mentioned earlier. I see [PTPState] PTP_Unexpected Type=1 STATE=3 EVENT=0 in the camera log (from the ShowCameraLog above, probably when the USB cable is disconnected)
edit: I believe the final message generates the UPnP announce stuff.

4) If USB PTP is not connected (running a standalone script on the camera)
The (8,0,0) (0,0,0) sequence above goes to state 4 and then to 0xa and 0x9 (I think 0xa is an error state).  The camera doesn't generate any network traffic.
« Last Edit: 13 / October / 2013, 17:39:03 by reyalp »
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14079
Re: Wireless investigations
« Reply #9 on: 13 / October / 2013, 22:18:52 »
Network related errors seem to go to a different log system than the normal camera log or camera console. This can be sent to the camera stdout with the EpilogShow eventproc.

sub_FF0B9818 writes to this log.
Don't forget what the H stands for.

 

Related Topics