Capturing data traffic between a Sony DSC-QX10 and a smartphone - Non-CANON cameras development - CHDK Forum

Capturing data traffic between a Sony DSC-QX10 and a smartphone

  • 23 Replies
  • 31546 Views
Capturing data traffic between a Sony DSC-QX10 and a smartphone
« on: 28 / September / 2013, 16:40:00 »
Advertisements
Sony published recently the documentation of some part of the API of their "WLAN enabled" cameras:
https://camera.developer.sony.com/pages/documents/view/?id=camera_api

The zip file linked on that page contains two PDF files, one with a very nice introduction how to set up a connection to a camera, the other describes a part of the camera API.

Unfortunately, the API documentation is quite incomplete.

Nevertheless, I was quite curious and bought a Sony DSC-QX10 a few days ago and then spent the evenings cobbling together a little proxy server that allows to log the data exchange between the camera and a smartphone running Sony's playmemories app.

After looking at the first captured logs, I have really mixed feelings. The API seems to be really easy to use and understand -- but the more interesting features seem to be locked down with a kind of challenge-respone mechanism -- even some of the API calls used by playmemories seems to be locked down, like setTouchAFPosition, used to set the auto focus point by tapping on the preview image on the phone.

But this is a _very premature_ conclusion: I haven't yet written even a single line of code for the camera -- up to now, i was just curious, how playmemories controls the camera with (yet) undocumented API calls.

Anyway, if anybody wants to tinker too with a WLAN enabled camera from Sony that works with playmemories: I'll attach its sourcecode to this post.


0. Introduction


The basic protocol is very well described, and the API uses HTTP as the transport protocol and JSON as the data format, so the data exchanged between camera and controlling application can be easily monitored and analyzed.

I wrote a small HTTP and UPnP proxy that can capture the interesting data exchanged between playmemories -- still quite hackish, but working.

The core idea: Use a regular PC or laptop as a proxy that routes data between the camera and the smartphone.

I used the Twisted async framework for Python as the basis for the proxy. Setting this up under Linux should be easy, just install the Twisted package of your favourite distribution -- but I have no idea how the stuff would work under Windows or MacOS...


1. Network configuration

"ASCII drawing" of the setup:

Code: [Select]
    camera
      |
      |   Laptop's WLAN port connected to
      |   the camera's access point
      |   SSID: DIRECT-xxxx:DSC-QX10
      |
    Laptop
      |
      |   Ethernet cable
      |
    access point
      |
      |   WLAN connection
      |   SSID: DIRECT-xxxx:DSC-QX10a
      |
    smartphone running playmemories

The laptop is connected via WLAN with the camera's internal access point, and via an Ethernet cable with a second access point. The smarthone with the playmemories app connects to the network of the access point.

(Thinking again about the setup: It might even work to just let the smartphone login to the camera's AP but then let playmemories select the proxy instead of the real camera -- would be worth a try,
but that would probably require some modifications of the proxy...)


1.1 SSID of the access point

Playmemories connects only to WLAN networks with certain SSIDs. I tried a few variants of the QX10's SSID, "DIRECT-xxxx:D<SC-QX10", where "xxxx" look like four random characters -- but they are not completely random:

If you just change one of the "xxxx" characters of the camera's SSID and use the new string as the SSID for the AP, playmemories does not recognize the AP as a possible camera. But appending an "a" to the original camera's SSID works. Other changes of the part after the ':' may work too.


1.2 Notes about DHCP on the laptop.

The AP must provide DHCP for the smartphone; the camera's internal AP has a DHCP server too which means that the laptop sees two DHCP servers -- and this can lead to some confusion, at least under Ubuntu. I'd recommend to assign static IP addresses to the two interfaces of the laptop.

Static IP addresses are also more convenient for the proxy setup -- you don't need to check any possible changes of the IP address when you start the proxy.


2. Proxy installation.

Install the Twisted package(s) of your Linux distribution (python-twisted for Debian and Ubuntu).

In the file socaprox.py, adjust the IP addresses OWN_C_IP (address the WLAN interface of the laptop in the network of the camera's AP), CAMERA_IP (address of the camera itself (yes, i know, the proxy could discover this address automatically...) and OWN_OTHER_IP (the address of the Ethernet interface of the laptop in the network of the second AP).

Run "chmod 755 socaprox.py", and finally start the program.

stderr will print a a few messages; if the server starts properly, you should see

Code: [Select]
    Got discovery response from camera.
    HTTP proxies started

If these messages do not appear, something went wrong -- time to start debugging ;)

The proxy prints the captured API calls to stdout as JSON strings: one line contains the data for one request/response pair. Reading this stuff is hard -- send stdout output instead into a file.

The file show_log.py shows the logged data in a better readable format.

(and yes, I know that show_log.py could in theory easily read the output from socaprox.py -- I'm just too lazy to do the small required changes right now ;)

[message size limit reached -- next part will follow an a reply]

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #1 on: 28 / September / 2013, 16:40:45 »
3. Some notes about the logs I captured:

The first two requests do not appear in Sony's developer documentation. They are GET requests to 10.0.0.1:10000/sony/index.html and to 10.0.0.1:10000/sony/orbjs/dist/orb-client.min.js

The first URL is an HTML page with a few Javascript fragments that also contains a 'script type="text/javascript"' tag to load the JS library from the second URL.

The next interesting request is the regular API call "getEvent" that returns a lot of data about the camera. The request returns, amongst other things, a quite long list of possible API calls -- many of them are not documented by Sony.

The next interesting call:

Code: [Select]
POST /sony/accessControl
SENT
{   u'id': 1,
    u'method': u'actEnableMethods',
    u'params': [   {   u'developerID': u'',
                       u'developerName': u'',
                       u'methods': u'',
                       u'sg': u''}],
    u'version': u'1.0'}
RECEIVED
{   u'id': 1, u'result': [{   u'dg': u'zDFPMDnW4337ozkf'}]}

"dg" seems to be some sort of challenge value generated by the camera. It varies between requests.

The next call goes to the same URL

POST /sony/accessControl
SENT
{   u'id': 2,
    u'method': u'actEnableMethods',
    u'params': [   {   u'developerID': u'7DED695E-75AC-4ea9-8A85-E5F8CA0AF2F3',
                       u'developerName': u'Sony Corporation',
                       u'methods': u'camera/setFlashMode:camera/getFlashMode:camera/getSupportedFlashMode:camera/getAvailableFlashMode:camera/setExposureCompensation:camera/getExposureCompensation:camera/getSupportedExposureCompensation:camera/getAvailableExposureCompensation:camera/setSteadyMode:camera/getSteadyMode:camera/getSupportedSteadyMode:camera/getAvailableSteadyMode:camera/setViewAngle:camera/getViewAngle:camera/getSupportedViewAngle:camera/getAvailableViewAngle:camera/setMovieQuality:camera/getMovieQuality:camera/getSupportedMovieQuality:camera/getAvailableMovieQuality:camera/setFocusMode:camera/getFocusMode:camera/getSupportedFocusMode:camera/getAvailableFocusMode:camera/setStillSize:camera/getStillSize:camera/getSupportedStillSize:camera/getAvailableStillSize:camera/setBeepMode:camera/getBeepMode:camera/getSupportedBeepMode:camera/getAvailableBeepMode:camera/setCameraFunction:camera/getCameraFunction:camera/getSupportedCameraFunction:camera/getAvailableCameraFunction:camera/setLiveviewSize:camera/getLiveviewSize:camera/getSupportedLiveviewSize:camera/getAvailableLiveviewSize:camera/setTouchAFPosition:camera/getTouchAFPosition:camera/cancelTouchAFPosition:camera/setFNumber:camera/getFNumber:camera/getSupportedFNumber:camera/getAvailableFNumber:camera/setShutterSpeed:camera/getShutterSpeed:camera/getSupportedShutterSpeed:camera/getAvailableShutterSpeed:camera/setIsoSpeedRate:camera/getIsoSpeedRate:camera/getSupportedIsoSpeedRate:camera/getAvailableIsoSpeedRate:camera/setExposureMode:camera/getExposureMode:camera/getSupportedExposureMode:camera/getAvailableExposureMode:camera/setWhiteBalance:camera/getWhiteBalance:camera/getSupportedWhiteBalance:camera/getAvailableWhiteBalance:camera/setProgramShift:camera/getSupportedProgramShift:camera/getStorageInformation:camera/startLiveviewWithSize:camera/startIntervalStillRec:camera/stopIntervalStillRec:camera/actFormatStorage:system/setCurrentTime',
                       u'sg': u'qNnoXv+UaAmbu1KC+W1M7VE1xwFKBIBpB4xd4lYTKkY='}],
    u'version': u'1.0'}
RECEIVED
{   u'id': 2, u'result': [{   u'dg': u''}]}


Note the "sg" field in the request data. It varies too from request to request; my guess is that the app has some secret to generate the data expected by the camera for the given "dg" challenge. (see for example http://en.wikipedia.org/wiki/Challenge_response )

The "methods" field of the request data looks like the camera indeed supports some of the features one would expect:

setFlashMode, setSteadyMode, setViewAngle (whatever this does....), setFocusMode, setFNumber, setShutterSpeed


First thought how to continue:

Extend the existing proxy so that it turns into a real Python library.
Then it is perhaps possible to use playmemories to unlock the "secret
features" but to use these features from another application.

That's it for today -- now a glass of wine and it's bed time for me.

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #2 on: 17 / October / 2013, 11:21:41 »
Working on that challgenge thing, in the meantime here's the list of the private APIs:

Code: [Select]
camera/setFlashMode
camera/getFlashMode
camera/getSupportedFlashMode
camera/getAvailableFlashMode
camera/setExposureCompensation
camera/getExposureCompensation
camera/getSupportedExposureCompensation
camera/getAvailableExposureCompensation
camera/setSteadyMode
camera/getSteadyMode
camera/getSupportedSteadyMode
camera/getAvailableSteadyMode
camera/setViewAngle
camera/getViewAngle
camera/getSupportedViewAngle
camera/getAvailableViewAngle
camera/setMovieQuality
camera/getMovieQuality
camera/getSupportedMovieQuality
camera/getAvailableMovieQuality
camera/setFocusMode
camera/getFocusMode
camera/getSupportedFocusMode
camera/getAvailableFocusMode
camera/setStillSize
camera/getStillSize
camera/getSupportedStillSize
camera/getAvailableStillSize
camera/setBeepMode
camera/getBeepMode
camera/getSupportedBeepMode
camera/getAvailableBeepMode
camera/setCameraFunction
camera/getCameraFunction
camera/getSupportedCameraFunction
camera/getAvailableCameraFunction
camera/setLiveviewSize
camera/getLiveviewSize
camera/getSupportedLiveviewSize
camera/getAvailableLiveviewSize
camera/setTouchAFPosition
camera/getTouchAFPosition
camera/cancelTouchAFPosition
camera/setFNumber
camera/getFNumber
camera/getSupportedFNumber
camera/getAvailableFNumber
camera/setShutterSpeed
camera/getShutterSpeed
camera/getSupportedShutterSpeed
camera/getAvailableShutterSpeed
camera/setIsoSpeedRate
camera/getIsoSpeedRate
camera/getSupportedIsoSpeedRate
camera/getAvailableIsoSpeedRate
camera/setExposureMode
camera/getExposureMode
camera/getSupportedExposureMode
camera/getAvailableExposureMode
camera/setWhiteBalance
camera/getWhiteBalance
camera/getSupportedWhiteBalance
camera/getAvailableWhiteBalance
camera/setProgramShift
camera/getSupportedProgramShift
camera/getStorageInformation
camera/startLiveviewWithSize
camera/startIntervalStillRec
camera/stopIntervalStillRec
camera/actFormatStorage
system/setCurrentTime

Keep in mind that the PlayMemory App was made for the NEX series, so some options here obvioulsy don't make sense for the QX cameras.

More to come...

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #3 on: 17 / October / 2013, 12:36:53 »
Okay so the C/R code is highly obfuscated and apparently in parts also in a native library (tp.so) with a method signature of:
Code: [Select]
public native byte[] getTp(Context paramContext, String paramString);
The actual code flow is like:
Code: [Select]
1. Call getTp() on the dg value passed by the camera
2. Convert the resulting byte array in a string with a (apparently) rather complicated algorithm which works with a few static byte arrays as well (probably some sort of salt or something)
3. Use the resulting string together with the devName, devID and the api method list to authenticate with the camera

I have smali and java code for part 2, so if anyone has too much time or a lot of experience with deobfuscating code, let me know ;)


*

Offline c10ud

  • ***
  • 245
Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #4 on: 18 / October / 2013, 04:29:43 »
Hello, I've never heard of this Sony stuff but looks nice!

I found some resources that may help you in making a python library (or what else you're trying to do):

"WPPMM is Remote shooting application for Windows Phone using Sony Camera Remote API."
https://github.com/opelzz/WPPMM

"sample program for Sony NEX-6 (Smart Remocon App)" (looking at this sample you don't seem to actually need the use of magic values?)
https://gist.github.com/yoggy/5780885





Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #5 on: 18 / October / 2013, 07:13:39 »
Keep in mind that the PlayMemory App was made for the NEX series, so some options here obvioulsy don't make sense for the QX cameras.

Right, I noticed thia already -- you'll get the "error 15, Unsupported Operation" in this case. But there are still some interesting calls like get/setTouchAFPosition that are supported by the QX10 but are "protected" by the silly C/R call.

Okay so the C/R code is highly obfuscated and apparently in parts also in a native library (tp.so) with a method signature of:
Code: [Select]
public native byte[] getTp(Context paramContext, String paramString);

This is great! So, people who want to write an Android app can already try to load tp.so and call getTp(). (Won't help me personally -- I'm more interested to use the camera with an ordinary Linux machine.)

Quote
I have smali and java code for part 2, so if anyone has too much time or a lot of experience with deobfuscating code, let me know ;)

Thanks -- but analyzing this stuff probably requires much more time than I can/want to afford...

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #6 on: 18 / October / 2013, 08:06:14 »
Hello, I've never heard of this Sony stuff but looks nice!

Well, yes and no. My impression is that Sony does not really know what they want. As I wrote, the small part of the API they documented is really well documented. But if you want to do some more serious stuff, you hit one or two limits: The first limit is this silly challenge/response dance needed to unlock many of the API calls. While I found my way to unlock these calls with the little proxy, doing this is just cumbersome -- carrying around an extra AP just to unlock the API is a bit annoying. (OK, it should be possible to set up a USB/WLAN adapter on Linux as an AP for the smartphone and playmemories, but doing all this is still cumbersome...)

Secondly, as Diamondback noticed, the API of the QX10 has serious limits. The NEX cameras may be much better in this respect.

Quote
I found some resources that may help you in making a python library (or what else you're trying to do):

Thanks, but i already worked a bit on a small Python library. A simple class that implements "device discovery" and provides a "pythonic" interface for the API is quite easy to write. (If anybody is interested in details, let me know.)  My main problem is more mental right now: While I like the general approach of the API quite much, I am not sure if I really want to do these unlock tricks in the long term. And even if it is unlocked: ISTM that you even can't set things like the shutter speed or the ISO value for the QX10. The NEX cameras will most likely allow this -- but then I'm also just fine to continue to play with Canon cameras and CHDK  :)

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #7 on: 26 / October / 2013, 03:51:17 »
Okay so the C/R code is highly obfuscated and apparently in parts also in a native library (tp.so) with a method signature of:
Code: [Select]
public native byte[] getTp(Context paramContext, String paramString);
The actual code flow is like:
Code: [Select]
1. Call getTp() on the dg value passed by the camera
2. Convert the resulting byte array in a string with a (apparently) rather complicated algorithm which works with a few static byte arrays as well (probably some sort of salt or something)
3. Use the resulting string together with the devName, devID and the api method list to authenticate with the camera

I have smali and java code for part 2, so if anyone has too much time or a lot of experience with deobfuscating code, let me know ;)

Do you think we can extract the messy authentication part in reusable library for Android at least i.e. the obfuscated java + the shared object.

This would be good win for Android developers. I too wanted to develop laptop based software but....I am thinking of using emulator form my laptop to authenticate the connection will see if it works


Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #8 on: 03 / November / 2013, 08:22:33 »
The actual code flow is like:
Code: [Select]
1. Call getTp() on the dg value passed by the camera
2. Convert the resulting byte array in a string with a (apparently) rather complicated algorithm which works with a few static byte arrays as well (probably some sort of salt or something)
3. Use the resulting string together with the devName, devID and the api method list to authenticate with the camera

I have smali and java code for part 2, so if anyone has too much time or a lot of experience with deobfuscating code, let me know ;)

The smali code for part 2 looks like an apache base64 encoder, so it should not be a problem to reverse it.

Re: Capturing data traffic between a Sony DSC-QX10 and a smartphone
« Reply #9 on: 04 / November / 2013, 03:19:44 »
Okay so the C/R code is highly obfuscated and apparently in parts also in a native library (tp.so) with a method signature of:
Code: [Select]
public native byte[] getTp(Context paramContext, String paramString);

This is great! So, people who want to write an Android app can already try to load tp.so and call getTp().


Yes, I've tried it and it generates the response successfully, but there is a problem - looks like the code from tp.so lib uses the package signature to generate this responce, so using it in different package (or modified playmemories packages) will generate wrong response code.
« Last Edit: 04 / November / 2013, 03:27:14 by observe7 »

 

Related Topics