rsint with network socket for control using ipc

quesarilla

rsint with network socket for control using ipc

« on: 14 / August / 2021, 17:37:22 »

Advertisements

I'm trying to expand the functionality of remoteshoot so that it is able to take interactive commands and report results to a client using network sockets.

I am using the luaSockets library(module/package??) to generate the tcp connection, however when running the line socket=require("socket") the program exits with exit code -1073741819.

I have tested the command on the interpreter and it works fine, I'm even able to generate a tcp connection.
The chdkptp installation is also able to run lua code on the cli locally.
The offending line is at the top of main.lua
My LUA paths are set up correctly
I'm running LUA version 5.3.6 on 64bit mode
Chdkptp version is r921
Platform is windows 10 x64
The program does not show any error message, it simply exits.

Does anyone have an idea of what could be causing the import statement to crash the program? or where I can find an exit code table for lua?

Logged

reyalp

13353

Re: rsint with network socket for control using ipc

« Reply #1 on: 14 / August / 2021, 18:33:46 »

Quote from: quesarilla on 14 / August / 2021, 17:37:22

I'm trying to expand the functionality of remoteshoot so that it is able to take interactive commands and report results to a client using network sockets.

If you're using the CLI, you might be better off just running chdkptp in a remote terminal application, or writing your own glue wrapper that pipes the input and input.

If you specifically only need to use rsint, you could write an external program and use the -pipe option, or perhaps even just use netcat.

Quote

I am using the luaSockets library(module/package??) to generate the tcp connection, however when running the line socket=require("socket") the program exits with exit code -1073741819.

I don't know what the error means specifically, but my first guess would be some incompatibility between the chdkptp binary and the luasocket dll.
Are you building chdkptp yourself? The builds I release on assembla are have Lua statically linked to the core and aren't really intended to work with modules in dlls.

At a minimum, your luaSocket would likely need to be built with the same runtime libraries and so n that the chdkptp build is, probably with mys2 of a similar generation to what I built with.

One reasonably straightforward approach might be to build luasocket statically into your own build of chdkptp, using how lfs is done as an example.

Quote

I have tested the command on the interpreter and it works fine, I'm even able to generate a tcp connection.

Is "the interpreter" here is a stock Lua interpreter (built how?), or chdkptp CLI?

Quote

I'm running LUA version 5.3.6 on 64bit mode
Chdkptp version is r921

All of the official builds of chdkptp though the current release (which is r964, not r921) are built with Lua 5.2. If you need to use Lua 5.3, I recommend building chdkptp from the latest svn source.

Logged

Don't forget what the H stands for.

quesarilla

Re: rsint with network socket for control using ipc

« Reply #2 on: 14 / August / 2021, 19:43:38 »

Quote from: reyalp on 14 / August / 2021, 18:33:46

If you're using the CLI, you might be better off just running chdkptp in a remote terminal application, or writing your own glue wrapper that pipes the input and input.

There are some software considerations that don't allow me to use a terminal.

Quote

If you specifically only need to use rsint, you could write an external program and use the -pipe option, or perhaps even just use netcat.

My goal is to minimize the time it takes to get the images from an array of 6 cameras to the computer. Multicam comes at ~11s, threaded rs at ~4.5s, while rsint at ~1.5s. rsint seems to be my best bet, but I'm open to suggestions.

Quote

I don't know what the error means specifically, but my first guess would be some incompatibility between the chdkptp binary and the luasocket dll.
Are you building chdkptp yourself? The builds I release on assembla are have Lua statically linked to the core and aren't really intended to work with modules in dlls.

I'm using pre compiled binaries for both chdkptp and lua.

Quote

Is "the interpreter" here is a stock Lua interpreter (built how?), or chdkptp CLI?

Yeah, it's a stock lua interpreter. I got the binaries from sourceforge, not sure how it was built.

Thanks for the prompt reply, recompiling sources would be my preferred approach if my target system wasn't a SoC without internet access. I think I will move the network logic to a different program and use the pipe switch.

Logged

reyalp

13353

Re: rsint with network socket for control using ipc

« Reply #3 on: 14 / August / 2021, 21:43:17 »

Quote from: quesarilla on 14 / August / 2021, 19:43:38

My goal is to minimize the time it takes to get the images from an array of 6 cameras to the computer. Multicam comes at ~11s, threaded rs at ~4.5s, while rsint at ~1.5s. rsint seems to be my best bet, but I'm open to suggestions.

If you can describe your architecture in a bit more detail I may be able to offer some suggestions.

I'm not sure what you mean by "threaded" rs, do you mean multiple instances of chdkptp each running a single "rs" command?

If your plan is to run multiple instances of chdkptp with rsint, and have them triggered all at the same time over a network, this seems like it should be possible using rsint -pipe. You should just need a program that emits the required characters when it sees the network packet.

I was able to get this to work using netcat over tcp, like

Code: [Select]

rsint -pipe="nc -l 0.0.0.0 4567"

and then entering the rs commands from a remote netcat.

Quote

I'm using pre compiled binaries for both chdkptp and lua.

I definitely would not expect Lua dll modules built for some arbitrary binary Lua distribution to work in the chdkptp binaries that I distribute.

Logged

Don't forget what the H stands for.

quesarilla

Re: rsint with network socket for control using ipc

« Reply #4 on: 16 / August / 2021, 14:43:47 »

Quote from: reyalp on 14 / August / 2021, 21:43:17

If you can describe your architecture in a bit more detail I may be able to offer some suggestions.

6 s110 powershot cameras connected to a raspberry pi thru a 2.0 usb hub, running rasbian 4.19.97.
Commands to the pi are sent over ssh by a central computer.

Quote

I'm not sure what you mean by "threaded" rs, do you mean multiple instances of chdkptp each running a single "rs" command?

Yes, multiple instances of chdkptp running a single rs command.

Quote

I was able to get this to work using netcat over tcp, like
Code: [Select]
rsint -pipe="nc -l 0.0.0.0 4567"and then entering the rs commands from a remote netcat.

This is more or less the functionality I want, except the communication is one directional so synchronization is a bit trickier. Based on your suggestion I'm thinking that something like this may work better:

Code: [Select]

socat "chdkptp -c -e=rsint" 0.0.0.0:5555

I haven't tested it yet, but will post an update when I get around to.

Logged

reyalp

13353

Re: rsint with network socket for control using ipc

« Reply #5 on: 16 / August / 2021, 16:03:20 »

Quote from: quesarilla on 16 / August / 2021, 14:43:47

Quote from: reyalp on 14 / August / 2021, 21:43:17
If you can describe your architecture in a bit more detail I may be able to offer some suggestions.
6 s110 powershot cameras connected to a raspberry pi thru a 2.0 usb hub, running rasbian 4.19.97.
Commands to the pi are sent over ssh by a central computer.

From your previous posts, I thought chdkptp was running on windows 10. Getting luasocket to work on a pi should be significantly simpler, although not completely trivial. You'd likely still need to build chdkptp on a pi. The basic approach I would try is to install the lua5.2 dev package, link chdkptp with that (editing config.mk to point to the system installed lua, rather than using setup-ext-libs.bash, similar to the instructions for using the LGI GUI), and then install luasocket as a regular lua 5.2 module, using the package manager or luarocks. Compiling chdkptp on a pi is straightforward if you don't need a GUI, and you can easily copy the resulting binary to any similarly configured pi.

If all the cams are connected to a single pi, you'll be limited by USB2 bandwidth = ~50 Mbyte/s at best (assuming the pi and its storage can keep up), so the rsint shooting rate for the full rig will likely be significantly lower than a single cam.

Quote

This is more or less the functionality I want, except the communication is one directional so synchronization is a bit trickier. Based on your suggestion I'm thinking that something like this may work better:

Yeah, if you want to control the whole process remotely, rather than just triggering shots within a single rs section, you're probably better off controlling chdkptp's standard input. A dedicated python wrapper might end up being better than trying to do with with a generic "dumb" network tunnel. A dedicated wrapper could listen to one socket and control all the chdkptp instances.

Logged

Don't forget what the H stands for.

quesarilla

Re: rsint with network socket for control using ipc

« Reply #6 on: 16 / August / 2021, 17:24:12 »

Quote from: reyalp on 16 / August / 2021, 16:03:20

From your previous posts, I thought chdkptp was running on windows 10. Getting luasocket to work on a pi should be significantly simpler, although not completely trivial. You'd likely still need to build chdkptp on a pi. The basic approach I would try is to install the lua5.2 dev package, link chdkptp with that (editing config.mk to point to the system installed lua, rather than using setup-ext-libs.bash, similar to the instructions for using the LGI GUI), and then install luasocket as a regular lua 5.2 module, using the package manager or luarocks. Compiling chdkptp on a pi is straightforward if you don't need a GUI, and you can easily copy the resulting binary to any similarly configured pi.

All my tests are done on a w10 laptop, besides the pi doesn't have internet access so I just cross-compile everything.

Quote

If all the cams are connected to a single pi, you'll be limited by USB2 bandwidth = ~50 Mbyte/s at best (assuming the pi and its storage can keep up), so the rsint shooting rate for the full rig will likely be significantly lower than a single cam.

That is a good point, I am not aware of what sort of storage the pi currently uses but I'm guessing it's an UHS I SD card. So ~30mBps might be the best I get without having to upgrade hardware.

Quote

Yeah, if you want to control the whole process remotely, rather than just triggering shots within a single rs section, you're probably better off controlling chdkptp's standard input. A dedicated python wrapper might end up being better than trying to do with with a generic "dumb" network tunnel. A dedicated wrapper could listen to one socket and control all the chdkptp instances.

That should be easy enough.

As an aside, in case anybody was wondering how to set up something as described by the title, I tested the socat command using:

Code: [Select]

socat -L lockfile EXEC:"./chdkptp -i" TCP4-LISTEN:5555

It worked just fine.

Logged

quesarilla

Re: rsint with network socket for control using ipc

« Reply #7 on: 08 / September / 2021, 00:55:32 »

Update:

I created the python wrapper; it enters rsint mode on 6 different cameras, then emits the character to capture images.

The script was tested by setting all cameras to rsint mode, emitting the 's' character to capture the images, then timing the period it took to complete the operation on all cameras.

After a hard reboot of the cameras, it only took 1.4s on average to complete concurrent shots (20 trials, totaling 120 images in 28s).

However, sometime later the same action started to take 2.6s on average (120 trials, totaling 720images in ~310s). The increment wasn't gradual, and was relatively consistent throughout.

Can this discrepancy be related to CHDK code on the camera side?

Additional info:
CHDK version 1.4.1
CHDKPTP version: r921
Platform: powershot s110
Subplatform: 102b

OS: raspbian stretch
COMPUTER: raspberry pi 4

Logged

reyalp

13353

Re: rsint with network socket for control using ipc

« Reply #8 on: 08 / September / 2021, 18:45:56 »

Quote from: quesarilla on 08 / September / 2021, 00:55:32

The script was tested by setting all cameras to rsint mode, emitting the 's' character to capture the images, then timing the period it took to complete the operation on all cameras.

How are you determining when cameras complete the operation?

Quote

However, sometime later the same action started to take 2.6s on average (120 trials, totaling 720images in ~310s). The increment wasn't gradual, and was relatively consistent throughout.

Can this discrepancy be related to CHDK code on the camera side?

IMO, this is quite unlikely. Generally speaking, CHDK scripted shooting can shoot at a steady rate for thousands of shots, and I wouldn't expect it to degrade by seconds due to CHDK issues. I've been surprised before though

A few things that come to mind:
* Did lighting or exposure change? If you're in low light (including regular indoor lighting) exposure time can be relatively significant. High ISO can also add significant time for noise reduction processes.
* Did other settings on the cameras change? Raw, jpeg resolution etc, continuous mode.
* Did the scene change? More complex scenes result in larger images. This can be dramatic if you're comparing say, an out of focus scene vs a highly detailed one.
* Could it be related to IO performance on the pi? I'm not sure how you'd measure this, but most SD cards are designed for very different workloads and there's definitely been reports of them behaving poorly under load on pi.

My approach to debugging this would be
1) Check the exposure/setting stuff mentioned above
2) Try to reproduce it with a single camera. If you can, everything will be simpler.
3) Try to time the various stages of the shooting process and find out where the delay appears. For example, the time between when the 's' is sent and data becomes available, and the time from the first and last data for a given shot being written. The simplest way to measure these would probably be to add logging to the chdkptp Lua code.
For the 's' command, you could log from rsint.lua in m.rsint_once just before

Code: [Select]

con:write_msg(cmdname..' '..rest)

For receiving data, you could log from chdku.lua in chdku.rc_handler_file (assuming you're using jpeg) at the start and end of each loop iteration, and after the final fh:close()

You can get subsecond timestamps using something like
printf('%s\n',ustime.new():format('%c.%_u'))
Or monotonic microsecond counter with ticktime.get()

If you set cli_verbose=2 information will be output for each chunk received, but it doesn't include timestamps. You could override cli.dbgmsg to include timestamps.

Quote

Additional info:
CHDK version 1.4.1
CHDKPTP version: r921

FWIW, the current stable CHDK is 1.5.1 and most recent chdkptp release is r964. I'm not aware of any changes that would be directly relevant to your problem, although there were some fixes in the PTP and remoteshoot.

Logged

Don't forget what the H stands for.

quesarilla

Re: rsint with network socket for control using ipc

« Reply #9 on: 09 / September / 2021, 10:54:56 »

Quote from: reyalp on 08 / September / 2021, 18:45:56

How are you determining when cameras complete the operation?

Program blocks until chdkptp writes to stdout, then check if it is the rsint prompt.

Quote

* Did lighting or exposure change? If you're in low light (including regular indoor lighting) exposure time can be relatively significant. High ISO can also add significant time for noise reduction processes.

Does the camera autocompensate for exposure if I set them manually? mm that could be a problem.
Here is the command I use to enter rsint mode:

Code: [Select]

rsint /home/pi/images/${serial}/${name} -tv=0.200 -sv=80 -av=11.0 -sd=5m

Quote

* Did other settings on the cameras change? Raw, jpeg resolution etc, continuous mode.

No, all I do is enter a loop that queries the time, emits the 's' character, validates the response, then gets the time again.

Quote

* Did the scene change? More complex scenes result in larger images. This can be dramatic if you're comparing say, an out of focus scene vs a highly detailed one.

The cameras were aimed at my ceiling, which is plain white.

Quote

* Could it be related to IO performance on the pi? I'm not sure how you'd measure this, but most SD cards are designed for very different workloads and there's definitely been reports of them behaving poorly under load on pi.

I also ran the test on a virtual machine (debian bullseye, amd64, kernel v 5.10) using a single camera and saw the same degradation.
Initial shots took between 1.1s and 1.6s, then around the 20th shot it started to take ~2.6s per shot. Surprisingly it wasn't much faster than running the same on 6 cameras, so I'm guessing that bus speed is not a limiting factor here?

The camera makes a sound when capturing a shot as similar to the one produced when switching between rec and play mode, and it's roughly proportional to the time it takes for the image capture to complete.

Quote

My approach to debugging this would be
1) Check the exposure/setting stuff mentioned above
2) Try to reproduce it with a single camera. If you can, everything will be simpler.
3) Try to time the various stages of the shooting process and find out where the delay appears. For example, the time between when the 's' is sent and data becomes available, and the time from the first and last data for a given shot being written. The simplest way to measure these would probably be to add logging to the chdkptp Lua code.
For the 's' command, you could log from rsint.lua in m.rsint_once just before

Ok, I'll test this on my main computer. Dumping 's\n' on the stdin should do it.

Quote

FWIW, the current stable CHDK is 1.5.1 and most recent chdkptp release is r964. I'm not aware of any changes that would be directly relevant to your problem, although there were some fixes in the PTP and remoteshoot.

I tried compiling it from sources on the vm I mentioned above but couldn't set up the toolchain properly, kept getting arm-gcc-eabi not found errors.

Logged