StereoSync Project - Synchronizing multiple cameras for stereophotgraphy - General Discussion and Assistance

Recyclojunk64

127

StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« on: 28 / December / 2020, 04:04:17 »

Advertisements

Hi all,

I've been away from the CHDK forum for a long time caught up with many other projects (as well as uni then onto full time job). My triple S110 rig has been working well (thanks to all the help from this forum) and it's had a lot of use since 2014/2015 when I stopped working on it (I have some photos uploaded to https://stereopix.net/josh/ and http://phereo.com/joshn). But now my main project is building an upgraded version of it using 3 Canon G7X cameras.

For anyone who hasn't seen the project yet it is a way to synchronize two or more Canon cameras for stereophotography, synchronizing the shooting settings (aperture, ISO, exposure time, ND filter state etc.) from the master camera to the slave cameras, then synchronizing the shot to less than 1ms difference. This allows the user to control the whole stereo rig using only the buttons on the master camera. It can do continuous shooting with exposure bracketing, and also works with some scripts such as interval shooting.

This time I'd like to make the rig more reproducible by others, unlike the last rig which required soldering in wires onto the camera PCB. I have a few ideas of how to make this work without any hardware modifications to the camera, such as powering them all on at the same time by permanently holding down the power or wifi buttons (and masking when it is on) and applying power to the dummy batteries instead of soldering wires to the on/off switches to put them in parallel. Input can be done through the temp-sense terminal on the dummy battery, output can possibly be through the AF LED read with a photodiode but if there is an electrical output I can tap into that will be better (will make another post for this). Last time I did not actually use any dummy batteries or photodiodes like I planned to, since I was already soldering wires for the power button I just soldered wires to everything I wanted to access including power then brought that out to a separate connector on the outside of the camera.

Last time I also wrote the changes straight into the 2014 CHDK code without making them as patches and never published them anywhere (though I did send to some forum members who requested it). I'm thinking the best thing to do this time would be to write as patches and publish it on a repository somewhere so that anyone else who wants to use it can easily download it and apply it to the latest CHDK code?

Logged

reyalp

14117

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #1 on: 28 / December / 2020, 16:18:24 »

Quote from: Recyclojunk64 on 28 / December / 2020, 04:04:17

I've been away from the CHDK forum for a long time caught up with many other projects (as well as uni then onto full time job). My triple S110 rig has been working well (thanks to all the help from this forum) and it's had a lot of use since 2014/2015 when I stopped working on it (I have some photos uploaded to https://stereopix.net/josh/ and http://phereo.com/joshn).

Nice.

Quote

Input can be done through the temp-sense terminal on the dummy battery, output can possibly be through the AF LED read with a photodiode but if there is an electrical output I can tap into that will be better (will make another post for this).

FWIW, as I mentioned in the other thread, on G7X you can control HDMI power for output, and for input you have your choice of HDMI HPD and analog AV detect in addition to battery temp and USB power. HDMI power + HDMI HPD is attractive because the HDMI connector doesn't interfere with using USB for shot triggering, doesn't require an external power source, and unlike the proprietary connector used for analog AV + USB, breakout connectors are readily available.

Quote

Last time I also wrote the changes straight into the 2014 CHDK code without making them as patches and never published them anywhere (though I did send to some forum members who requested it). I'm thinking the best thing to do this time would be to write as patches and publish it on a repository somewhere so that anyone else who wants to use it can easily download it and apply it to the latest CHDK code?

You could certainly do that. You're welcome to use a thread here and make a page on the wiki, and you code can be hosted on github, assembla or whatever you prefer. If the code changes can be made relatively self contained, they could potentially be included in official source under a camera.h define.

Logged

Don't forget what the H stands for.

Recyclojunk64

127

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #2 on: 14 / March / 2022, 10:06:18 »

Update: I have now built a rig using 3 Canon G7X Mark II cameras, and the StereoSync code additions are at a usable state for single-shot mode, but still lots features to be added and some bugs to fix with how I'm controlling the slaves. Currently the additions are all in a separate stereosync.c and stereosync.h file, with the exception of some asm code in boot.c and some menu entries in gui.c. Still lots of work to do before it's in a releasable state with bug fixes, but the groundwork for the project has been mostly laid.

This time I'm using the 4th battery terminal for communication, which is essentially just a half-duplex UART and can be connected straight to the slave 4th battery terminals (not sure how many but 3 works) without any external electronics required, except for some optional protection zener diodes. The transmit and receive asm functions are modified to transmit and receive variable-length packets which I will go into more detail on the thread about the 4th battery terminal: https://chdk.setepontos.com/index.php?topic=14196

Currently I still need to get the precision sync working. The normal sync precision is usually about 0.1ms to 0.2ms deviation, but sometimes it extends to about 3ms of deviation, without precision sync enabled. How can I go about finding the address for GPIO_VSYNC_CURRENT for a digic 7 processor? I did try the usual value of 0xC0F070C8 that worked for digic 5 and previous cameras but it crashed the G7X2 camera. I have the firmware loaded into Ghidra but not sure what to look for.

Another thing, to initiate shooting on the slave cameras, I am currently simulating button presses on the slave cameras. However this does not work well in continuous shooting mode as the timing of the fullpress release is critical to stop the one of the slaves from going to an additional shot while the other cameras don't. So I would like to find a better way to do this, currently thinking of using levent_post_to_ui_by_name unless anyone has a better idea of how to do this? I'm currently using state machines to keep track of the shooting and syncing process on the master and slaves so there is the potential to make this quite robust.

Here is a photo of the stereo rig. It was designed with the intent of being CNC'd from aluminium and anodized, but unfortunately my opportunity to get them machined for free was lost when my machinist friend from work decided to quit just after I finished the design but before he had a chance to make the parts

Therefore I had to just to 3D print them for now (and add some aluminium bars to give it some rigidity) until I make friends with our new machinist

The power button is an NC push button that was intended to power on all cameras at once, by permanently holding down the WiFi button on the side and disconnecting then reconnecting power which turns them on. But I found afterwards that even when the cameras are off, if any of the buttons with switch-on-ability are held down they will draw about 150mA each which drains the batteries. So for now I just turn them on separately. The USB port is just for charging, and the XT30 is for attaching an additional battery.

« Last Edit: 15 / March / 2022, 08:59:46 by Recyclojunk64 »

Logged

reyalp

14117

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #3 on: 14 / March / 2022, 20:26:54 »

Quote from: Recyclojunk64 on 14 / March / 2022, 10:06:18

Update: I have now built a rig using 3 Canon G7X Mark II cameras, and the StereoSync code additions are at a usable state for single-shot mode, but still lots features to be added and some bugs to fix with how I'm controlling the slaves.

Nice work, and thanks for posting

Quote

Currently I still need to get the precision sync working. The normal sync precision is usually about 0.1ms to 0.2ms deviation, but sometimes it extends to about 3ms of deviation, without precision sync enabled. How can I go about finding the address for GPIO_VSYNC_CURRENT for a digic 7 processor? I did try the usual value of 0xC0F070C8 that worked for digic 5 and previous cameras but it crashed the G7X2 camera. I have the firmware loaded into Ghidra but not sure what to look for.

Digic 6 and later use a completely different display architecture, so it's not at all surprising the old MMIOs aren't valid, and quite likely you'll need an entirely different method. (Note usb_sync.c assumes GPIO_VSYNC_UPDATE and GPIO_VSYNC_MAX are the same for all cameras)

Quote

Another thing, to initiate shooting on the slave cameras, I am currently simulating button presses on the slave cameras. However this does not work well in continuous shooting mode as the timing of the fullpress release is critical to stop the one of the slaves from going to an additional shot while the other cameras don't. So I would like to find a better way to do this, currently thinking of using levent_post_to_ui_by_name unless anyone has a better idea of how to do this?

I very much doubt using levents will help. If you want to release full press before another shot is triggered in continuous mode, I'd suggest doing it as soon as the remote hook is reached. I'm not clear if you are controlling this from custom C code or script, but from script it would look like what rawopint does.

Logged

Don't forget what the H stands for.

Mlapse

584
S95 S110

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #4 on: 15 / March / 2022, 07:37:02 »

very nice, does the master cam only supply info to the slaves to keep them syncronised? or is it used to make shots as well?

Logged

frustration is a key ingredient in progress

Recyclojunk64

127

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #5 on: 15 / March / 2022, 08:56:35 »

Quote from: reyalp on 14 / March / 2022, 20:26:54

Digic 6 and later use a completely different display architecture, so it's not at all surprising the old MMIOs aren't valid, and quite likely you'll need an entirely different method. (Note usb_sync.c assumes GPIO_VSYNC_UPDATE and GPIO_VSYNC_MAX are the same for all cameras)

I might first test the sync on my spare G7X2 camera with just the standard USB sync (and CRT oscilloscope) to rule out any sync issues with the battery 4th terminal comms, then try to figure out what to do if the problem persists. I'll probably leave this for now cause the continuous-shooting mode is more immediately important as I want to do some exposure stacking for HDR.

Quote

I very much doubt using levents will help. If you want to release full press before another shot is triggered in continuous mode, I'd suggest doing it as soon as the remote hook is reached. I'm not clear if you are controlling this from custom C code or script, but from script it would look like what rawopint does.

Thanks, that sounds pretty easy to implement then so may go that route. It's all custom C code. I'm going to improve the slave state machines so that the master more or less just monitors and requests slave states, rather than sending button presses and releases. Then the slaves can press and release exactly when they need to. For the master, I might mask the full press button then have the master state machine simulate full button presses similar to what the user does but the release delayed until the next remote hook if the next shot has already started.

Quote from: Mlapse on 15 / March / 2022, 07:37:02

very nice, does the master cam only supply info to the slaves to keep them syncronised? or is it used to make shots as well?

The master cam also partakes in the shooting. In case you were wondering the reason I have 3 cameras instead of just 2 is because I wanted to be able to adjust the stereo base for further away or closer objects. So I have 3 different stereo bases which I can pick afterwards when displaying the stereo image. Much less effort than having a slider rail and needing to adjust it all the time.

« Last Edit: 15 / March / 2022, 09:01:16 by Recyclojunk64 »

Logged

Mlapse

584
S95 S110

Re: StereoSync Project - Synchronizing multiple cameras for stereophotgraphy

« Reply #6 on: 15 / March / 2022, 10:50:11 »

Quote from: Recyclojunk64 on 15 / March / 2022, 08:56:35

The master cam also partakes in the shooting. In case you were wondering the reason I have 3 cameras instead of just 2 is because I wanted to be able to adjust the stereo base for further away or closer objects. So I have 3 different stereo bases which I can pick afterwards when displaying the stereo image. Much less effort than having a slider rail and needing to adjust it all the time.

i was thinking of a setup with 2 cams where i combine the usb on one trigger, but this sound much more interesting.
i have a few s110 lying around that could be used.

Logged

frustration is a key ingredient in progress