is it possible to download images while a script is running? - General Discussion and Assistance - CHDK Forum

is it possible to download images while a script is running?

  • 5 Replies
  • 109 Views
*

Offline Mlapse

  • ****
  • 318
  • S95 S110
Advertisements
I have this setup that takes one shot every 30 seconds.
downloading and erasing the same sd card would loose a fair amount of time.
hence, i have 2 identical sd cards that I exchange.
but I would like to minimize contact with the camera.

remote shoot with chdkptp is naturally an option.

getting even larger sd cards would also be an option, but i don't feel much for this one.

and so I was wondering
would it be possible to make a setup with chdkptp or some other way where i connect a laptop to the usb and that files would be automatically downloaded and erased between shots..
that would give me some 25 seconds between shots that the script is waiting.
i assume it involves switching to playback and back...so that might cost some time.

don't know how that should work and/or if that is possible, maybe per 5 or 10 at a time and then wait until after the next shot has been saved by the script?
« Last Edit: Yesterday at 16:14:29 by Mlapse »
frustration is a key ingredient in progress

*

Offline reyalp

  • ******
  • 13394
Re: is it possible to download images while a script is running?
« Reply #1 on: Yesterday at 18:30:53 »
remote shoot with chdkptp is naturally an option.
If you decide to go that route, the "glue script" approach used by dolomiti_timelapse lets you use existing chdk scripts mostly unmodified https://chdk.setepontos.com/index.php?topic=14302.0
Your mileage may vary depending on the specific script

manoweb's webcams are an example of an an alternative approach, using a custom script driven by the host side (https://chdk.setepontos.com/index.php?topic=14302.msg145505#msg145505https://chdk.setepontos.com/index.php?topic=14302.msg147074#msg147074)

The choice between these is mostly personal preference, IMO.
Quote
would it be possible to make a setup with chdkptp or some other way where i connect a laptop to the usb and that files would be automatically downloaded and erased between shots..
that would give me some 25 seconds between shots that the script is waiting.
i assume it involves switching to playback and back...so that might cost some time.
It should be possible, but coordinating between the script and download process will require some thought and experimentation.

You do not have to switch to playback to download files with chdkptp. Downloading while the camera shoots might even be OK, although I wouldn't recommend it.

You can technically download while a (non-ptp aware) script is running using dl -nolua or con:download(), but you must know the exact filename and path beforehand. The multiple download commands mdl and imdl need to run a camera-side script to list the files.

An alternative is to make your own camera side script provide the list of files, like multicam.lua download_images.

Quote
don't know how that should work and/or if that is possible, maybe per 5 or 10 at a time and then wait until after the next shot has been saved by the script?
A few considerations:
The "connection error" "communication error" (edit: corrected message) limit. As described in https://chdk.setepontos.com/index.php?topic=14302.msg147190#msg147190 the camera may not allow a USB connection if it thinks too many files are present. One image every 30s for one day = 2880 which may or may not be OK. More than one day would likely fail. If the camera isn't switched to play back, this error may be avoided, but some cameras crash when you try to switch to play or shutdown after deleting files in rec. Dealing with this will require experimentation with your particular setup.

If the camera isn't connected to USB all the time, you need to be careful with play/rec switching. By default, when you connect USB, the camera will attempt to switch to play mode. Additionally, if you switch with normal switching functions after switching while USB is present, switching may fail.

You could hide the USB status until the camera script is ready using the USB remote setting (with set_config_value, clearing it when the script is ready after detecting usb power), or force the camera to think it's always present using usb_force_present(1) (see https://chdk.fandom.com/wiki/Lua/PTP_Scripting)

To download images, your script needs provide the PC side with a list, or you need to exit and restart the script between downloads so you can use something like imdl/mdl.

Since you want to download a few images at a time, and need support in the script to handle USB showing up anyway, I'd lean to making the script provide the file list.

Providing a file list also needs some thought. It will likely be too large to for the camera side script to keep in RAM. The chdkptp multi-file download scripts handle by sending batches of filenames in multiple messages, with the PC side collecting the full list. Re-using this code should be straightforward (take ff_imglist and its dependencies from chdkptp rlibs, and borrow the multicam download_images code), but if you have a lot of files, just building the list could take longer than your shot interval.

Another alternative would be to have your script log each shot to a file, and have the PC side download that.

Putting this together, I'd probably go for something like this:
script uses usb_force_present(1) at startup to prevent USB connections from messing up play/rec state, and uses switch_mode_usb to ensure mode switching is compatible

Script main loop checks for messages with read_usb_msg. Using the timeout option, you can use this as a replacement for the regular sleep in your interval script, since it just returns nil if no messages are received in the specified time.

When you connect the PC, the chdkptp script sends a message, and the camera side script responds with file info when it's ready for downloads to begin. If necessary, you can use additional message to tell the script when a batch of files is done etc.

Another alternative approach would be to use a wifi SD card.

OK, that was a long rambling post, I hope some of it is useful  :-[
« Last Edit: Today at 19:02:19 by reyalp »
Don't forget what the H stands for.

*

Offline Mlapse

  • ****
  • 318
  • S95 S110
it most definately is usefull.
now see if i can be equally usefull by putting all you wrote to good use.

although my feeling is that remote shoot is very professional and the fact that you can see "live" what the cam is doing is very appealing.
i think i like the approach of downloading during a running script better in my case because the camera setup can remain lean.

a file list seems fairly easy, as you wrote: i could use the log file. after adding get_image_dir() with each line it should have all i need.
the only trouble is naturally that the log file keeps growing after you downloaded it, but it's not that important at this moment.

looked into the wifi card approach but then i think the remote shoot would be more reliable...and with a wifi card there still has to be a dedicated pc since card sizes are small and range is limited.

the connection error might be the real trouble, i was thinking of using cards that allowed me to take 2 weeks worth of pictures, giving me some flexibility in when to download......that's around 40.000 jpgs, but planning to remove them once a week when the card is only half full. I'm guessing i'll have to test that first. because if that does not work the whole idea stops.......1000 pictures is not much in my setup....each day just after 8:15 AM I'm there i'd say
hopefully that's as conservative as the number of shots a camera can make according to canon.
but I have had a similar experience with my a480..that i could not download from the cam after long shoots and had to use a card reader to get the pictures off....and those were around 20.000

either way i'm going to work on a chdkptp setup this winter.  8) ??? :P

« Last Edit: Today at 04:33:54 by Mlapse »
frustration is a key ingredient in progress

*

Offline Caefix

  • *****
  • 604
  • Sorry, busy deleting test shots...
the only trouble is naturally that the log file keeps growing after you downloaded it,
Code: [Select]
function oneline(file,line)
file=file or "A/oneline.log"
line=line or string.format("%s/IMG_%04d.JPG",get_image_dir(),get_exp_count())
   w=io.open(file,"w")
   w:write(line)
   w:close()
end

oneline()
Or You use a Oneline.log  :), could be enhanced to a Batchfile.bat, called from Hostlua with os.execute()

connect a laptop to the usb and that files would be automatically downloaded and erased between shots..
"Erased" is not the same for CHDK & Canon until reboot().  :-[
Maybe script could reboot to a sleep(100000) at the end of daywork before it awakes(tomorrow) 50secs earlier with a sleep(50000) first...
« Last Edit: Today at 13:37:14 by Caefix »
All lifetime is a loan from eternity.


*

Offline Mlapse

  • ****
  • 318
  • S95 S110
i dunno, 20.000+ oneline logs are a bit much for me, but i could always give it a try if it becomes an issue.

i understand that erase will only take effect after a reboot, but that has to happen anyway, so that's all sweet.
probably best to let the reboot be triggered after the last files & folders are removed, so it starts clean with a 100_XXXX folder.


frustration is a key ingredient in progress

*

Offline reyalp

  • ******
  • 13394
the connection error might be the real trouble, i was thinking of using cards that allowed me to take 2 weeks worth of pictures, giving me some flexibility in when to download......that's around 40.000 jpgs, but planning to remove them once a week when the card is only half full. I'm guessing i'll have to test that first. because if that does not work the whole idea stops.......1000 pictures is not much in my setup....each day just after 8:15 AM I'm there i'd say
hopefully that's as conservative as the number of shots a camera can make according to canon.
Yes, I think this is a real concern, you'll want to test before investing too much time in the other parts.  My impression is that 2000-ish is generally OK, but I haven't tested sufficiently to know if the limit is consistent for a given camera.

This would be a worthwhile area for reverse engineering. I assume the limit has something to do with handling standard PTP "objects". Since CHDK PTP doesn't depend on them, it's possible we could bypass it somehow. But it's annoying to test...

Note the actual error is "communication error" rather than "connection" as I stated earlier.
Quote
but I have had a similar experience with my a480..that i could not download from the cam after long shoots and had to use a card reader to get the pictures off....and those were around 20.000
Yes, I would definitely expect 20,000 to be a problem, but as I mentioned above, you may not encounter it if you don't switch to playback.
Don't forget what the H stands for.

 

Related Topics