limiting factors of CHDK for extended time-lapse

Sdack

195

Re: limiting factors of CHDK for extended time-lapse

« Reply #70 on: 17 / August / 2018, 22:43:18 »

Advertisements

Ok I'm strugging already, in my bash script to retrieve the list of images on the camera
I can connect with the camera using this bash script

Code: [Select]

#!/bin/bash
/root/chdkptp/chdkptp.sh -c -e"rec"

But I'm not sure how to get the list of files.

I found the ls command in the CHDKPTP help file but it doesn't explain recursion and lua doesn't appear to accept the * wildcard like bash does ie.

Code: [Select]

ls DCIM/*.jpg
In an interactive CLI session, I can run ls DCIM but it only gives a list of subdirectories.
I need to retrieve and store a list of all individual images as a text file, or CSV, or maybe JSON.
Any pointers?
I tried to find commands in these pages
Lua/PTP Scripting http://chdk.wikia.com/wiki/Lua/PTP_Scripting
CHDK scripting http://chdk.wikia.com/wiki/CHDK_scripting#shut_down
Is there a more complete guide somewhere?
Thanks in advance
Sdack

Logged

reyalp

14080

Re: limiting factors of CHDK for extended time-lapse

« Reply #71 on: 17 / August / 2018, 23:42:05 »

I'd suggest the imdl command, which by default automatically downloads all images under the DCIM directory.

There are options to control which files are downloaded and how they are named. Use help imdl for details.

The imls command lists images.

https://app.assembla.com/spaces/chdkptp/wiki/CLI_Quickstart has some general information about chdkptp commands

Logged

Don't forget what the H stands for.

Sdack

195

Re: limiting factors of CHDK for extended time-lapse

« Reply #72 on: 18 / August / 2018, 04:27:54 »

Awesome thanks

Logged

Sdack

195

Re: limiting factors of CHDK for extended time-lapse

« Reply #73 on: 18 / August / 2018, 06:35:44 »

Hello again,
I just spent an entire evening trying to figure out how to write out a list of images on the camera out to a file that I can download to my Raspberry Pi. I would simply have downloaded the images and continued from there on the Raspberry Pi but it takes a minute and a half to download 60 images, which, the way I'm managing the power, would mean having to skip an image capture. If I only grab the image list and the most recent image, each hour, I can be in and out in a matter of seconds and leave the main download til the end of the day.

Not knowing how to invoke this as a CHDKPTP command, I thought it might be simper to us PTP to invoke a camera side lua script and stumbled across this promising thread..

https://chdk.setepontos.com/index.php?topic=3029.msg28490#msg28490

To my eyes the following code from the above link would seem to create a list in a file named dcimList

Code: [Select]

--checks directories in DCIM for image files
function getImageList()
 log.print("-start- getImageList \n\n")
 local imgTable = {}
 --get directories that may contain images
 local dcimList = os.listdir("A/DCIM", false)
 if(dcimList) then
  local dirCount = table.getn(dcimList)
  log.print("dirCount:"..dirCount.."\n")
  table.sort(dcimList)
   --loop through directories
   local i = 0
   while ( dirCount ~= i) do
    i = i + 1
    --get file list
    log.print("A/DCIM/"..dcimList[i].."\n")
    local imgDirList = os.listdir("A/DCIM/"..dcimList[i], false)
    if(imgDirList) then
     local imgCount = table.getn(imgDirList)
     log.print("imgCount:"..imgCount)
     table.sort(imgDirList)
     if(imgCount ~= nil) then
      --loop through files, add to list
      local a = 0
      while (imgCount ~= a) do
       a = a + 1
       print("A/DCIM/"..dcimList[i].."/"..imgDirList[a])
       log.print("A/DCIM/"..dcimList[i].."/"..imgDirList[a])
       table.insert(imgTable,"A/DCIM/"..dcimList[i].."/"..imgDirList[a])
      end
     else
      os.remove("A/DCIM/"..dcimList[i])
      table.remove(dcimList,i)
     end
    else
     os.remove("A/DCIM/"..dcimList[i])
     table.remove(dcimList,itemCount)
    end
   end
 end
 
 log.print("-end- getImageList ")
 return imgTable
end

However, when I invoke the script over PTP like this

Code: [Select]

=loadfile('A/CHDK/SCRIPTS/NH_list.lua')()I get no errors but no list in a file either. Is it intended to be used 'on the camera' I did try running it that way too but still got no text file output.
The code is rather old so maybe things have changed, or I'm missing something
Cheers
S

Logged

waterwingz

12138

Re: limiting factors of CHDK for extended time-lapse

« Reply #74 on: 18 / August / 2018, 10:13:40 »

Quote from: Sdack on 18 / August / 2018, 06:35:44

Not knowing how to invoke this as a CHDKPTP command, I thought it might be simper to us PTP

I have no doubt using a CHDKPTP command would be easier. Reyalp can explain that in his sleep so I won't bother puzzling it out myself. Meanwhile, I've attached a simple bash script that will dump the image list from the command line to your screen.

Quote

to invoke a camera side lua script and stumbled across this promising thread..
...
However, when I invoke the script over PTP like this
Code: [Select]
=loadfile('A/CHDK/SCRIPTS/NH_list.lua')()I get no errors but no list in a file either. Is it intended to be used 'on the camera' I did try running it that way too but still got no text file output.

What you cut&pasted was just a function. If you put that into a file and try to run it as a Lua script, you need to call the function from within the script.

I took a quick look and the code you found assumes some other files (libraries) are also loaded so it won't run "as is" in any case.

I hacked something to make it work - attached. It works but note that it gets all files stored under the A/DCIM folder - images, videos, and some Canon data files ending in .CTG. I wasn't going to spend too much time on it because the CHDKPTP was is bound to be a better idea.

NH_list.lua (1.63 kB - downloaded 9 times.)

ptpls.sh (0.06 kB - downloaded 13 times.)

« Last Edit: 18 / August / 2018, 10:35:41 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14080

Re: limiting factors of CHDK for extended time-lapse

« Reply #75 on: 18 / August / 2018, 16:08:25 »

Quote from: Sdack on 18 / August / 2018, 06:35:44

Hello again,
I just spent an entire evening trying to figure out how to write out a list of images on the camera out to a file that I can download to my Raspberry Pi. I would simply have downloaded the images and continued from there on the Raspberry Pi but it takes a minute and a half to download 60 images, which, the way I'm managing the power, would mean having to skip an image capture. If I only grab the image list and the most recent image, each hour, I can be in and out in a matter of seconds and leave the main download til the end of the day.

So you want to download one hourly image and then download all the images at the end of the day?

With imdl, you can use -last=1 to download the most recently shot image. Note that "last" is based on the file counter so might have issues when the folder changes, but I think with one shot it should be OK.

If you want lists, the imls command takes most of the same sorting and filtering options as imdl. You can generate a list sorted by date and just download the last line.

Quote

Not knowing how to invoke this as a CHDKPTP command, I thought it might be simper to us PTP to invoke a camera side lua script and stumbled across this promising thread..

https://chdk.setepontos.com/index.php?topic=3029.msg28490#msg28490

Just a caution, that thread is from 2009. That doesn't mean it's wrong, but better options are often available.

Quote

I get no errors but no list in a file either. Is it intended to be used 'on the camera' I did try running it that way too but still got no text file output.

I'm not sure exactly what you put in your file, but if it was all the code in the code tags, it just defines the functions without invoking them. It also assumes there is a "log" module which does something (probably posted earlier in the thread)

I'd suggest using chdkptp's file listing function will be the more straightforward path.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: limiting factors of CHDK for extended time-lapse

« Reply #76 on: 18 / August / 2018, 16:10:42 »

Quote from: reyalp on 18 / August / 2018, 16:08:25

Quote
I get no errors but no list in a file either. Is it intended to be used 'on the camera' I did try running it that way too but still got no text file output.
I'm not sure exactly what you put in your file, but if it was all the code in the code tags, it just defines the functions without invoking them. It also assumes there is a "log" module which does something (probably posted earlier in the thread)

..echo ...

Logged

Ported : A1200 SD940 G10 Powershot N G16

Sdack

195

Re: limiting factors of CHDK for extended time-lapse

« Reply #77 on: 18 / August / 2018, 18:54:27 »

Thanks for the suggestions,

I had one of those sleepless nights where the ideas kept going around in my head.

I got up eventually to write down some pseudo code before going back to sleep.

I haven't checked it this morning yet.. might well be gobbledygook.

Mostly it was about how to manage the sharing of duties between the Raspberry Pi and the microprocessor. I feel there's great potential for a super low power draw device, without losing all the benefits of internet connectivity.

If I can use the Raspberry Pi and data modem for a total of only 1 hour a day (2Wh) and the Arduino and camera for the remainder of the time (23hs * .2Wh), my total daily draw will be @ 2.46

I hope to be able to power the system by a single, lithium phosphate cell, at 3.2V, buck converting up to 5V and 3.7V. with a modest 6V solar panel providing renewable supply.
Am I crazy?

Logged

waterwingz

12138

Re: limiting factors of CHDK for extended time-lapse

« Reply #78 on: 18 / August / 2018, 19:11:36 »

Quote from: Sdack on 18 / August / 2018, 18:54:27

If I can use the Raspberry Pi and data modem for a total of only 1 hour a day (2Wh) and the Arduino and camera for the remainder of the time (23hs * .2Wh), my total daily draw will be @ 2.46. I hope to be able to power the system by a single, lithium phosphate cell, at 3.2V, buck converting up to 5V and 3.7V. with a modest 6V solar panel providing renewable supply.
Am I crazy?

If you power the camera off except for the six seconds where it's actually shooting you could do better than that? Except for the wear & tear on the mechanism.

The other question might be why you need the Pi running for a whole hour? I guess it comes down to image size, how many images per day (assuming you don't shoot in the dark), and the data transfer rate of the modem? Going to a higher compression on the image size will cost you a bit on quality but might be worth the trade off?

« Last Edit: 18 / August / 2018, 19:23:40 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

Sdack

195

Re: limiting factors of CHDK for extended time-lapse

« Reply #79 on: 18 / August / 2018, 21:30:28 »

Quote

If you power the camera off except for the six seconds where it's actually shooting you could do better than that? Except for the wear & tear on the mechanism.

The 23hs * 0.2Wh is the Arduino booting the Ixus autorunning the Oneshot script which adds up to 0.46 Wh.

Quote

The other question might be why you need the Pi running for a whole hour? I guess it comes down to image size, how many images per day (assuming you don't shoot in the dark), and the data transfer rate of the modem? Going to a higher compression on the image size will cost you a bit on quality but might be worth the trade off?

The full hour of the Raspberry Pi is based on booting up for 5 minutes each hour during the day (12 hours * 5 mins = 60 mins). A 5 minute window would give me regular reporting plus the option to log in and reconfigure the interval.
I could also interrupt the program to extend the window for more fiddling or housework.

I tested image transfer speed yesterday and got 60 full resolution images uploaded in 90 seconds, so I figured I'd leave the total at a round 60 minutes.

One of my 'sleepless inspirations' was to get the Raspberry Pi to pull all images from the camera during each 5 min window and to upload 6 images, one for each ten minutes ie. 06:30, 06:10, 06:20, 06:30, 06:40, 06:50

Dream inspired workflow

Initial power starts Raspberry Pi which boots to idle mode.
A cron job starts an edited version of multilapse.
The edited bash script uploads any existing ten-minute-images to the cloud.
multilapse powers up the camera via the relay
The camera autoruns Oneshot.lua.
The camera senses power at the USB port and enters CHDKPTP mode.
multilapse.lua runs the camera re-configured to take 5 minutes worth of images.
Prior to shutdown, the Pi switches a latching relay to wake the Arduino.
Arduino boots the camera.
The camera autoruns Oneshot.lua.
Lack of power at the USB port initiates quick shot and shutdown.
The Arduino repeats this sequence 55 times, taking us to around 40 seconds before the next hour.
The Arduino switches on the relay for the Raspberry Pi.
The Arduino toggles it's own power relay to off (I can't see why this isn't possible. It's a microprocessor, there's no hard disk to corrupt.
After shooting hours, a different cron job will run on the Raspberry Pi, to upload the remaining images then it will power up the Arduino and shut down. The Arduino will have a real time clock, with the working day programmed into it, so it knows to sleep til morning.

This method relies on both machines being in reasonable clock sync but I suspect it's possible to achieve this.

Any suggestion welcome
Sdack

« Last Edit: 18 / August / 2018, 21:33:57 by Sdack »

Logged