how to reboot cams in multicam setup? - page 5 - LUA Scripting - CHDK Forum
« previous next »
Pages: 1 2 3 4 [5] 6 7 8   Go Down

how to reboot cams in multicam setup?

  • 70 Replies
  • 13588 Views
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #40 on: 05 / August / 2022, 02:51:25 »
Advertisements
Quote from: Mlapse on 04 / August / 2022, 03:28:13
it makes a picture just before that so it might be shooting related, i made a 1 sec sleep before download and 6 sec after, since it seemed to help prevent i/0 errors. else not much going on.
for your understanding, i tried to clean the code and comments up a bit...if it is still too messy to understand, ask me. and if you want to improve or add something, feel free :)
Thanks. I mostly just wanted to see what the sequence of events was.

Looking at the script, I don't think my original theory can be correct. You are using the 'shoot' command, which doesn't use the hooks. Additionally, since you are using cmdwait, a crash in shoot really cause the connection error to occur in the shoot command, since chdkptp will poll for messages waiting for it to finish.

So that suggests the problem really is happening in imglist. Which is odd. One thing that might be possible is the camera side function is running to long without yielding. The camera side script uses set_yield to keep automatic sleeps from messing with sync. But with lastimg=1, I'd expect the list function to be quick, so I don't really think this is a good theory. Anyway, you try modifying cmds.imglist around the ff_imglist call like
Code: [Select]
set_yield(nil,nil) -- default yield
local status,err = ff_imglist(args)
set_yield(-1,-1) -- no auto yield
edited 2022/8/5: The code above originally used 0,0 for default, which is wrong

If it is really triggered by imglist, another thing to test would be if you can reproduce it just by repeatedly calling imglist with some files on the cam, without doing any of the shooting/downloading etc. I would assume the assert only happens in rec mode.

Quote
it is not for using lastimg, it is for when the pc connects fresh and there are images on the card:
What I meant was, if you use lastimg with DCIM missing, the error you get will refer to the current image directory, otherwise, it will refer to DCIM.

Quote
and i noticed this before, do you use a script to harvest all the lua files posted on this website or is that an external bot?
Are you referring to the download count on attachments? I don't use any scripts to download anything from this site.
« Last Edit: 06 / August / 2022, 00:56:06 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #41 on: 05 / August / 2022, 04:04:18 »
Quote from: reyalp on 05 / August / 2022, 02:51:25
Thanks. I mostly just wanted to see what the sequence of events was.

So that suggests the problem really is happening in imglist. Which is odd. One thing that might be possible is the camera side function is running to long without yielding. The camera side script uses set_yield to keep automatic sleeps from messing with sync. But with lastimg=1, I'd expect the list function to be quick, so I don't really think this is a good theory. Anyway, you try modifying cmds.imglist around the ff_imglist call like
Code: [Select]
set_yield(0,0) -- default yield
local status,err = ff_imglist(args)
set_yield(-1,-1) -- no auto yield
If it is really triggered by imglist, another thing to test would be if you can reproduce it just by repeatedly calling imglist with some files on the cam, without doing any of the shooting/downloading etc. I would assume the assert only happens in rec mode.


It is running with the changed code. i have not seen the imglist hang for seconds since this change. yes i have by now, maybe not as long or frequent as before, but certainly over 1 second. the cams still appear to be synced in shooting though.
....will take more time to know for sure.

the change that is visible is that before this code it would cycle trough a few imglist that were instant resolved and then one or 2 that took longer, and eventually it would stop at one of those long waits.
now it appears to take more time as more shots are being made. i have not seen an instant imglist response in the last hour. and an hour later it averages closer to 4 seconds.
by now it takes more than 10 seconds to complete imglist and there are 350 images recorded per camera.
even if it doesn't crash just yet, after 8 hours of runtime there isn't enough time left between shots to perform imglist and the download...but i'll let it go on until it fails.

i think this is the problem, imglist isn't very efficient with lots of images...or runtime
and that's strange, since i use the same in a local script for the logfile:

string.format('%s/IMG_%04d.JPG',get_image_dir(),get_exp_count())
and that never leads to such long times....something pc side should be flushed?

fyi...while editing this text imglist takes up 12 sec by now.
and probably due to all those sleeps next shot is not triggered in time anymore, but delayed with several seconds....and growing more out of pace with each shot.
around 600 shots it skipped a shot and i stopped the interval.


Quote
What I meant was, if you use lastimg with DCIM missing, the error you get will refer to the current image directory, otherwise, it will refer to DCIM.
at start i think it would be best to know if something is there to download or if something has been downloaded, either way. maybe i can use the error as a true or false.
Quote
Are you referring to the download count on attachments? I don't use any scripts to download anything from this site.
i thought the website might be set up to archive all posted lua files in one place seperate from where the links of uploaded files refer to.
« Last Edit: 05 / August / 2022, 08:18:29 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #42 on: 06 / August / 2022, 00:54:59 »
Quote from: Mlapse on 05 / August / 2022, 04:04:18
It is running with the changed code. i have not seen the imglist hang for seconds since this change. yes i have by now, maybe not as long or frequent as before, but certainly over 1 second. the cams still appear to be synced in shooting though.
....will take more time to know for sure.

the change that is visible is that before this code it would cycle trough a few imglist that were instant resolved and then one or 2 that took longer, and eventually it would stop at one of those long waits.
now it appears to take more time as more shots are being made. i have not seen an instant imglist response in the last hour. and an hour later it averages closer to 4 seconds.
by now it takes more than 10 seconds to complete imglist and there are 350 images recorded per camera.
even if it doesn't crash just yet, after 8 hours of runtime there isn't enough time left between shots to perform imglist and the download...but i'll let it go on until it fails.
So if I understand this, it does not crash with the set_yield change, but takes increasingly long as there are more images?

Quote
i think this is the problem, imglist isn't very efficient with lots of images...or runtime
and that's strange, since i use the same in a local script for the logfile:

string.format('%s/IMG_%04d.JPG',get_image_dir(),get_exp_count())
and that never leads to such long times....something pc side should be flushed?
Yeah, imglist is a lot less efficient than generating the filename directly. With lastimg=1, it will iterate over all images in the current image directory.

Quote
fyi...while editing this text imglist takes up 12 sec by now.
and probably due to all those sleeps next shot is not triggered in time anymore, but delayed with several seconds....and growing more out of pace with each shot.
around 600 shots it skipped a shot and i stopped the interval.
Ooops, the code I gave you was wrong. In ubasic, set_yield 0,0 restores defaults, but in Lua, it just sets them to the lowest (yield as frequently as possible) values. You should use set_yield(nil,nil) instead.

I suspect that will be much better, but if that's still too slow, you could likely speed it up somewhat by adjusting the set_yield parameters, something like
Code: [Select]
set_yield(-1,100) -- yield every 100 ms
local status,err = ff_imglist(args)
set_yield(-1,-1) -- no auto yield
It's not clear how far you can go before hitting the crash again. I'd kind of expect 100ms to be fine, but that's just a guess. The relative benefit drops quickly as you use larger numbers, so I wouldn't bother trying to find the exact highest value that still works.

If that still isn't enough, you should probably write your own command that just generates the full filename directly, like the code you quoted above.

Quote
at start i think it would be best to know if something is there to download or if something has been downloaded, either way. maybe i can use the error as a true or false.
With the existing code, the warning will only be printed. However, download_images does return the list, so you could check if it's empty. Do keep in mind that the list is organized by camera ID, and different cameras could have different numbers of images.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #43 on: 06 / August / 2022, 02:16:16 »
Quote from: reyalp on 06 / August / 2022, 00:54:59
So if I understand this, it does not crash with the set_yield change, but takes increasingly long as there are more images?
true as far as i understand, otherwise it is some memory leak issue that just grows when time passes
Quote
Yeah, imglist is a lot less efficient than generating the filename directly. With lastimg=1, it will iterate over all images in the current image directory.

If that still isn't enough, you should probably write your own command that just generates the full filename directly, like the code you quoted above.
i don't understand: getlastimg with that code is already in the script. how is it that it still iterated over the files instead of using that directly generated code when i call it with download_images({lastimg=1,?

i'd think something like this:  ::) ALERT: incomplete and not working code below  ::)
if lastimg==1 then printf('only one') local list=self:cmd('getlastimg') else -- list all images local list=self:imglist(opts) end

but if i understand you, i have to write a function mc:download_last_image()?
a new download_images command that specifically only uses getlastimg and does not have cmds.imglist?
Quote
With the existing code, the warning will only be printed. However, download_images does return the list, so you could check if it's empty. Do keep in mind that the list is organized by camera ID, and different cameras could have different numbers of images.
ill see if i can make a true or false from download_images...don't care how many files are actually downloaded. just want to know if there has been something downloaded.
« Last Edit: 06 / August / 2022, 09:45:01 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #44 on: 06 / August / 2022, 16:01:20 »
Quote from: Mlapse on 06 / August / 2022, 02:16:16
i don't understand: getlastimg with that code is already in the script. how is it that it still iterated over the files instead of using that directly generated code when i call it with download_images({lastimg=1,?
Because download_images was written download images based on a bunch of possible criteria, re-using the capabilities and code of the imdl command.

Quote
i'd think something like this:  ::) ALERT: incomplete and not working code below  ::)
if lastimg==1 then printf('only one') local list=self:cmd('getlastimg') else -- list all images local list=self:imglist(opts) end
I'd suggest trying with the corrected set_yield mentioned in the previous post first. It's inefficient, but if it's fast enough you don't have to worry about it.

Quote
ill see if i can make a true or false from download_images...don't care how many files are actually downloaded. just want to know if there has been something downloaded.
Well, you have to decide what condition is true or false: Any camera has/does not have images? All cameras have/do not have images? Of course if everything works as planned, every camera should have the same number of images, but you should probably decide what happens if that isn't the case.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #45 on: 06 / August / 2022, 16:13:23 »
Quote from: reyalp on 06 / August / 2022, 16:01:20
I'd suggest trying with the corrected set_yield mentioned in the previous post first. It's inefficient, but if it's fast enough you don't have to worry about it.
still takes up a lot of time, hasn't run long enough yet to skip frames though. so it is an improvement. just not enough for my setup.
« Last Edit: 06 / August / 2022, 16:15:27 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #46 on: 06 / August / 2022, 22:57:22 »
Quote from: Mlapse on 06 / August / 2022, 16:13:23
still takes up a lot of time, hasn't run long enough yet to skip frames though. so it is an improvement. just not enough for my setup.
Fair enough. If you just want to download the image identified by cmds.getlastimg, you could use something like
Code: [Select]
function mc:download_last_image(opts)
local opts=util.extend_table({},opts)
local status, rstatus = mc:cmdwait('getlastimg')
if not status then
warnf('getlastimg failed\n')
return
end
for lcon in mc:icams() do
if rstatus[lcon.mc_id] then
local fn=rstatus[lcon.mc_id].status.msg
local lfn=('%02d_%s'):format(lcon.mc_id,fsutil.basename_cam(fn))
if opts.pretend or opts.verbose then
printf('%d: %s->%s\n',lcon.mc_id,fn,lfn)
end
if not opts.pretend then
local status,err=lcon:download_pcall(fn,lfn)
if not status then
warnf('%d: download failed %s\n',lcon.mc_id,tostring(err))
end
end
else
warnf('%d: missing status\n',lcon.mc_id)
end
end
end
Note that this is missing many of the features of download_images
- getlastimg simply generates the filename based on the current image directory and file counter. The file may or may not exist. If it doesn't exist, you'll get a warning from the download command.
- It does not include subst string renaming logic of download_images. You have to construct the local (PC side) filename yourself, in the line that sets lfn.
- The file creation time will be when the file was downloaded, rather than preserving the camera time.
- If the local filename contains sub-directories, they are not automatically created. You could use fsutil.mkdir_parent to do this.
- download_images lastimg downloads any images (raw, dng, jpeg) with a matching image number. getlastimg is hardcoded to only do the jpeg.

I might write a better version of this later and add it to multicam. Or I might not  ;)
« Last Edit: 06 / August / 2022, 23:07:02 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #47 on: 07 / August / 2022, 12:49:12 »
Quote from: reyalp on 06 / August / 2022, 22:57:22
Fair enough. If you just want to download the image identified by cmds.getlastimg, you could use something like
Code: [Select]
function mc:download_last_image(opts)
local opts=util.extend_table({},opts)
local status, rstatus = mc:cmdwait('getlastimg')
if not status then
warnf('getlastimg failed\n')
return
end
for lcon in mc:icams() do
if rstatus[lcon.mc_id] then
local fn=rstatus[lcon.mc_id].status.msg
local lfn=('%02d_%s'):format(lcon.mc_id,fsutil.basename_cam(fn))
if opts.pretend or opts.verbose then
printf('%d: %s->%s\n',lcon.mc_id,fn,lfn)
end
if not opts.pretend then
local status,err=lcon:download_pcall(fn,lfn)
if not status then
warnf('%d: download failed %s\n',lcon.mc_id,tostring(err))
end
end
else
warnf('%d: missing status\n',lcon.mc_id)
end
end
end
Note that this is missing many of the features of download_images
- getlastimg simply generates the filename based on the current image directory and file counter. The file may or may not exist. If it doesn't exist, you'll get a warning from the download command.
- It does not include subst string renaming logic of download_images. You have to construct the local (PC side) filename yourself, in the line that sets lfn.
- The file creation time will be when the file was downloaded, rather than preserving the camera time.
- If the local filename contains sub-directories, they are not automatically created. You could use fsutil.mkdir_parent to do this.
- download_images lastimg downloads any images (raw, dng, jpeg) with a matching image number. getlastimg is hardcoded to only do the jpeg.

I might write a better version of this later and add it to multicam. Or I might not  ;)

i'll work on tweaking and testing it for my needs...some smart delay to prevent timeouts when >10 second exposure shots are made. 

was trying to change the output path since it is hard to test for days with this, but since that gave an i/o error i thought i had the syntax wrong.
the path seemed correct to me when printed.
Code: [Select]
local lfn=('%02d/%s/IMG_%s.JPG'):format(lcon.mc_id,string.sub(fsutil.dirname_cam(fn),8),imagenr)
after some other tries that failed, i set it with an extra fsutil assuming that there couldn't be anything wrong with that path, other than an extra A/DCIM but it gave the same i/o error
Code: [Select]
local lfn=('%02d/%s/%s'):format(lcon.mc_id,fsutil.dirname_cam(fn),fsutil.basename_cam(fn))
so haven't had much luck yet with altering lfn, the only change that was succesfull was turning _ in / ......but that's probably me again, i'll try some more and as soon as it works put it to the test.
« Last Edit: 07 / August / 2022, 17:57:58 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #48 on: 07 / August / 2022, 18:10:01 »
Quote from: Mlapse on 07 / August / 2022, 12:49:12
was trying to change the output path since it is hard to test for days with this, but since that gave an i/o error i thought i had the syntax wrong.
It's a lot more likely I'll be able to help if you quote the specific error. However, in this case, my first guess would be this
Quote
- If the local filename contains sub-directories, they are not automatically created. You could use fsutil.mkdir_parent to do this.
For example, something like
Code: [Select]
fsutil.mkdir_parent(lfn)
local status,err=lcon:download_pcall(fn,lfn)
Most of the functions in fsutil are at least somewhat documented, and you can find examples by searching for where they are used in the source. You might find fsutil.parse_image_path_cam useful.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #49 on: 08 / August / 2022, 02:46:21 »
this is your code and downloads as expected, if i change _ to / it downloads into a folder named with cam id. so far, so good:
Code: [Select]
local lfn=('%02d_%s'):format(lcon.mc_id,fsutil.basename_cam(fn))

so then i wrote in an extra /%s and added the folder structure, tried other variations on this as well, all ending with the same I/O error

for the output herebelow, this is what i used......i couldn't come up with anything more elegant yet with the same output ;)
Code: [Select]
local lfn=('%02d/%s/IMG_%04d.JPG'):format(lcon.mc_id,string.sub(fsutil.dirname_cam(fn),8),imagenr) 
printf('%d: %s->%s\n',lcon.mc_id,fn,lfn)

this is the output:

Quote
getlastimg
1: true msg A/DCIM/101_0808/IMG_0001.JPG
2: true msg A/DCIM/101_0808/IMG_0001.JPG
1: A/DCIM/101_0808/IMG_0001.JPG->01/101_0808/IMG_0879.JPG
WARNING: 1: download failed I/O error
2: A/DCIM/101_0808/IMG_0001.JPG->02/101_0808/IMG_0879.JPG
WARNING: 2: download failed I/O error
ERROR: call failed:I/O error

ERROR: error on line 7
attempted to close non-present device 001:007

as you can see output path seems to have the correct structure, but nothing is downloaded.
the mistake i made was that folder 01 and 02 were already there from other tests, so it did not have to be created. and thus i did not notice the code was missing something to create folders.
despite your remark #5...i was still mulling over #3 that it could become a hassle that the cam time was lost and i mistakenly thought that I/O would always be related to the cam connection.
i'll include the mkdir and try again later.
« Last Edit: 08 / August / 2022, 12:01:22 by Mlapse »
Logged
frustration is a key ingredient in progress
Pages: 1 2 3 4 [5] 6 7 8   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal