how to reboot cams in multicam setup? - page 6 - LUA Scripting - CHDK Forum
« previous next »
Pages: 1 2 3 4 5 [6] 7 8   Go Down

how to reboot cams in multicam setup?

  • 70 Replies
  • 13594 Views
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #50 on: 09 / August / 2022, 00:19:10 »
Advertisements
Quote from: Mlapse on 08 / August / 2022, 02:46:21
this is your code and downloads as expected, if i change _ to / it downloads into a folder named with cam id. so far, so good:
As I said before If the local filename contains sub-directories, they are not automatically created. You could use fsutil.mkdir_parent to do this.

That means that download will fail unless all the directory structure in the path already exists.

Quote
despite your remark #5...i was still mulling over #3 that it could become a hassle that the cam time was lost and i mistakenly thought that I/O would always be related to the cam connection.
No, I/O error can refer to local files too.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #51 on: 10 / August / 2022, 14:37:20 »
i have by now tested your new code that uses getlastimg for downloads and it works as expected.

although it has only run for one day that's already 3x longer than the best run before.
it looks solved.
gone are the skipped frames, i/o errors and other conflicts that popped up by requesting a list of more than a few hundred images between each interval.  :D
there always seems plenty of time left now.
« Last Edit: 10 / August / 2022, 14:49:04 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #52 on: 11 / August / 2022, 02:09:21 »
while i've seen no i/o errors between shots while trying to download alas it didn't hold up....after half a day of making 120 images before play/delete/record it got an i/o error again.

and yes, this is similar to what i experienced with the download, although it deletes every 120 frames and was hoping it wouldn't count the deleted ones again..or take less time.
but time to create the list still keeps growing, it also indexes the no longer available images. This means that there is no advantage to delete files early with this method.
was a bit to be expected.
is there another way to solve this or is the only solution to delete all before reboot?
i won't use delete/lastimg, because that would result in one move to play and back to record per shot and i think that's a bit too stressfull for the hardware.

i know by now it takes some 5 minutes to delete a days worth of images...and thought that was a bit long.
but my ideas to delete in parts were not well considered.


« Last Edit: 11 / August / 2022, 15:40:27 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #53 on: 11 / August / 2022, 20:17:57 »
Quote from: Mlapse on 11 / August / 2022, 02:09:21
while i've seen no i/o errors between shots while trying to download alas it didn't hold up....after half a day of making 120 images before play/delete/record it got an i/o error again.
Where in the script did the IO error occur (shooting, downloading, switching between play and rec, etc) ?
Is the IO error a camera crash, or something else?
If it's a camera crash, check the romlog. If it's not the same (identical ASSERT and line number) as posted previously, post it.

Quote
but time to create the list still keeps growing, it also indexes the no longer available images. This means that there is no advantage to delete files early with this method.
Yes, deleting from CHDK without rebooting leaves "ghost" images until reboot.

Quote
is there another way to solve this or is the only solution to delete all before reboot?
Deleting through higher level Canon functions should avoid "ghost" images. Two possible ways of doing this are using key clicks to navigate the menus and delete all, or the previously mentioned PT_EraseAllFile. Both of these options have significant drawbacks: Navigating the menu with key presses tends to be unreliable, and PT_EraseAllFile has the various caveats and implementation difficulties discussed in the linked thread.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #54 on: 12 / August / 2022, 02:41:02 »
Quote from: reyalp on 11 / August / 2022, 20:17:57
Where in the script did the IO error occur (shooting, downloading, switching between play and rec, etc) ?
Is the IO error a camera crash, or something else?
If it's a camera crash, check the romlog. If it's not the same (identical ASSERT and line number) as posted previously, post it.

cam crash after moving to play, but before delete had done it's job.

Quote
PT_EraseAllFile has the various caveats and implementation difficulties discussed in the linked thread.
i'll read up.....
not relevant until i am ready to try:
if i understand correctly i have to allocate memory for this to work and that is cam specific.
is there a method for me to check what memory range is usable for a specific cam model or is that beyond me?
« Last Edit: 12 / August / 2022, 06:24:17 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #55 on: 13 / August / 2022, 01:21:02 »
Quote from: Mlapse on 12 / August / 2022, 02:41:02
cam crash after moving to play, but before delete had done it's job.
That appears to be a completely different crash
Code: [Select]
ASSERT!! ReadFDir.c Line 466
Occured Time  2022:08:11 05:00:08
Task ID: 18874410
Task name: PhySw
The fact that it's in PhySw strongly suggests it was triggered by CHDK code, probably script.

The assert appears to be an out of memory error.

Can you tell what chdkptp side call was in progress when the error occurred?

In any case, the first thing to check is if these cameras have low free memory generally. You could use
Code: [Select]
mc:cmdwait('call return get_meminfo()')

If the camera has reasonable free memory, it's likely the error is caused by Lua garbage collection not being aggressive enough. You could try calling collectgarbage('collect') prior the point where the error occurred, like
Code: [Select]
mc:cmdwait('call collectgarbage("collect")')

It's also possible that listing the files to delete triggers the issue. There's some knobs that can be twisted there too, but I'd try the above first.

Quote
if i understand correctly i have to allocate memory for this to work and that is cam specific.
is there a method for me to check what memory range is usable for a specific cam model or is that beyond me?
I'll have to read up myself to remember how that worked  ;)
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #56 on: 13 / August / 2022, 02:09:36 »
Quote from: reyalp on 13 / August / 2022, 01:21:02
Can you tell what chdkptp side call was in progress when the error occurred?

In any case, the first thing to check is if these cameras have low free memory generally. You could use
If the camera has reasonable free memory, it's likely the error is caused by Lua garbage collection not being aggressive enough. You could try calling collectgarbage('collect') prior the point where the error occurred, like
It's also possible that listing the files to delete triggers the issue. There's some knobs that can be twisted there too, but I'd try the above first.

it had just finished moving to play and probably was busy listing the files, it had not yet deleted them, since the images were on the camera after the crash.

i can imagine it's the sheer volume of pictures that causes low free memory with some 600-800 on the card.
this makes me wonder if it is at all possible to create a stable setup with a pc connected in my situation where eventually it will make almost 3000 shots a day.
most of these issues i've encountered so far revolve around 'too much'

Quote
I'll have to read up myself to remember how that worked  ;)

not needed yet i have to figure out other things first.
« Last Edit: 13 / August / 2022, 02:26:22 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #57 on: 13 / August / 2022, 17:00:49 »
Quote from: Mlapse on 13 / August / 2022, 02:09:36
it had just finished moving to play and probably was busy listing the files, it had not yet deleted them, since the images were on the camera after the crash.

i can imagine it's the sheer volume of pictures that causes low free memory with some 600-800 on the card.
If the port has a reasonable amount of free memory, it the sheer volume of pictures should not be a problem. As mentioned in my previous post, you should check how much RAM is free under "normal" circumstances.

However, the Lua garbage collector doesn't know the amount of RAM available. It just uses heuristics to try to maintain a "reasonable" level based on program behavior. This is almost always fine for programs running on PCs with gigabytes of RAM, but leads to sub-optimal behavior for certain usage patterns, for example, code that generates lot of unique strings quickly. delete_files_list does this (listing the files also generates lots of unique strings, but already has explicit garbage collection). So IMO this is the most likely cause, and should have a straightforward solution.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #58 on: 13 / August / 2022, 17:58:33 »
will do that memory test in a few days.
but i don't think the low memory is caused by the script alone i think it is the combination with canon indexing when it moves to play.
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #59 on: 13 / August / 2022, 23:01:32 »
Quote from: Mlapse on 13 / August / 2022, 17:58:33
will do that memory test in a few days.
but i don't think the low memory is caused by the script alone i think it is the combination with canon indexing when it moves to play.
That's possible, but given that I had to add extra garbage collection logic to imglist, it is extremely like that delete can suffer the same problem. I do know a thing or two about this stuff  ;)

So in chdkptp r1182 I added batching and gc logic to multicam delete_images_list, similar to what imglist uses. There are two new options:
* batch_size controls the maximum files deleted per command. The default is 50. A garbage collection step is done once per batch. The defaults are similar to what imglist uses.
* rpretend is like pretend, but instead of just listing the files to delete on the PC side, it sends each file name to the camera side script which checks whether it exists.

Other changes:
* The output of verbose and pretend is a bit different
* Deleting should also be noticeably quicker.
* If the list is empty, a harmless warning is printed and nothing else is done.

If you don't want to deal with updating all of chdkptp or the multicam.lua file, you should be able to take just the mc:delete_files_list_cam function from the PC side and cmds.delete_files from the camera side.
Logged
Don't forget what the H stands for.
Pages: 1 2 3 4 5 [6] 7 8   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal