how to reboot cams in multicam setup? - page 7 - LUA Scripting - CHDK Forum
« previous next »
Pages: 1 2 3 4 5 6 [7] 8   Go Down

how to reboot cams in multicam setup?

  • 70 Replies
  • 13568 Views
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #60 on: 15 / August / 2022, 17:17:21 »
Advertisements
i did run a new build of 1182, from a zip this time. and it worked in one go ;)
however, i did see 2 warnings, maybe it's nothing because it seems to work.
but decided to keep using my 1167 for now and added the 2 functions in my working copy of multicam as you suggested.
it is running for a while......i'll let you be the judge of this. and let it run until it stops.

i've entered 2 requests for memory, one a second after moving to play before delete, the other a second after deleting 120 files and one hour runtime.
the first time this happened was after 45 minutes runtime.
Cam #1 is 103a
Cam #2 is 102b (could be upgraded, since canon offers 102b firmware for download)

i thought they were the same, but looking at it i think cam #1 has a 312 bytes newer chdk version 1.6

as you can see in the csv, free memory is already less than it was...and other things more.
 

the warning while compiling the downloaded zip:

-I../extlibs/built/lua53/include -fPIC -c -o lsignal.o lsignal.c
lsignal.c: In function ‘l_raise’:
lsignal.c:265:7: warning: unused variable ‘args’ [-Wunused-variable]
  265 |   int args = lua_gettop(L);
      |       ^~~~
lsignal.c: In function ‘l_signal’:
lsignal.c:219:5: warning: ‘sig’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  219 |     lua_pushnumber(L, sig);

« Last Edit: 15 / August / 2022, 17:35:10 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #61 on: 15 / August / 2022, 18:21:07 »
Quote from: Mlapse on 15 / August / 2022, 17:17:21
i did run a new build of 1182, from a zip this time. and it worked in one go ;)
however, i did see 2 warnings, maybe it's nothing because it seems to work.
Those two warnings are normal and harmless, and in any case, signal is an optional module that isn't used unless you specifically add code to use it.
Quote
but decided to keep using my 1167 for now and added the 2 functions in my working copy of multicam as you suggested.
it is running for a while......i'll let you be the judge of this. and let it run until it stops.

i've entered 2 requests for memory, one a second after moving to play before delete, the other a second after deleting 120 files and one hour runtime.
the first time this happened was after 45 minutes runtime.
Cam #1 is 103a
Cam #2 is 102b (could be upgraded, since canon offers 102b firmware for download)

i thought they were the same, but looking at it i think cam #1 has a 312 bytes newer chdk version 1.6

as you can see in the csv, free memory is already less than it was...and other things more.
Those free numbers show that this isn't a severely memory-limited port at least. Images on the card causing some memory use is expected. If you want, you can also track the amount of memory used by Lua, using
Code: [Select]
mc:cmdwait('call return collectgarbage("count")')
Note this value is in KB, not bytes.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #62 on: 16 / August / 2022, 05:14:40 »
my only reference is that the cam running a script makes around 60.000 shots in 3 weeks before it goes to play and reboots. never seen a memory error. but that does nothing of what multicam does with listing images, downloading or deleting them.
since prior to delete there is already a lot less memory available since after last delete. there might be something else that keeps memory occupied.

now your new code made the best run yet. with some 16 hours of runtime. where most of the hours free memory lost about 40000. but some hours had double losses.
however, it wouldn't make reboot, although i would have expected it to make another round or 2 before the memory error.
both cams are still active in play after the error, not one has shut down. so that's an improvement.
and no romlog needed since it printed the free memory error in the terminal.


Quote
1: {{free_size=109608,total_size=2686544,chdk_size=144408,chdk_start=2669704,name="combined",allocated_peak=2617896,free_block_count=1036,allocated_count=14644,free_block_max_size=73216,chdk_malloc=true,allocated_size=2576936}}
2: {{free_size=122208,total_size=2686864,chdk_size=144088,chdk_start=2669704,name="combined",allocated_peak=2606864,free_block_count=1062,allocated_count=14643,free_block_max_size=87520,chdk_malloc=true,allocated_size=2564656}}
sleep done 1660644008
ERROR: call failed:unexpected error: :297: not enough memory

ERROR: error on line 7

since it was after moving to play, sleep, printing memory and sleep done: the next lines of code would have been:
Code: [Select]
files=self:find_files('A/DCIM',{ff_func='find_files_all_fn'})
self:delete_files_list(files)
« Last Edit: 16 / August / 2022, 14:30:08 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #63 on: 16 / August / 2022, 17:58:57 »
Quote from: Mlapse on 16 / August / 2022, 05:14:40
both cams are still active in play after the error, not one has shut down. so that's an improvement.
and no romlog needed since it printed the free memory error in the terminal.
When you run low on memory, it's basically random what specific call goes over the limit. If it's in Lua, you get a nice out of memory error and the script ends. If it happens in a firmware function, you mostly likely get an assert and the camera shuts down. If it happens elsewhere in CHDK, you may get a crash or other unexpected behavior.

Quote
1: {{free_size=109608,total_size=2686544,chdk_size=144408,chdk_start=2669704,name="combined",allocated_peak=2617896,free_block_count=1036,allocated_count=14644,free_block_max_size=73216,chdk_malloc=true,allocated_size=2576936}}
2: {{free_size=122208,total_size=2686864,chdk_size=144088,chdk_start=2669704,name="combined",allocated_peak=2606864,free_block_count=1062,allocated_count=14643,free_block_max_size=87520,chdk_malloc=true,allocated_size=2564656}}
sleep done 1660644008
ERROR: call failed:unexpected error: :297: not enough memory

ERROR: error on line 7
Those show very low free memory, so the error is expected. It would be helpful to understand what fraction of the memory use is due to the camera having lots of images, what fraction is used by Lua, and what fraction of the Lua portion is due to uncollected garbage. To check this, I'd suggest
1) at the same point you do get_meminfo, record the result of collectgarbage("count") as suggested earlier
2) After that, call collectgarbage("collect")
3) call get_meminfo and collectgarbage("count") again

One other thing we can do is make some more memory available. Attached are s110 builds with ARAM enabled, which should add another ~130KB

Note it would still be good to get the information described above
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #64 on: 17 / August / 2022, 14:13:43 »
i chose a bit more aggressive garbage collect and it's running now and it may just make a reboot this time, but that's not sure...by the looks of it it is going the be very close.
the funny thing is, first you had an increase in memory after the step garbage collect, now it ends about 10k lower per delete round.

i did have a fault at reboot that i do not understand, but the script went on. it had already connected, went to record and then a device apparently disappeared for a moment:
although the cmdwait, maybe a small sleep between rec and init_sync ?

Quote
rec
1: true
2: true
1: {
 cmd="tick",
 status=13150
}
attempted to close non-present device 001:008
attempted to close non-present device 001:009
1: ticks=10 min=-16 max=1 mean=-4.005181 sd=4.785645
1: sends=10 min=1 max=12 mean=3.207099 sd=3.233062
2: {
 


looking at the available data i've got a theory about the mechanics.
maybe i'm wrong but the available facts fit the results.

we have this file number limit that is a bit of a vague figure.
as far as i can tell it is related to the amount of free memory.

f.i. on the s110 it's 3011 files.

when the cam is connected via usb it will keep a list for ptp usage in memory.
this list is what causes the lack of memory.

when you run a standalone it does not make the ptp list, hence there is no memory problem.
because i'm making 2880 images before reboot and these cams only handle 3011 files before the ptp is refused because of low memory it is all getting very tight memory wise.
« Last Edit: 17 / August / 2022, 18:54:57 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #65 on: 17 / August / 2022, 22:34:02 »
Quote from: Mlapse on 17 / August / 2022, 14:13:43
i chose a bit more aggressive garbage collect and it's running now and it may just make a reboot this time, but that's not sure...by the looks of it it is going the be very close.
I don't understand. What does "a bit more aggressive garbage collect" mean?
Quote
the funny thing is, first you had an increase in memory after the step garbage collect, now it ends about 10k lower per delete round.
I don't understand what this means either.


Quote
i did have a fault at reboot that i do not understand, but the script went on. it had already connected, went to record and then a device apparently disappeared for a moment:
although the cmdwait, maybe a small sleep between rec and init_sync ?
Are you referring to the "non-present device" message?
This likely happens because your script (at least in the version I have) does not disconnect before rebooting. It does not necessarily indicate a problem, though I would generally suggest disconnecting cleanly, with a delay on the camera side to ensure the disconnect can happen before the reboot.

Quote
we have this file number limit that is a bit of a vague figure.
as far as i can tell it is related to the amount of free memory.
It may be that Canon chose the limit due to memory constraints, but the actual limit is a specific number hard-coded in the Canon code. Previously discussed in https://chdk.setepontos.com/index.php?topic=14446.msg147215#msg147215

The number of *images* you get may vary slightly because things like folders and FI2 files count against the total.

Quote
when you run a standalone it does not make the ptp list, hence there is no memory problem.
because i'm making 2880 images before reboot and these cams only handle 3011 files before the ptp is refused because of low memory it is all getting very tight memory wise.
It is possible that the PTP object list consumes significant memory, but some per-image overhead without PTP has also been observed. Observed numbers are better than speculation ;)
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #66 on: 18 / August / 2022, 03:13:08 »
only 4 hours to go, too little memory left. one cam dead again.
still, the best run with 20 hours on the clock since (re)boot.


assuming my analysis is correct. i mentioned this idea some posts after the link you posted about file limit. it was about possible ways to break the file limit and it might help reduce this memory problem:
a solution to this could be if we were able to redirect images to be saved outside of dcim.
that way they are not indexed and thus memory stays available.
or to keep all chdk pathways intact, redirect where canon looks for images, so it indexes an empty folder.

i wouldn't know if one of these ideas are possible at all and what other complications would arise from that.

and that brings me to the other option that might work: going to 2 reboots a day.
this wouldn't be the best idea from a stability or durability point of view with twice as much voltage drops and current spikes, connects, mechanical wear and dust, but there would still be enough memory after 12 hours.
unless there is a different option that i can do, this will be my next try.

that's all i can come up with to possibly solve this.
 

the log contains first 4 deletes and a reboot plus following hours, then a few from 4-5 hours later and the last 2 delete entries.
all missing ones show a steady decline in free memory so i felt no need to include them.
« Last Edit: 18 / August / 2022, 16:08:47 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #67 on: 19 / August / 2022, 16:23:51 »
it's not that i made 24 hours yet, only 12, but it was time for a reboot. now there is still enough memory left and reboot works flawless.....

eh, that is, this time after going to record i did not get the non present device, but false msg timeout.
i'll have to see about that disconnect again.

nonetheless the script continued so i'll let it go for a while.
these meminfo numbers are just before delete/reboot, but i forgot to set the count around the reboot, so i don't have that, but it was probably the same as the others with some 130 before and 190 after delete.
and free memory was fairly close to yesterdays run after 12 hours, although that was a delete only.

it is sort of a solution  :( and it at least gives me the possibility to work on the script and see if i can get it useful for my specific goal.
but before i started, i was already thinking that one reboot per day was much for a long term project. so where does that leave me with 2?


Quote
call return get_meminfo()
1: {{chdk_start=2669704,allocated_count=9793,chdk_malloc=true,allocated_peak=2423352,allocated_size=2282896,free_block_count=1145,free_size=403648,free_block_max_size=284056,total_size=2686544,chdk_size=144408,name="combined"}}
2: {{chdk_start=2669704,allocated_count=9792,chdk_malloc=true,allocated_peak=2416304,allocated_size=2270136,free_block_count=1259,free_size=416728,free_block_max_size=295440,total_size=2686864,chdk_size=144088,name="combined"}}
« Last Edit: 19 / August / 2022, 16:49:36 by Mlapse »
Logged
frustration is a key ingredient in progress
*

Offline reyalp

  • ******
  • 14125
Re: how to reboot cams in multicam setup?
« Reply #68 on: 20 / August / 2022, 03:20:24 »
Quote from: Mlapse on 18 / August / 2022, 03:13:08
only 4 hours to go, too little memory left. one cam dead again.
It would be helpful to see the code you used. Doesn't need to be "cleaned up", I just want to see the sequence of calls involved.

Quote
assuming my analysis is correct. i mentioned this idea some posts after the link you posted about file limit. it was about possible ways to break the file limit and it might help reduce this memory problem:
a solution to this could be if we were able to redirect images to be saved outside of dcim.
that way they are not indexed and thus memory stays available.
I don't think this would help, unfortunately. It would be functionally equivalent to remoteshoot or deleting the images before switching to playback, which crashes many cameras (AFAIK including S110)

Quote
or to keep all chdk pathways intact, redirect where canon looks for images, so it indexes an empty folder.
The problem is some element of the per-image information is created at shooting time, not on indexing. It might be possible to work around this (for example by hacking MetaCTG task https://chdk.setepontos.com/index.php?topic=11481.msg115202#msg115202) but there's no ready-made solution.

Quote
and that brings me to the other option that might work: going to 2 reboots a day.
this wouldn't be the best idea from a stability or durability point of view with twice as much voltage drops and current spikes, connects, mechanical wear and dust, but there would still be enough memory after 12 hours.
unless there is a different option that i can do, this will be my next try.
Another possible option would be to enable exmem or steal UI memory, which might perhaps leave enough to run a day instead of two. But I'm not sure rebooting twice a day is so bad, I don't know that we have any real info on what wears a camera out.

Quote
these meminfo numbers are just before delete/reboot, but i forgot to set the count around the reboot, so i don't have that, but it was probably the same as the others with some 130 before and 190 after delete.
Going up after delete seems odd, that's one of the reasons I'd like to see what code you are using.
Logged
Don't forget what the H stands for.
*

Offline Mlapse

  • *****
  • 584
  • S95 S110
Re: how to reboot cams in multicam setup?
« Reply #69 on: 20 / August / 2022, 04:39:07 »
Quote from: reyalp on 20 / August / 2022, 03:20:24

It would be helpful to see the code you used. Doesn't need to be "cleaned up", I just want to see the sequence of calls involved.

be my guest.
Quote
I don't think this would help, unfortunately. It would be functionally equivalent to remoteshoot or deleting the images before switching to playback, which crashes many cameras (AFAIK including S110)
ok. bad idea. won't give it any more thought then.
Quote
The problem is some element of the per-image information is created at shooting time, not on indexing. It might be possible to work around this (for example by hacking MetaCTG task https://chdk.setepontos.com/index.php?topic=11481.msg115202#msg115202) but there's no ready-made solution.

Another possible option would be to enable exmem or steal UI memory, which might perhaps leave enough to run a day instead of two. But I'm not sure rebooting twice a day is so bad, I don't know that we have any real info on what wears a camera out.
i wouldn't know or understand what these possible solutions would involve.
but i can tell you from years of experience that whatever info is created by canon during shoot or play is not relevant for the low free memory results i'm experiencing now.
i might not have memory data of these events. but 60.000 images before reboot is normal in my current setup. not because of low memory but to reset the tick count.
while with a 5v powered usb cable i can't even make 3000 shots without a low memory count...as i pointed out in that other post about file numbers.
it does not have to be connected to a pc to create these 'connection' problems.
sadly, it is with intent designed this way.
must have been hard on the engineers to go out of their way not to follow industry standards and design something, well, less.

first thought here is, can i cut the 5v line but keep the data lines and still connect to the camera? and would that perhaps prevent ptp-indexing from canon becoming active? probably not because the usb client doesn't get power.
however, it does not does the usb client handshake unless ptp-indexing is within specs.
it all comes back to a way of cheating ptp indexing i guess.

the wear i encounter are based on experiences with cams that only move to play/shut down/(re)boots twice a month.
this does not include any connection trouble or pc interference.

aside from human interference by me ;) it mostly involves stuck shutters, misalignments and, over time, dust. i'd say influenced but not completely related to the lens motor or electronics.
more related to production choices.
and then there are stopped cam moments due to power surges at reboot.
in reality there are always a few days missing or unusable each year...that could be less if i would detect it sooner. this is where a pc connection could be helpfull.

but that will change and not for the better when you make the cam do those 'riskfull' actions 15 or, with 2 reboots a day, even 30 times as much.
all those lens actions would probably keep the lens casing more dust free. but that's not really positive either.
and somewhat lower level you have 2x360 more relais switches between play and record needed to 'pre-delete' the files, those come with their own power surges and spikes.

if chances of failure are now only 1:500.000 (twice a year) i couldn't guess what it would become with these extra moments of risk without trying.
f.i. this year i already had 2 moments where reboot didn't complete and one where focus was so far off that the results were completely blurred for 6 hours when it suddenly decided to go there were it was supposed to be. (MF saved in custom setting, not by chdk)

Quote
Going up after delete seems odd, that's one of the reasons I'd like to see what code you are using.
i've enclosed some data from the last 2x12 hours run.
« Last Edit: 20 / August / 2022, 15:32:57 by Mlapse »
Logged
frustration is a key ingredient in progress
Pages: 1 2 3 4 5 6 [7] 8   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal