CHDK for longtime timelapse - page 8 - General Help and Assistance on using CHDK stable releases - CHDK Forum
« previous next »
Pages: 1 ... 3 4 5 6 7 [8] 9 10 11 12 13 14   Go Down

CHDK for longtime timelapse

  • 133 Replies
  • 27161 Views
*

Offline dolomiti_timelapse

  • **
  • 96
Re: CHDK for longtime timelapse
« Reply #70 on: 07 / October / 2021, 08:23:43 »
Advertisements
Quote from: c_joerg on 03 / October / 2021, 08:41:22
Thank you for this information. A very detailed description of the error.

Quote from: dolomiti_timelapse on 03 / October / 2021, 07:28:38
I would be very happy if someone of you can read the ROMLOGs and find a cause :)

Unfortunately, that is beyond my capabilities ... ;)

Quote from: dolomiti_timelapse on 03 / October / 2021, 07:28:38
Do you have a script for recording temperatures and voltage during timelapse? I have not searched for such a script up to know to be honest.  :-[
During the time lapse, rawopint records everything. I once attached a script that records the values without shooting.

Hi!

thanks for your reply!

Yes I actually meant between the time lapse (when rawopint is not running) not during! My mistake! I will try out your script!
Logged
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw
*

Offline dolomiti_timelapse

  • **
  • 96
Re: CHDK for longtime timelapse
« Reply #71 on: 07 / October / 2021, 08:46:36 »
Quote from: reyalp on 03 / October / 2021, 17:19:44

Do you mean the files themselves are not removed (still visible with the chdkptp ls command, for example) or that they continue to appear in the Canon UI as "unknown file" or whatever? The latter is normal, CHDK only removes them from the file system. If the files still appear in ls, something is broken.


ok, yes! Your are right: the files on the SD card are deleted. only in the Canon UI they continue to appear.
Maybe the question is then: Why does it take so long to delete the evident non existing files during executing photo erase in the canon menu? If I do it a second time it is actually very fast. But in the first run it seems to delete a ton of pictures.

Quote
As c_joerg said, rawopint records this in rawopint.csv. The columns are vbatt, tsensor, topt and tbatt
I'm sorry, I meant between not during. :-[

Quote
In the camera log for these, I see a lot of lines with "SS:SetFace". If the Canon menu AF setting is set to something Face, AiAF, or Tracking, I'd suggest trying setting it to Center instead.
Strange, the AF was and is set to FlexiZone. Normally the camera is set to AF-mode after a restart. Sometimes if I remember I do change the AF to MF in the Canon UI manually which I think is better.

Quote
To make sure the cameras is "woken up" before setting focus or running rawopint, you could send a =click"shoot_half".

I'll look a bit more later into where in the firmware the crashes actually happen to see if that provides any clues.

None of these appear to be a result of code *directly* called by CHDK, they all come from tasks the CHDK doesn't touch. This doesn't mean CHDK isn't the cause (it almost certainly is) but it's not a simple matter of us calling a function with the wrong parameters or something like that.

=click"shoot_half" seems a good idea to start with. I will try that out.

Hmm.. in summary, the ROMLOGs unfortunately did not bring to light the cause of the error I was looking for.
In any case, many thanks for the great help!


Logged
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw
*

Offline Caefix

  • *****
  • 947
  • Sorry, busy deleting test shots...
Re: CHDK for longtime timelapse
« Reply #72 on: 07 / October / 2021, 14:01:43 »
Quote from: reyalp on 03 / October / 2021, 17:19:44
In the camera log for these, I see a lot of lines with "SS:SetFace".
Canon menu also offers a "Blink detection."
Maybe triggered by a  :P in the clouds or a lensflare constellation ?
« Last Edit: 07 / October / 2021, 14:11:51 by Caefix »
Logged
All lifetime is a loan from eternity.
*

Offline reyalp

  • ******
  • 14111
Re: CHDK for longtime timelapse
« Reply #73 on: 07 / October / 2021, 15:22:42 »
Quote from: dolomiti_timelapse on 07 / October / 2021, 08:46:36
Strange, the AF was and is set to FlexiZone. Normally the camera is set to AF-mode after a restart. Sometimes if I remember I do change the AF to MF in the Canon UI manually which I think is better.
I believe some parts of canon face tracking are active all the time, regardless of focus mode, so it's possible those messages are normal even in MF.
Quote
Maybe the question is then: Why does it take so long to delete the evident non existing files during executing photo erase in the canon menu? If I do it a second time it is actually very fast. But in the first run it seems to delete a ton of pictures.
Canon keeps other data structures related to the pictures (in memory and also in the metactg directory) and probably tries to access each of the non-existing files. Somewhat related discussion of the "indexing" behavior in https://chdk.setepontos.com/index.php?topic=13714.msg140106#msg140106 and https://chdk.setepontos.com/index.php?topic=11481.msg115202#msg115202

If your problem is related to imrm, it's possible the PT_EraseAllFile function discussed there could be an alternative.
Edit: Oops, not only did I forget about the original discussion of that function, I forgot that we discussed it in that thread quite recently  :-[

Quote
Hmm.. in summary, the ROMLOGs unfortunately did not bring to light the cause of the error I was looking for.
Yes, unfortunately this is often the case. If you can narrow down the crashes to a simple test case, that would obviously be helpful, but difficult to do remotely. Timestamped logs from the chdkptp side might help understand the circumstances of the romlogs.
« Last Edit: 07 / October / 2021, 15:44:30 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline dolomiti_timelapse

  • **
  • 96
Re: CHDK for longtime timelapse
« Reply #74 on: 23 / October / 2021, 11:22:30 »
Quote from: reyalp on 03 / October / 2021, 17:19:44
#1, #5 and #6 are all
Code: [Select]
ASSERT!! MotionVector.c Line 338
I think we've seen this one before, but search didn't turn much immediately useful. It is likely something sensor / live view related. I have a romlog from c_joerg with this one on g1x too, but I don't remember the circumstances.
It was previous seen in https://chdk.setepontos.com/index.php?topic=11078.msg108983#msg108983 related to focus override.

In the camera log for these, I see a lot of lines with "SS:SetFace". If the Canon menu AF setting is set to something Face, AiAF, or Tracking, I'd suggest trying setting it to Center instead.

#4
Code: [Select]
ASSERT!! LiveImageTask.c Line 523
Occured Time  2021:08:24 05:10:19
ISR : vector=10
Which I don't recall seeing before, but again seems sensor / live view related.

On thing that can cause sensor related crashes is if the sensor is turned off when CHDK calls functions that expect it to be on. This could be an issue if you have the screen off setting enabled in the canon firmware. To make sure the cameras is "woken up" before setting focus or running rawopint, you could send a =click"shoot_half".

Code: [Select]
ASSERT!! LiveImageTask.c Line 523
has not occurred anymore after sending click"shoot_half". This may have solved the problem.

Code: [Select]
ASSERT!! MotionVector.c Line 338
on the other hand keeps crashing the G1X. My workaround for now is to test if the camera is connected and if not switch the camera on with the servo-arduino "package".
Yesterday I had a new issue: only after I manually delete alle dummy files on the camera, the camera after a restart connected again to the PC, only a restart was not enough.
Before deleting manually the photos the camera printed a network connection error on display when powering on.

I found this https://chdk.setepontos.com/index.php?topic=12523.22 and it seems to be the same issue regarding MotionVector.c. @srsa_4c suggests: "Moving the hooks around seems to have been enough."
What does he mean with moving the hooks around? what are hooks and how do I move them around? :D

@davedude reported that the error occurred much less frequently when the camera sleeps for a longer period. Since the error does not happen during rawopint (I assume that rawopint was programmed correctly in this respect), is there a way to send the camera in sleep mode for some hours and to terminate the ptp connection shortly after the sleep command for continuing the photos conversion on the pc in the mean time?

Logged
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw
*

Offline reyalp

  • ******
  • 14111
Re: CHDK for longtime timelapse
« Reply #75 on: 23 / October / 2021, 19:25:01 »
Quote from: dolomiti_timelapse on 23 / October / 2021, 11:22:30
has not occurred anymore after sending click"shoot_half". This may have solved the problem.
Good to know.
Quote
Code: [Select]
ASSERT!! MotionVector.c Line 338
on the other hand keeps crashing the G1X. My workaround for now is to test if the camera is connected and if not switch the camera on with the servo-arduino "package".
It would be helpful to know exactly when this crash happens in the sequence of running scripts, shooting etc. If you can post the full batch file and modified rawopint that might also help, at this point I don't really have a clear idea what your using of all the stuff we've discussed in various threads.

Quote
Yesterday I had a new issue: only after I manually delete alle dummy files on the camera, the camera after a restart connected again to the PC, only a restart was not enough.
Before deleting manually the photos the camera printed a network connection error on display when powering on.
Did it say "CONNECTION ERROR" alone? Or did it actually mention "network"? Edit: Or "COMMUNICATION ERROR", which is the message I was thinking of.

"CONNECTION COMMUNICATION ERROR" is documented in the Canon manuals as a USB related error that happens when the camera thinks it has lots of images on the card. The manual typically says this happens when there are 1000 or more, but in my experience it usually a lot more (I just downloaded 2233 images from my elph130 without getting that error.)

If the camera crashed and the dummy files are still present, the camera might think there are enough files to trigger it though.

Have the full sequence of what your script does would help here too.
Quote
I found this https://chdk.setepontos.com/index.php?topic=12523.22 and it seems to be the same issue regarding MotionVector.c. @srsa_4c suggests: "Moving the hooks around seems to have been enough."
What does he mean with moving the hooks around? what are hooks and how do I move them around? :D
That refers to hooks in the ports capt_seq.c code. It would likely only be relevant if the crash happens in the shooting process or relatively soon after.
Quote
Since the error does not happen during rawopint (I assume that rawopint was programmed correctly in this respect),
I wouldn't ;) CHDK behavior is generally not well specified. We're poking the guts of a poorly understood, reverse engineered firmware with well over 100 distinct variants, so "correct" is often what has been found to work on some particular camera under some particular conditions. Some things can be unambiguously *incorrect* however :haha
Quote
is there a way to send the camera in sleep mode for some hours and to terminate the ptp connection shortly after the sleep command for continuing the photos conversion on the pc in the mean time?
I think the "sleep" davedude referred is the CHDK script "sleep" function, which just yields the CHDK scripting engine so that kbd_taskcan do the other stuff it does. No script running would be equivalent to continuously sleeping (at least, from a CPU hogging point of view)

If you're think of a camera power saving sleep mode, the only one I'm aware of is when the sensor and screen turn off in rec mode. This does not affect PTP / USB functionality.
« Last Edit: 25 / October / 2021, 19:05:40 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline dolomiti_timelapse

  • **
  • 96
Re: CHDK for longtime timelapse
« Reply #76 on: 26 / October / 2021, 09:11:25 »
It took me over one hour to write this post. Then I wanted to post it. Then I was logged out and the hole message was gone  :'( very frustrating!
So this is the second attend and no longer so detailed

Quote from: reyalp on 23 / October / 2021, 19:25:01
It would be helpful to know exactly when this crash happens in the sequence of running scripts, shooting etc. If you can post the full batch file and modified rawopint that might also help, at this point I don't really have a clear idea what your using of all the stuff we've discussed in various threads.
I was able to generate 8 ROMLOGS "ASSERT!! MotionVector.c Line 338" but only form 2 time lapse log files I have accurate information what happend.

1)
The log file says:
display off
taking photos
ERROR: I/O error
ERROR: error on line 38
attempted to close non-present device bus-0:\\.\libusb0-0001--0x04a9-0x3233

The chdkptp code before line 38 is like:
!print'taking photos'
rs "D:/sunset/input/" -script=D:/DolomitiTimelapse/sourcecode/chdk/rawopint_rs_dummy.lua -shots=720 -int=15 <<<--- THIS IS LINE 38
exec sys.sleep(1000)

no pictures have been taken that run.

2)
The log file says:
set focus to manual
set focus to infinity
ERROR: I/O error
ERROR: error on line 24
attempted to close non-present device bus-0:\\.\libusb0-0001--0x04a9-0x3233

The (older) chdkptp code around line 24 is like:
!print'set focus to infinity'
luar set_focus(-1) --set focus distance to infinite
luar sleep(1000)
luar set_capture_mode(5) --set to manual mode
luar set_user_av96(416) --set AV value  <<<--- THIS IS LINE 24
luar set_tv96_direct(-360) --set TV value to 6.3s
luar set_iso_mode(800) --set ISO 800
luar set_raw_nr(1) --dark frame: 0=Auto 1=OFF, 2=ON
!print'display off'

I added all the fiels to a zip file:

FIRMWARE: g1x-100e-1.6.0-5910-full
downloaded from here: https://forum.chdk-treff.de/download_dev.php
I use the DE version because this compilation keeps certain settings like "Enable Lua Native Calls" even after a reboot or normal power off and on. So I can create remote ROMLOGs for example.

RAWOPINT.LUA: rawopint-0.25
downloaded https://chdk.fandom.com/wiki/Lua/Scripts:_Raw_Meter_Intervalometer#Links
The changes I made are:
 
Code: [Select]
411  timeout = 30000 -> timeout = 1000 --display should get darker sooner.
660  'draw_gauge_y_pct', 'smooth'} do -> 'draw_gauge_y_pct', 'smooth', 'filter_intensity'} do --for moving average filter
684  self.meter_y_count = self.meter_height / self.meter_step -> self.meter_y_count = self.meter_height / self.meter_step * 5 / 10 -- crop of final time lapse video
932  -> local last_ev_change_decimal -- added for moving average filter
936  -> last_ev_change_decimal = self.ev_change_decimal -- added for moving average filter
944 -> last_ev_change_decimal = 0 -- added for moving average filter
1120-1141 replaced with ->     
log:set{
        d_ev_raw = ev_change
    }

    -- smooth out rapid changes exponential moving average implemented
    if self.smooth then
        local ev_change_smooth = (last_ev_change * 10 + last_ev_change_decimal) * self.filter_intensity +
                                     (10 - self.filter_intensity) * ev_change * 10
        ev_change = ev_change_smooth / 100
        last_ev_change_decimal = (ev_change_smooth % 100) / 10 -- not sure if modulo works this way in lua
    end
    self.ev_change = ev_change
    self.ev_change_decimal = last_ev_change_decimal

1697 'over_frac', 'over_weight', 'under_frac', 'under_weight', 'bv_ev_shift', 'd_ev', 'desc'}, -> 'over_frac', 'over_weight', 'under_frac', 'under_weight', 'bv_ev_shift', 'd_ev', 'd_ev_raw', 'desc'}, --log purpose
1809 -> filter_intensity = ui_filter_intensity, --changing filter intensity over powershell script
2003 ->         -- in cont release shoot full as soon as the final shot starts
        -- to avoid extra shots and delays
        if cont and i == ui_shots then
            release('shoot_full')
        end
2066 ->
        --create dummyfile for avoiding freezing when changing form play to rec after long time
        rlib_shoot_filedummy()

I used https://www.textcompare.org/ I think that is the better way than the list I provided above to see the differences.

RAWOPINT_RS_DUMMY.lua: see the custom setting in the file

CHDKPTP: in the zip file I added the sunset configuration
As you may notice I use the button commands to navigate through the menu to delete the dummy files, because "imrm -quiet" seems not to do the job.

Powershell: calling chdkptp via
Code: [Select]
chdkptp -c -e"source path/to/sunset_chkd_dummy.chdkptp"
Quote
Did it say "CONNECTION ERROR" alone? Or did it actually mention "network"? Edit: Or "COMMUNICATION ERROR", which is the message I was thinking of.

"CONNECTION COMMUNICATION ERROR" is documented in the Canon manuals as a USB related error that happens when the camera thinks it has lots of images on the card. The manual typically says this happens when there are 1000 or more, but in my experience it usually a lot more (I just downloaded 2233 images from my elph130 without getting that error.)

If the camera crashed and the dummy files are still present, the camera might think there are enough files to trigger it though.

Have the full sequence of what your script does would help here too.
Yes you are 100% right. It was COMMUNICATION ERROR. Your explanation seems to describe exactly the situation that night, where I first had to delete alle dummy files on camera before being able to connect the camera to the PC via USB.
« Last Edit: 26 / October / 2021, 09:16:25 by dolomiti_timelapse »
Logged
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw
*

Offline Caefix

  • *****
  • 947
  • Sorry, busy deleting test shots...
Re: CHDK for longtime timelapse
« Reply #77 on: 26 / October / 2021, 11:54:57 »
Quote from: dolomiti_timelapse on 26 / October / 2021, 09:11:25
It took me over one hour to write this post. Then I wanted to post it. Then I was logged out and the hole message was gone  :'( very frustrating!
Just click "Login" (Forever) to avoid  :P &  :o ... :D
« Last Edit: 26 / October / 2021, 12:30:31 by Caefix »
Logged
All lifetime is a loan from eternity.
*

Offline Caefix

  • *****
  • 947
  • Sorry, busy deleting test shots...
Re: CHDK for longtime timelapse
« Reply #78 on: 26 / October / 2021, 16:48:21 »
 ;) There seems a save way to let Canon delete the pics.
Could it open an option to avoid the dummy files, maybe with a "wheel_left() sleep(1000)" before the delete to keep the recent one?  :-[
Code: [Select]
--[[--
@title !Erase_displayed_file
@param s Time to read (sec)
@default s 3
@range s 1 10
@param b Key to enter deletemenu
@default b 0
@values b <erase> <down> <set> <up>
@param r Cam reacts
@default r 0
@values r fast medium slow
--]]--

function erase_displayed_file(M3)
  click(button)
    sleep(500)
  click("right")
    sleep(500+r*500) -- enough ??
  press("set")
    sleep(500)
  release("set")
    sleep(2000+r*500) -- on G1x enough to delete Raw & JPG or movie
-- "Busy" -- :)
end

if get_mode() or 0~=0 then -- :)
  click("shoot_half") -- 1 min to lens retract if set.
 -- click("playback")   -- not on M3 & g1x
 -- press("playback")   -- not on M3 & g1x
 --   sleep(1000)
 -- release("playback")
  set_record(0)
  while get_mode() do  sleep(100) end
end

print_screen(-8769)
print(get_exp_count(),get_jpg_count(),get_raw_count())
    sleep(s*1000)

s=s or 1

if b==0 then button="erase" end -- G1x
if b==1 then button="down" end -- M3
if b==2 then button="set" end  -- Sx610
if b==3 then button="up" end   --

erase=1
while erase do
-- click <menu> in 1st line /.. to exit
    erase=file_browser(get_image_dir()) --- press <left> to delete some files manually.
    if erase then erase_displayed_file() end
    print(erase,-get_free_disk_space()) --
    print(get_exp_count(),get_jpg_count(),get_raw_count()) --
    sleep(s*1000)
end

click("shoot_half")
    print(get_exp_count(),get_jpg_count(),get_raw_count()) --
    sleep(3000)
exit_alt(1)


Edit: if get_mode() or 0~=0 then --  :haha
« Last Edit: 27 / October / 2021, 11:57:22 by Caefix »
Logged
All lifetime is a loan from eternity.
*

Offline reyalp

  • ******
  • 14111
Re: CHDK for longtime timelapse
« Reply #79 on: 26 / October / 2021, 21:31:01 »
Quote from: dolomiti_timelapse on 26 / October / 2021, 09:11:25
It took me over one hour to write this post. Then I wanted to post it. Then I was logged out and the hole message was gone  :'( very frustrating!
Yes, it is, and it happens even with the "always stay logged in option" caefix mentioned. It's happened to me enough that I try to copy my post to clipboard before posting, but sometimes I still forget  >:(

Quote
I was able to generate 8 ROMLOGS "ASSERT!! MotionVector.c Line 338" but only form 2 time lapse log files I have accurate information what happend.
Do you have the specific romlogs that go with these two crashes? Not a big deal if not, but if you get more, it would be good to save romlog and chdkptp log together.

Quote
The chdkptp code before line 38 is like:
!print'taking photos'
rs "D:/sunset/input/" -script=D:/DolomitiTimelapse/sourcecode/chdk/rawopint_rs_dummy.lua -shots=720 -int=15 <<<--- THIS IS LINE 38
exec sys.sleep(1000)

no pictures have been taken that run.
So that one happened in rawopint, before the first shot, but we don't know where. Doesn't provide much to go on.

Quote
2)
The log file says:
set focus to manual
set focus to infinity
ERROR: I/O error
ERROR: error on line 24
attempted to close non-present device bus-0:\\.\libusb0-0001--0x04a9-0x3233

The (older) chdkptp code around line 24 is like:
!print'set focus to infinity'
luar set_focus(-1) --set focus distance to infinite
luar sleep(1000)
luar set_capture_mode(5) --set to manual mode
luar set_user_av96(416) --set AV value  <<<--- THIS IS LINE 24
For this one, it's possible that trying to set user av immediately after mode switch could cause problems.  I would probably put a few hundred ms sleep between them, but given the other crash, I don't have high expectations.

Quote
I used https://www.textcompare.org/ I think that is the better way than the list I provided above to see the differences.
Thanks. With the file you provided, I can use diff tools locally.

Quote
Yes you are 100% right. It was COMMUNICATION ERROR. Your explanation seems to describe exactly the situation that night, where I first had to delete alle dummy files on camera before being able to connect the camera to the PC via USB.
Yeah that seems to fit. Your runs are short enough that it probably only happens if there are undeleted files from several runs, so I think the main lesson is to make sure everything is deleted if there's a crash or error.

If it's a lot of trouble to have someone physically do this when PTP fails, you could set up a startup script to check for and delete any files. You'd probably need to reboot again to get PTP working.

@Caefix:
I don't understand how this script is supposed to help dolomiti_timelapse's problems, or indeed how it will work at all. It would be much more helpful to explain the idea than a complicated script.
Logged
Don't forget what the H stands for.
Pages: 1 ... 3 4 5 6 7 [8] 9 10 11 12 13 14   Go Up
« previous next »
 

Related Topics
SimplePortal 2.3.6 © 2008-2014, SimplePortal