CHDK for longtime timelapse - page 2 - General Help and Assistance on using CHDK stable releases - CHDK Forum

CHDK for longtime timelapse

  • 73 Replies
  • 4633 Views
Re: CHDK for longtime timelapse
« Reply #10 on: 07 / April / 2021, 15:59:36 »
Advertisements

If the camera is in AV mode, what you have now should be mostly OK, but it could be off if the -av or -sv value is different from the Canon UI settings. The USER_AV in my example controls the manual AV setting, which is used if the camera is in M or AV. If you set this, it will be used for the Canon exposure calculation in AV mode. The regular AV override is the final value actually used in the shot. If you set it, the canon firmware isn't aware of it when it does the exposure calculation.

sv is similar, but it has more complication. If you want to use one of the normal ISO settings available in the Canon UI (like 80, 100 etc) using -isomode will have less chance of tripping you up.



Thanks for the -isomode tip. Now I understand the difference between sv, svm and isomode. ;)


Quote
Some information on scripting chdkptp is
https://app.assembla.com/spaces/chdkptp/wiki/CLI_Quickstart
https://app.assembla.com/spaces/chdkptp/wiki/Scripting_Guide

Thanks. I found set_lcd_display and other userful comands.  8)
I will write a myfile.chdkptp the next days. I do prefer configuration files.

Now the code is like this and it works quite well.
Code: [Select]
chdkptp -c -e"clock -sync" -e"luar set_lcd_display(0)" -e"rs $downloadCamera -av=4.0 -isomode=80 -shots=600 -int=15 -sdmode=MF -sd=1495mm"

I think I will run the camera for some time and see how it performs.
Have "discovered" the Canon G1X.... "discovered" it. I think it will be the successor when my current camera dies. CHDK seems to work well with the G1X and it seems to be very popular on the forum.... still not a ML camera with no moving mechanical parts, but maybe good enough.

If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw

Re: CHDK for longtime timelapse
« Reply #11 on: 19 / May / 2021, 13:29:08 »
Hello everyone,

I just wanted to give a little update incl. the code I use now to anyone who might be interested.

The Canon SX100IS is doing fine so far. I think, since I started the project it has now shoot more than 80'000 photos. The code I wrote for uploading the timelapse was far less reliable than the camera. The camera worked so far as a swiss clockwork. Some more hot pixels have emerged. I don't know how much of an issue this will become in the future but it's ok for now. The sun is not shining directly into the camera. I think this is only in the winter months the case. So it should not be an issue neither.

my last code I used for a while now was (e.g. for sunrise)

a file named config.chdkptp
Code: [Select]
clock -sync
luar set_lcd_display(0) --switch of display during taking photos
luar set_mf(1) --set focus to manual
luar set_focus(1495) --set focus distance
luar set_user_av96(320) --set AV value
luar -- AV modes: 3.2  3.5  4.0  4.5 5.0  5.6  6.3  7.1  8.0
luar --                  320 350 380 410 440 470 500 530 560
luar set_ev(-96) --set EV value
luar -- EV modes: -192  -160  -128  -96  -64  -32  0  32  64  96  128  160  192
luar --                 -2      -12/3 -11/3 -1   -2/3 -1/3 0  1/3 2/3 1   11/3 12/3 2
rs "D:/sunrise/input/" -av=3.2 -isomode=200 -shots=180 -int=15 -sdmode=MF -sd=1495mm
luar sleep(13000)
rs "D:/sunrise/input/" -av=3.2 -isomode=80 -shots=70 -int=15 -sdmode=MF -sd=1495mm
luar sleep(13000)
luar set_user_av96(380) --set AV value
rs "D:/sunrise/input/" -av=4.0 -isomode=80 -shots=350 -int=15 -sdmode=MF -sd=1495mm

and a powershell command:
Code: [Select]
chdkptp -c -e"source D:/sunrise/config.chdkptp"

as you may notice this code contains lot of try and error information and it is somehow redundant, which is bad but I was not able to understand which if the luar or the ptp code was more dominant but at the end I end up using both.
I had quite issues with avoiding autofocus before each shoot. so that's why I used in addition to the ptp code also
Code: [Select]
luar set_mf(1) --set focus to manual
luar set_focus(1495) --set focus distance

At the moment I use the rawopint v0.25 with rawopint_rs.lua from reyalp. Thank you very much for that!

I added support for using an external script to shoot with chdkptp remoteshoot. This turned out to be both simpler and better than I expected, so thanks for suggesting it  :D

the code I use now for the config.chdkptp is like this for sunrise:
Code: [Select]
clock -sync
luar set_lcd_display(0) --switch of display during taking photos
luar set_mf(1) --set focus to manual
luar set_focus(1495) --set focus distance
luar set_capture_mode(5) --set to manual mode
luar set_user_av96(320) --set AV value
luar set_tv96_direct(-256) --set TV value to 6.3s
luar set_iso_mode(200) --set ISO
luar set_raw_nr(0) --auto dark frame
rs "D:/sunrise/input/" -script=D:/DolomitiTimelapse/sourcecode/rawopint_rs.lua -shots=600 -int=15
luar sleep(2000)
d A/rawopint.csv D:/sunrise/

and like this for sunset:
Code: [Select]
clock -sync
luar set_lcd_display(0) --switch of display during taking photos
luar set_mf(1) --set focus to manual
luar set_focus(1495) --set focus distance
luar set_capture_mode(4) --set to AV mode
luar set_user_av96(320) --set AV value
luar set_ev(-96) --set EV value
luar set_iso_mode(80) --set ISO
luar set_raw_nr(0) --auto dark frame
rs "D:/sunset/input/" -script=D:/DolomitiTimelapse/sourcecode/rawopint_rs.lua -shots=600 -int=15
luar sleep(2000)
d A/rawopint.csv D:/sunset/



It works quite good now, but I'm still trying to reverse engineering the value ranges of some parameters as I haven't quite figured out the correlation of some parameters form the value in the log and the one is defined in the config file.  For example how the correlation between d_ev and ui_max_ev_change_e or ui_bv_ev_shift_pct and bv96. But time will tell.

I also tried raw shooting but I was not so successful as I needed. Exporting one dng file with RawTherapee would take on the old computer I'm using at leased 10 seconds and it completely crashed my computer several times. But I kind of like the idea of using RawTherapee for such tasks, thanks for the hint.

maybe in the future I will buy a Canon g1x as the jpgs out of the camera seems quite useful. The Canon sx100is is okay but yeah it's a bit compromising.
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw

*

Offline reyalp

  • ******
  • 13391
Re: CHDK for longtime timelapse
« Reply #12 on: 20 / May / 2021, 18:57:36 »
I just wanted to give a little update incl. the code I use now to anyone who might be interested.
Thanks, it's always nice to hear how these things turn out :)

Quote
as you may notice this code contains lot of try and error information and it is somehow redundant, which is bad but I was not able to understand which if the luar or the ptp code was more dominant but at the end I end up using both.
I had quite issues with avoiding autofocus before each shoot. so that's why I used in addition to the ptp code also
FWIW, -sd=1495 -sdmode=MF should really be equivalent to the luar commands, but bugs are always possible.

A couple other comments.

If you want things to run more quickly, you can combine multiple luar commands. This probably doesn't matter much to you, but executing each luar can take 1/10th of second or more, so may be relevant in other cases.

This can be done by putting everything on a single line like
Code: [Select]
luar set_lcd_display(0) set_mf(1) ...
or, if it's too ugly in one line, in a lua file like
Code: [Select]
set_lcd_display(0) --switch of display during taking photos
set_mf(1) --set focus to manual
set_focus(1495) --set focus distance
...
which you can execute in the .chdkptp file with
Code: [Select]
luar <myfile.lua
The '<' here is interpreted by chdkptp, so would need to be protected from the shell if you used it on the command line.
This file must only contain camera side Lua commands, not chdkptp CLI commands like 'clock' or 'rs'

For comments, you can use # at the start of a line in a .chdkptp file to avoid sending them to the camera.

Quote
At the moment I use the rawopint v0.25 with rawopint_rs.lua from reyalp. Thank you very much for that!
FWIW, depending on settings, rawopint may set some of these values (like screen off, dark frame, focus), but the default is usually to leave at alone, in which case setting it beforehand like you are doing should work.

If you want, you could put this lua code in the rawopint_rs.lua glue script, there's no requirement it only contain settings. This would make your code run in the same script invocation as the rawopint code, which is needed for some overrides (but not any of the ones you are using, AFAIK)

You may be interested in the bv_ev_shift options to handle sunrise and sunset.

Quote
It works quite good now, but I'm still trying to reverse engineering the value ranges of some parameters as I haven't quite figured out the correlation of some parameters form the value in the log and the one is defined in the config file.  For example how the correlation between d_ev and ui_max_ev_change_e or ui_bv_ev_shift_pct and bv96. But time will tell.
It's a bit complicated, the values in the menu are set up to make it less inconvenient to enter in within the limits of the CHDK script menu, and then they are converted to values the script can use.

Initial conversion happens in the script code just below the header. Some additional conversions are done later, but in the rawopint_rs.lua glue script you must set the values as if they came from the CHDK menu.

See https://chdk.fandom.com/wiki/CHDK_Script_Header#Lua_Shorthand_Parameter_Syntax for information about the script header / menu syntax.

To take ev change as an example, the CHDK script menu definitions is:
Code: [Select]
#ui_max_ev_change_e=3 "Max Ev change" {1/16 1/8 1/4 1/3 1/2 1}
This is an "enum" menu item, whose value is the 0 based index of the menu selection (1/16 = 0, 1/8 = 1 etc.) The values in the brackets are just display values in the CHDK UI, they don't end up in the ui_max_ev_change_e variable.

The following code:
Code: [Select]
ui_max_ev_change = ({96/16,96/8,96/4,96/3,96/2,96})[(ui_max_ev_change_e + 1)]
Converts it to an APEX*96 value, using the enum value as a lookup in a Lua array (1 based index), with the values expressed as fractions of 96 to make the conversion clear. So enum value 0 looks up entry 1 in the table which is 96/16 = 1/16th of t stop = 6 APEX*96 units.

All the variables prefixed with ui_ are either set from the CHDK menu (or, in your case, the rawopint_rs.lua glue script) or derived directly from them like the example above. I use the convention that _e is an enum as above, _t is table (where the .value member is the string value of the user selection), and everything else is a plain number or boolean.

Later in the script, these are passed to the various modules (mostly exp) with the init calls, often with additional conversion.

If you have a camera you can work with directly, you can see what values menu options produce by adjusting them in the script menu and then looking at the saved script configuration in CHDK/DATA/RAWOPINT.n (however, table values will only show the 0 based index of the table entry, not the string value)

Quote
I also tried raw shooting but I was not so successful as I needed. Exporting one dng file with RawTherapee would take on the old computer I'm using at leased 10 seconds and it completely crashed my computer several times. But I kind of like the idea of using RawTherapee for such tasks, thanks for the hint.
Yes, RawTherapee isn't lightweight.
Don't forget what the H stands for.

Re: CHDK for longtime timelapse
« Reply #13 on: 22 / May / 2021, 13:14:11 »

FWIW, -sd=1495 -sdmode=MF should really be equivalent to the luar commands, but bugs are always possible.

I'm 100% sure that the luar commands did worked and the "-sd=1495 -sdmode=MF" alone did not worked for the Canon XS100IS.
But I can't tell you if the "-sd=1495 -sdmode=MF" is necessary after the luar commands were added. (Because I'm still changing the code using anydesk as I'm not at home at the moment, this makes some test very difficult)
for the av setting I used both set_user_av96(320) and -av=3.2 otherwise the first picture would be different form the rest of the series.

Quote
For comments, you can use # at the start of a line in a .chdkptp file to avoid sending them to the camera.
Thank you! I get smarter every day :lol

Quote
If you want, you could put this lua code in the rawopint_rs.lua glue script, there's no requirement it only contain settings. This would make your code run in the same script invocation as the rawopint code, which is needed for some overrides (but not any of the ones you are using, AFAIK)
Yes that's a good point. Much cleaner code as you suggest!

Quote
You may be interested in the bv_ev_shift options
Yes I read the hole page, quite interesting. It's kind of fun to control a camera this way. I mean by loading code on the camera from a few 100km away.

Quote
See https://chdk.fandom.com/wiki/CHDK_Script_Header#Lua_Shorthand_Parameter_Syntax for information about the script header / menu syntax.

All the variables prefixed with ui_ are either set from the CHDK menu (or, in your case, the rawopint_rs.lua glue script) or derived directly from them like the example above. I use the convention that _e is an enum as above, _t is table (where the .value member is the string value of the user selection), and everything else is a plain number or boolean.

Thanks a lot. Now I understand much better how variables are declared, kind of confusing that table doesn't start at 0.

Code: [Select]
ui_max_ev_change = ({96/16,96/8,96/4,96/3,96/2,96})[(ui_max_ev_change_e + 1)]
I found this part in the code as I read some parts of the lua script to better understand how the values are used. So this means the max ev change would be "6" if I choose 0->96/16=6 but strangely even if with 1/8 or 1/4 the max ev changes on a normal sunset are not more than 6. But with 1/16 the sunset is kind of lagging behind. So these are my little investigations to " decipher" the code... also for the metering there is some more work to do to find out what values are most desirable.

I also got a change to test the Canon G1X this days. I'm already really excited about whether everything will work out with the camera.
« Last Edit: 22 / May / 2021, 13:26:17 by dolomiti_timelapse »
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw


*

Offline reyalp

  • ******
  • 13391
Re: CHDK for longtime timelapse
« Reply #14 on: 22 / May / 2021, 16:57:38 »
Yes I read the hole page, quite interesting. It's kind of fun to control a camera this way. I mean by loading code on the camera from a few 100km away.
Getting another cheap powershot you can test locally might save you a lot of time, even if it's not the same model.

Quote
So this means the max ev change would be "6" if I choose 0->96/16=6 but strangely even if with 1/8 or 1/4 the max ev changes on a normal sunset are not more than 6.
max_ev_change is the maximum it will ever use, which should definitely work beyond 6. The actual change depends depends on the metering. You need to examine the log to understand what it actually did. Plotting the "weight" and exposure related columns can be helpful. In case it's useful, I attached a simple jupyter notebook.

Quote
I also got a change to test the Canon G1X this days. I'm already really excited about whether everything will work out with the camera.
G1X has some quirks related to ISO, but I think c_joerg has used it quite a lot.
Don't forget what the H stands for.

Re: CHDK for longtime timelapse
« Reply #15 on: 23 / May / 2021, 05:44:04 »
NO WAAAYY!!!  😲 OMG!

I actually started writing a small script with matplotlib... The plots I saw here on the forum seemed to familiar to the matplotlib style... and I was wondering why everyone had the same plot style?! Now I know why!!
😄
I will definitely try out your jupyter notebook.
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw

*

Offline reyalp

  • ******
  • 13391
Re: CHDK for longtime timelapse
« Reply #16 on: 23 / May / 2021, 14:32:33 »
I actually started writing a small script with matplotlib... The plots I saw here on the forum seemed to familiar to the matplotlib style... and I was wondering why everyone had the same plot style?! Now I know why!!
FWIW, that one is a work in progress I started recently, so it's probably not the same, and it's quite primitive.

I previously used libreoffice calc  :-[, but that required an annoying amount of setup for each run.
Don't forget what the H stands for.

Re: CHDK for longtime timelapse
« Reply #17 on: 29 / May / 2021, 14:33:09 »
I moved the chat here since it's not related to the rawopint.lua anymore.

That's what I usually do. I try to expose the moon as brightly as possible but so that it isn’t overexposed. In post-production, I then pull the shadows up and the lights down. RAW is of course a must. The problem is the dynamic range of the old G1x. Then there is not so much in it. A reason for me to switch to the M100.

Interesting. The M-series is not an option for me because of the limited shutter life. Of course, an electronic shutter would be my favourite.

Today i did a first test with the G1X.

The Canon SX100IS was powered by a small and efficient self made power source which was connected directly to a USB 3.0 of the laptop. Which was really cool, because in an event of a power outage the battery of the laptop would supplying the camera and this would prevent the camera form switching off.
For the G1X I also build a power source today. But it works really bad.  :-[ The USB 3.0 max power output of the laptop is not enough.

FYI:
The Canon G1X consumes in Standby 1,05W,
when operation with screen on about 2,8W-3,3W
when rawopint script running 2,6W-3,1W
The maximum power the camera could consume was about 5W for a short time.

Attached a photo showing the setup...
« Last Edit: 29 / May / 2021, 15:01:41 by dolomiti_timelapse »
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw


*

Offline Ant

  • ****
  • 491
Re: CHDK for longtime timelapse
« Reply #18 on: 29 / May / 2021, 14:50:52 »
Interesting. The M-series is not an option for me because of the limited shutter life. Of course, an electronic shutter would be my favourite.

FYI: topic about electronic shutter

Re: CHDK for longtime timelapse
« Reply #19 on: 29 / May / 2021, 15:03:09 »

FYI: topic about electronic shutter

Thanks, I will have a look!
If you want to see a sunset or sunrise of Dolomiti Val Gardena shot with CHDK visit
Instagram: dolomiti_timelapse
YouTube: https://www.youtube.com/channel/UCEJHg--ujxLkjMrevJXh-Gw

 

Related Topics