Script Newbie - page 2 - General Help and Assistance on using CHDK stable releases - CHDK Forum
« previous next »
Pages: 1 [2] 3   Go Down

Script Newbie

  • 21 Replies
  • 5223 Views
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie
« Reply #10 on: 24 / November / 2017, 05:41:46 »
Advertisements
Quote
Day 1 NOV 24 00:20:00:2017 V: 2.733 T: 32 <no shot>
I decided to rebuild the card with 'stick' to start again and didn't get the 'card full' error. I found the 'debug' mode which seems to put ultimate into a repetitive loop and lots of shots were saved to the card. I then set it up to run all night expecting a card full of shots this morning. Instead I get 3 shots saved and the log showing no shot each time there should have been?

I think for CCTV alarm triggered still shots I need to learn some lua script writing because Ultimate has different concepts. This is my wish list for what I'm looking for:

Camera remotely triggered by external alarm or PIR sensor using CHDK remote usb.
Motion sensing is useful, but often gives false triggering in CCTV applications.
Camera shoots highest resolution medium to high compressed jpegs.not TV mode.
32Gb card can take thousands of shots without a shot limit and overwrites old shots at %(?) remaining.
Focus option to set infinity and fixed zoom otherwise set on camera.
Day night exposure - save color in good light and B/W auto ISO when dark or with IR illumination.
Led on camera flashing when script is running o.k
Script saves a shot list with day/date/ time and filename to quickly locate alarm events.
External power supply, therefore battery voltage monitoring not needed.
Daylight saving to keep shot times in sync with local time.
Option to add date to camera image (may be camera model dependent).

The bare minimum would be set up the camera manually, have it fired externally from usb and use a script to manage file number limit and card capacity. There are many things I don't know yet. Will CHDK external trigger still work when a script is running, does a running script take priority over normal CHDK functions and every button and camera setting has to be defined in the script?

I will spend a little more time with Ultimate, but I think it is too heavyweight to unpick and give me something more suited to CCTV. I understand the A560 is not the best of cameras but at 7Mpix, with good high compression options and manageable file sizes its still images are far superior to most HD and cheap 4K CCTV cameras which are often sold with poor lenses. I am using the A560 because there are plenty of cheap used about and it takes a 32Gb card (albeit FAT32). It is physically large enough to disassemble mod.and add wiring (to buttons) if I have to. I have experimented with CCTV for a while and concluded one clear sharp still image at the right time is better than 10- 30fps moving images with no detail. A simple still camera backup may be all you have when a hard disk saving 4 H264 video streams fails!
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: Script Newbie
« Reply #11 on: 24 / November / 2017, 09:27:35 »
Quote from: voxmagna on 24 / November / 2017, 05:41:46
I found the 'debug' mode which seems to put ultimate into a repetitive loop and lots of shots were saved to the card.
Enabling debug mode runs the script with a simulated real time clock function that changes much faster than a normal clock.  It's for testing your configuration obviously.

Quote
I then set it up to run all night expecting a card full of shots this morning. Instead I get 3 shots saved and the log showing no shot each time there should have been?
The intervalometer can be setup to shoot or not shoot based on time of day and/or ambient light level. The log entries that say "No Shot" indicate that one of those conditions was not satisfied.  If you used the script default settings, it only shoots between 9:00 AM and 5:00 PM.  So your "overnight" run would not actually result in any photos being taken.

Quote
I think for CCTV alarm triggered still shots I need to learn some lua script writing because Ultimate has different concepts.
I'd agree with this.

Quote
Will CHDK external trigger still work when a script is running,
There is script functions that lets it work smoothly with CHDK's USB remote functionality.

Quote
does a running script take priority over normal CHDK functions and every button and camera setting has to be defined in the script?
Yes & maybe.   Your script needs to manage any button or camera function that it actually wants to use. Other buttons are ignored while the script is running and camera setting not changed by the script remain as they were when the script started.

Quote
I will spend a little more time with Ultimate, but I think it is too heavyweight to unpick and give me something more suited to CCTV.
The script uses a lot of clearly named functions. Just borrow the pieces you need? 

Here's something that might be handy in your journey : CHDK_Scripting_Cross_Reference_Page
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie
« Reply #12 on: 24 / November / 2017, 10:36:31 »
Thanks for explaining the 'no shot'.  Ultimate was running in default settings, but at least it's working now. I did try the KAP intervalometer script. Again it did far more than I need, but it can bypass the timer to accept remote USB trigger and fire single shots. Unfortunately, it doesn't handle the Canon maximum file number limit, won't overwrite old files and stops at 'Card Full', something I think all intervalometer scripts should include to avoid the 'time bomb'.

I'll experiment borrowing sections of script and concentrate on file storage issues first, because that will kill my project if the camera stops shooting if I am on holiday. I am now using my 4Gb card with its partion reduced to 250mb to test how a script  responds to 'Card Full' and I've found some DOS scripts to produce vast numbers of sequentially named files for the camera and script to chew on.

One issue to cope with is stick formats all cards 4Gb and below as single partition FAT 16 and cards >4Gb as 2 partitions FAT16 for boot and FAT32 for CHDK and shots. I am not sure yet how a script will manage deleting old files if the camera firmware (or CHDK?) is  automatically generating new folders with each containing <2000 files if I understand correctly.

Thanks for your help,  I'll go quiet now to learn and try some simple scripts, but I can see me wearing out the card slot.
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: Script Newbie
« Reply #13 on: 24 / November / 2017, 12:42:11 »
Quote from: voxmagna on 24 / November / 2017, 10:36:31
Again it did far more than I need, but it can bypass the timer to accept remote USB trigger and fire single shots. Unfortunately, it doesn't handle the Canon maximum file number limit, won't overwrite old files and stops at 'Card Full', something I think all intervalometer scripts should include to avoid the 'time bomb'.
I'm thinking that I should add USB trigger mode as a standard script option. If I do that then the only thing missing from your wish list is color mode in daytime, b&w at night. I won't add that to the script but I will show you what needs to be done.  It's almost trivial 

Quote
One issue to cope with is stick formats all cards 4Gb and below as single partition FAT 16 and cards >4Gb as 2 partitions FAT16 for boot and FAT32 for CHDK and shots. I am not sure yet how a script will manage deleting old files if the camera firmware (or CHDK?) is  automatically generating new folders with each containing <2000 files if I understand correctly.
AFAIK, that should all work. The folders will all end up on the large partition and deleting the oldest files from there will work too.

Quote
but I can see me wearing out the card slot.
That's only an issue if you do things the hard way. There is an emulator plug-in for Notepad++ that will let you simulate everything in your PC and test the script.  And if you use chdkptp you can download scripts to your camera over a USB connection.  No need to ever remove your card again.  I'll post links when I get back to my PC later today.

Links :

Execute Lua scripts with hostlua & notepad++

CHDKPTP
« Last Edit: 24 / November / 2017, 18:25:45 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie
« Reply #14 on: 25 / November / 2017, 12:13:14 »
Thanks, I need time now for some more tests. In the meantime I found out one of my used A560s wouldn't work on external power. Took it apart and found a fast smd fuse blown. It went back together and now it's fixed, which doesn't always happen as there are many small parts to break. The previous owner probably put 12 volts on it! There are 3 fuses on the main board. 1 for external dc, 1 for battery power and 1 for something else. That's what I like about this model, it's big enough to disassemble with a magnifier and my ageing eyesight and it appears to have a larger program memory than the A570. Gotcha! the battery door operates a mini-mini switch and during testing whilst disassembled nothing works unless you fool the switch!
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: Script Newbie
« Reply #15 on: 26 / November / 2017, 19:14:09 »
ultimate.lua updated to handle remote trigger via USB remote.

Details here : 
Re: Ultimate Intervalometer - a script for shooting over a long duration - v4.7
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie
« Reply #16 on: 29 / November / 2017, 16:39:53 »
Thanks, It took me a while to figure out what libusb32 was doing and why it wouldn't install on my Win7_64. A big hole for Newbies to fall into! The Canon usb driver in its firmware only allows access to the DCIM photos folder, since it's purpose is to transfer photos between card and pc. However, to access the root folders and CHDK containing the scripts to modify you need a non-OE USB driver signed and using the same hardware PIDs that Windoze will accept as valid.

Unfortunately, Windows is locked down for that kind of change to its OS  and you can only install the USBlib driver as a super administrator, even though with user admin rights the file dropdowns says 'run as administrator'! Then there's the issue of Win 32/64 driver compatibility and what is hard to find is an explanation that the latest versions of libusb-win32 are actually Win32 AND Win64 compatible when installed with super admin rights. Now I can use 'Zadig' to do the driver install. Uninstalling it just requires the libusb driver to be deleted in Windoze and it will reaquire the OE Canon driver on next usb connection. If you are a Newbie wanting to avoid pulling your SD card in and out to try script mods you have to get the non OE usb driver working.

In my wish list I had day and night camera switching but I forgot that in CCTV cameras this is normally done with a separate photo sensor in the camera body. CCTV cameras will switch to high iso mode at night and operate in black and white where all the camera pixels can be used as monochrome to reduce noise. If the camera has IR sensitivity, night scenes can be illuminated with IR light running continuously or turned on by a PIR intruder alarm. The preferred IR wavelength is around 950 nm making the light invisible. Although I am still floundering with code, I was curious about the spylantern script, because there's a clever section of code in there that requires a local latitude (which defaulted to the authors location!).  I can see now that it is cunningly working out sunrise and sunset times, whilst factoring local daylight saving corection to keep the sunset/sunrise times in sync with local time.

I am away for a few days now but have downloaded your script with external trigger option and will feedback once I've run it. Many many thanks, I hope the modification will be useful for others wanting to do CCTV.
Logged
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie
« Reply #17 on: 10 / December / 2017, 07:42:32 »
Hi, can somebody please help me understand and confirm the 'hierarchy' and priority of camera setting options and settings when using CHDK and scripts?

 1. There are the normal Canon firmware options which can be accessed when the card write protect tab is off and the camera doesn't boot to CHDK. When the camera then auto boots to CHDK does it use the Canon settings or replace/overide them in its own menus?
 2. Mode options can be set using the Canon function wheel - AUTO, MAN, etc. Can I assume CHDK doesn't interfere with these pre-sets?
 3. Many new options can be set with CHDK, but do the Canon settings have priority if set before CHDK loads?
 4. If a script is loaded and running, does the script replace CHDK settings? i.e  is the CHDK base firmware still running and a script interacts with it or does a script replace CHDK. When I'm switching back and forth between a script menu and the CHDK menu, I'm not sure what is running or what has priority?

I'm trying to get my head around these three ways of changing camera 'variables': Canon firmware settings, CHDK, and Scripts and how each interacts. At a very basic level I need to know if scripts just change what CHDK does as default by reading the status of settings then changing them, or scripts have to initialise settings from scratch?

Incidentally, I've finally found out why these Canon Powershot 'A' series cameras give so much trouble with low battery warnings after testing my A560 on a bench power supply: The Canon firmware does more than just measure battery voltage. At power on it measures battery internal resistance by measuring the peak current against some reference value. The A560 takes about 350mA with the lcd lit and 80mA unlit so it must be calculating a peak value which will support flash recharging. I think the flaw is in the Canon firmware, because many low battery situations will support loads more shots. It's a shame the Canon algorithm appears to be responsible for dumping loads of re-chargeable batteries. Even a 2.5Amp bench supply can max out their internal resistance algorithm and put up occasional low battery warnings. The warnings remain persistent until the camera is re powered. I can stop this behaviour by adding a large capacitor to the supply wiring. I now test all my niMh batteries for internal resistance and the variation between brands is surprising. I also re-test after a few charge discharge cycles because some do not stay good. My battery of choice for consistent performance is the non-rechargeable Lithium AA. When I find Canons current sense resistor on my camera I will change its value. When people try to fix this problem by cleaning battery terminals, they are just trying to lower the battery internal resistance which the camera measures at power on. I think the real problem is with Canons measurement algorithm? The low battery voltage threshold set by Canon was for 1.5V cells like high capacity Duracells. There is a greater voltage margin for these cells than NiMh which start at 1.35V fully charged.
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: Script Newbie
« Reply #18 on: 10 / December / 2017, 11:01:31 »
Quote from: voxmagna on 10 / December / 2017, 07:42:32
Hi, can somebody please help me understand and confirm the 'hierarchy' and priority of camera setting options and settings when using CHDK and scripts?
When CHDK is loaded, it intercepts the launch of some of the "tasks" from the base Canon firmware and replaces the tasks with its own version.  The new tasks perform the same functions as the old tasks but are modified / augmented to include new CHDK functionality. 

One of the things added to the existing tasks is the script interpreter you use. When you ask CHDK to run your script, the other Canon task and the ones with CHDK things added still function normally. They run in parallel - a process known as multitasking.

From a hierarchy point of view, priority is essentially a question of which task made the most recent change. It's slightly more complicated than that of course. With things like exposure setting changes, CHDK grabs priority by making sure it makes the last change before each shot.  The Canon firmware does not even know it happens - it assumes its setting were used.  Also, when a CHDK script is running, CHDK intercepts all camera button presses so that the Canon firmware does not see them, thereby allowing the script to interact with them.

Quote
1. There are the normal Canon firmware options which can be accessed when the card write protect tab is off and the camera doesn't boot to CHDK. When the camera then auto boots to CHDK does it use the Canon settings or replace/overide them in its own menus?
Mostly the Canon menu settings and CHDK menu settings are independent.  So both are used as they don't overlap. CHDK does override a few of the Canon settings though - "Disable LCD Off" for example. The big exception is exposure settings - as I've hopefully explained above.

Quote
2. Mode options can be set using the Canon function wheel - AUTO, MAN, etc. Can I assume CHDK doesn't interfere with these pre-sets?
Normally it does not.  However, a script can change the active mode for example, or other Canon settings (via setting Property Case  or Param variables, or by calling Event Procedures).  This gets tricky as there is no coordination with the Canon firmware - the things changed by CHDK are done "blind" - and sometimes the results may be inconsistent or just plain wrong.  And we don't know how to change every Canon settings - just the ones discovered via painful trial and error.

Quote
3. Many new options can be set with CHDK, but do the Canon settings have priority if set before CHDK loads?
Where options overlap, CHDK wins.

Quote
4. If a script is loaded and running, does the script replace CHDK settings? i.e  is the CHDK base firmware still running and a script interacts with it or does a script replace CHDK. When I'm switching back and forth between a script menu and the CHDK menu, I'm not sure what is running or what has priority?
A running script essentially has priority over CHDK settings. But if it does not explicitly change a setting, the base CHDK software actions will still happen.

Quote
I'm trying to get my head around these three ways of changing camera 'variables': Canon firmware settings, CHDK, and Scripts and how each interacts. At a very basic level I need to know if scripts just change what CHDK does as default by reading the status of settings then changing them, or scripts have to initialise settings from scratch?
The basic guideline is that scripts change only what they want to change in order to achieve their purpose. However, once things are changed they will stay changed after the script terminates until the next camera restart, unless the script is well behaved and records the initial state of what it is going to modify and then resets to that state when the script exits.

Quote
Incidentally, I've finally found out why these Canon Powershot 'A' series cameras give so much trouble with low battery warnings after testing my A560 on a bench power supply: The Canon firmware does more than just measure battery voltage. At power on it measures battery internal resistance by measuring the peak current against some reference value. The A560 takes about 350mA with the lcd lit and 80mA unlit so it must be calculating a peak value which will support flash recharging. I think the flaw is in the Canon firmware, because many low battery situations will support loads more shots. It's a shame the Canon algorithm appears to be responsible for dumping loads of re-chargeable batteries. Even a 2.5Amp bench supply can max out their internal resistance algorithm and put up occasional low battery warnings. The warnings remain persistent until the camera is re powered. I can stop this behaviour by adding a large capacitor to the supply wiring. I now test all my niMh batteries for internal resistance and the variation between brands is surprising. I also re-test after a few charge discharge cycles because some do not stay good. My battery of choice for consistent performance is the non-rechargeable Lithium AA. When I find Canons current sense resistor on my camera I will change its value. When people try to fix this problem by cleaning battery terminals, they are just trying to lower the battery internal resistance which the camera measures at power on. I think the real problem is with Canons measurement algorithm? The low battery voltage threshold set by Canon was for 1.5V cells like high capacity Duracells. There is a greater voltage margin for these cells than NiMh which start at 1.35V fully charged.
This is a very interesting development! Can I suggest you edit your post, cut this out, and start a new thread based on just this paragraph?  That way it does not get buried here in a "script newbie" thread. 

FWIW, I suspect the camera does not have the sophistication to measure battery internal resistance directly. What it does see is the brief voltage dip caused by that internal resistance during high current loads (i.e. at startup, or when recharging the flash, moving the lens, writing to the SD card). The firmware is not very smart though - if it did several measurements over a few seconds (i.e. when the average current draw was only 350mA) it would probably make a different decision about low battery status.
« Last Edit: 10 / December / 2017, 11:10:53 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline voxmagna

  • *
  • 33
Re: Script Newbie, Card Full file delete handling.
« Reply #19 on: 11 / December / 2017, 06:00:11 »
Quote
Incidentally, if you scroll down through the script parameters to the one that says Action if card full and set it to [ Delete ] , you should be able to watch it shoot one shot, delete it, shoot another shot, delete it, over & over with your SD card configuration.

I have this working now on a small card which I filled up with jpegs to test the script. I haven't moved on to test with a >4gb card yet with many more files and folders maxed with Canons <2000 file limit and tested the timed reboot. The script appears to delete files when the shot counter gets to 10 remaining. In an Intervalometer situation, Ultimate could be running with card full all the time. Because the script checks and deletes one oldest file before (or after?) a shot is taken, the time to shoot and recover for the next shot or multiple shots is delayed by the delete file operation each time. I know Canons have a shot buffer, but relying on this with larger fine high quality shot saves whilst the camera software is working hard and Ultimate is deleting a file seems too chancy?

10 shots (space) remaining should be a sufficient buffer size to stop the camera launching its card full message when re-powering. Although somehow I did manage to get the shot counter down to zero without last file deletes and got trapped in a loop of 'card full' and unable to use the delete button.

However, I can make a suggestion that if possible keep the 10 (or n) shot remaining buffer, but avoid the script continually deleting ONE oldest file pre and post shooting. It would be better to have a second check done on shots left when Ultimate does its maintenance reboot and have the last 'n' older files deleted where 'n' can be set in the menu and be anything from 0-999. This would ensure that Ultimate is rarely deleting files during the camera active period when it is shooting or slowing down the camera whilst it does it. For small cards you would choose a small value for n, but large cards running a CCTV camera trap could use a much higher value since it is only the last few days of shots that may be important.
Logged
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics