proposal - script shooting hooks - page 12 - General Discussion and Assistance

waterwingz

12139

Re: proposal - script shooting hooks

« Reply #110 on: 05 / April / 2015, 10:40:17 »

Advertisements

Quote from: lapser on 04 / April / 2015, 23:53:31

I also found that it sometimes take a very long time to write a single line of the log file when you're writing a lot of picture data (raw files especially).

FWIW, I found something quite similar and charted it here :
Re: KAP & UAV Exposure Control Script

The "solution" was to buffer log writes in RAM and only squirt them to the SD card occasionally when the buffer got "big" or on script termination.

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: proposal - script shooting hooks

« Reply #111 on: 05 / April / 2015, 13:24:15 »

Quote from: waterwingz on 05 / April / 2015, 10:40:17

FWIW, I found something quite similar and charted it here :
Re: KAP & UAV Exposure Control Script

The "solution" was to buffer log writes in RAM and only squirt them to the SD card occasionally when the buffer got "big" or on script termination.

That's a good idea. I used a 1-line buffer in my script. If it takes longer than 200 msec to write a line, I put the next line in the buffer and write 2 lines the next time. If it takes >200msec and the buffer is full, I have to write anyway. But that doesn't happen too often.

The point is, the timing is more accurate with set_shot_interval(). But I thought of a way to test the pre-shot script hook flicker problem. Try adding msleep(10) after the script signals the pre-shot delay to end. That is, always start the shot after an msleep(10).

I think that makes the shutter timing more accurate, especially at faster shutter speeds. The shutter needs to open at a consistent time relative to the tick counter (I think).

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14080

Re: proposal - script shooting hooks

« Reply #112 on: 05 / April / 2015, 15:29:55 »

Quote from: c_joerg on 05 / April / 2015, 02:56:37

When I make pictures with 4s the interval goes to around 10s. This is a noise reduction which the cams (S110 and G1X) makes. There is nothing on the cam, to switch it off. I thought, I can switch it off with CHDK…

Right, this is dark frame subtraction, which you can force off in CHDK. What I was saying is that the flickering doesn't start when the shutter speed that reaches the length that triggers DFS, the shutter speed has been at 4 sec for many shots. The trigger seems to be ISO, but it is entirely possible there is some interaction between DFS and certain ISO levels. It's worth a try.

Quote

Would this help? With my knowledge right now I have not really understood what’s going on… 

I believe the current CHDK C code already does essentially the same thing lapser is suggesting. The CHDK code has changed substantially since he wrote his timelapse mods.

The fact that isoinc works shows this isn't a case of the ISO override code being completely broken, there is some other factor involved.

Quote

I have seen, you change the scale in histo:range. For the moon

Yes. histo:range now accepts a scale factor in the third parameter. This is one of the things I'm working on for the next version of the script.

Quote

Could you this also do in shot_histogram.c for get_histo_range?

I'm not inclined to change the shot_histogram code at this point. It should be fairly easy to convert your simple script to use the new raw hook code. If you want to post your code, I may be able to help.

Logged

Don't forget what the H stands for.

reyalp

14080

Re: proposal - script shooting hooks

« Reply #113 on: 05 / April / 2015, 15:53:56 »

Quote from: waterwingz on 05 / April / 2015, 10:40:17

Quote from: lapser on 04 / April / 2015, 23:53:31
I also found that it sometimes take a very long time to write a single line of the log file when you're writing a lot of picture data (raw files especially).
FWIW, I found something quite similar and charted it here :
Re: KAP & UAV Exposure Control Script

The "solution" was to buffer log writes in RAM and only squirt them to the SD card occasionally when the buffer got "big" or on script termination.

FWIW, I haven't seen a slowdown in logging in the rawopint script. The code for this keeps the file open and just calls write() for each line. This does mean you will lose a indeterminate amount of data if there is a crash, but the same is true if you buffer yourself in lua. You might see some old posts saying that keeping a file open is bad, but it turns this was a mistake from when we thought the FsIoNotify crash was due to having too many handles open.

The log code is a self contained CSV based log system, if anyone wants to borrow it, you can just copy everything from
-- log module
to
-- end log module
I'll put a separate post about this in the scripting forum.

Logged

Don't forget what the H stands for.

waterwingz

12139

Re: proposal - script shooting hooks

« Reply #114 on: 05 / April / 2015, 16:07:08 »

Quote from: reyalp on 05 / April / 2015, 15:53:56

FWIW, I haven't seen a slowdown in logging in the rawopint script. The code for this keeps the file open and just calls write() for each line.

The code that was running when I charted the "slowdowns" in the link I posted did a fopen/fclose for each line written to the log. So I guess it's understandable that it would slow down.

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: proposal - script shooting hooks

« Reply #115 on: 06 / April / 2015, 15:33:14 »

Quote from: waterwingz on 05 / April / 2015, 16:07:08

Quote from: reyalp on 05 / April / 2015, 15:53:56
FWIW, I haven't seen a slowdown in logging in the rawopint script. The code for this keeps the file open and just calls write() for each line.
The code that was running when I charted the "slowdowns" in the link I posted did a fopen/fclose for each line written to the log. So I guess it's understandable that it would slow down.

I noticed the occasional long delays in print() with continuous mode at intervals faster than 1 second for jpg, and 2 seconds for Canon raw (SX50 and G1X). Saving DNG files with CHDK is even slower (I never do that). I use print() for logging because I discovered it was a little faster than the Lua write() statement. I worked pretty hard to get the maximum shot rate in continuous mode.

I've been doing astrophotagraphy lately with the Canon EOS-M and Magic Lantern. There are a lot of bugs in the EOS-M port, so you can't even do basic things like exposure bracketing. There isn't any scripting available either. I sure wish Phil, Reyalp, and Waterwingz could work on it! You guys are fantastic.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

c_joerg

1248

Re: proposal - script shooting hooks

« Reply #116 on: 09 / April / 2015, 02:36:00 »

Hello,

I tried to run a new test with G1X but unfortunately the G1X starts with an E42 Error

Then I make a run with S110 and max Tv=4s. It looks good. May be I stopped too early. But here was no problem around sv96=576.
I used the last update from hookutil.lua and add the additional delay which you suggested.

I hope I get the G1X fixed…..

https://www.youtube.com/watch?v=p7RhF01WWCE

rawopint.csv (63.43 kB - downloaded 14 times.)

Logged

M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

c_joerg

1248

Re: proposal - script shooting hooks

« Reply #117 on: 10 / April / 2015, 18:05:02 »

I got the G1X fixed. One night without battery helps….

I run a test with G1X and max Tv=2s. It looks like that flicker starts at the same sv96 values as with the 4s run. I used the last update from hookutil.lua and add the additional delay which you suggested.

Does it make sense to run isoinc with G1X with a Tv=2s? So with additional tv settings?

http://youtu.be/XhYMedSxgws

rawopint9.csv (172.31 kB - downloaded 13 times.)

Logged

M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd

reyalp

14080

Re: proposal - script shooting hooks

« Reply #118 on: 10 / April / 2015, 22:22:26 »

Quote from: c_joerg on 10 / April / 2015, 18:05:02

I run a test with G1X and max Tv=2s. It looks like that flicker starts at the same sv96 values as with the 4s run.

Thanks, that's a useful data point.

One idea might be to try isoinc over this range, with a subject that is dark enough to require a similar exposure length.

Did you ever try with dark frame subtraction disabled? Skimming the manual, I don't see what exposure length triggers it on this camera, but on most I think 2 sec would.

I did see in the manual this camera has some settings related to auto-ISO: The "Max ISO Speed" "Rate of change". These might have some effect on CHDK ISO overrides, since the overrides use Canon auto mode internally.

The dynamic range corrections settings might also have some effect.

edit:
If you think it sounds like I'm grasping at straws... that would be an accurate assessment

« Last Edit: 10 / April / 2015, 22:29:34 by reyalp »

Logged

Don't forget what the H stands for.

c_joerg

1248

Re: proposal - script shooting hooks

« Reply #119 on: 11 / April / 2015, 03:54:53 »

Quote

Did you ever try with dark frame subtraction disabled?

Not yet, will be one of the next thinks which I do.

Quote

Skimming the manual, I don't see what exposure length triggers it on this camera, but on most I think 2 sec would.

I think also as start at 2. But I would expect a jump in the log file with the sleep values when it starts or not?

Quote

I did see in the manual this camera has some settings related to auto-ISO: The "Max ISO Speed" "Rate of change". These might have some effect on CHDK ISO overrides, since the overrides use Canon auto mode internally.

Ok, I thought when I use fixed ISO100, this has no effect.
The settings at this run where
Max ISO Speed = 1600
Rate of change = Standard
High ISO NR = Standard

Quote

The dynamic range corrections settings might also have some effect.

I have not looked about this yet, but should not have an effect on RAW data or?

Quote

If you think it sounds like I'm grasping at straws... that would be an accurate assessment

Everythink is OK. I’m really interested to get it run and what’s going on and why it not happened on S110….

edit:

Run isoinc with high Tv=8s on G1X. That’s it. Meter makes a jump…

One other thing. I tried to run from 520 to 640. But isoinc stops at 603. I saw this later so I have not seen if there was any exception in lua…

I will try to run with different parameters on S110 and G1X….

isoinc.csv (8.56 kB - downloaded 15 times.)

« Last Edit: 11 / April / 2015, 05:54:44 by c_joerg »

Logged

M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd