proposal - script shooting hooks - page 20 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 15 16 17 18 19 [20] 21 22 23 24 25 ... 30   Go Down

proposal - script shooting hooks

  • 290 Replies
  • 108111 Views
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #190 on: 07 / July / 2015, 17:25:05 »
Advertisements
Quote
It starts to get flicker on the end of the video. Not as much as on the last video in Reply #144
May be your newest version will reduce it…
I think this is mostly a flaw in the over exposure algorithm, which isn't really fixed in the new version :(

A big part of the problem is there is no way to know in advance how much you need to adjust exposure for given change in over exposure fraction. For something like the sun, a full stop might barely change the over exposed fraction at all, where as a large area of sky might go from completely over exposed to barely over exposed at all with a small fraction of a stop. I have some ideas for dealing with this (some kind of feedback, or additional damping, or measure of how much is near the limit), but it will take some time to develop and test.

You could change the "Max Ev change", but this will affect all exposure adjustments, and if it's too small, the script might not keep up with lighting changes. I could make the max change for "meter" and over/under separate options, but this would have side effects on the weight system, and there would still be no obvious way to pick a "good" value.
Logged
Don't forget what the H stands for.
*

Offline c_joerg

  • *****
  • 1248
Re: proposal - script shooting hooks
« Reply #191 on: 08 / July / 2015, 04:26:32 »
Quote
SD cards have their own CPU and do wear leveling and garbage collection so their performance can
be quite complicated, especially if they are used in a different way than the designers expected.

Yes I understand the problematic of SD Cards. I made a lot of test with speed on PC and in the cam. Results can be really random. I always format the SD card in the cam, hope this will be the best for the cam. On the last runs I used two different SDXC 64G cards which are formatted as eFAT. May be I should try them to format with FAT32.

Quote
I could make the log buffer to RAM for the whole run, but this would probably cause errors on long runs.

I would not change it…

Quote
You could also turn off logging, but then you wouldn't know how it's performing .

I will try it, when I do a fast run again…

Quote
A big part of the problem is there is no way to know in advance how much you need to adjust exposure
for given change in over exposure fraction. For something like the sun,
a full stop might barely change the over exposed fraction at all

The run is definitely different as on in Reply #144. On Reply #144 you really see an oscillation on meter96.
This was not seen on the last run.
Overexposure on the end of the run only exceed on two pictures.
The flicker is really not much. This can be reduced easy with deflicker filter...


Quote
You could change the "Max Ev change", but this will affect all exposure adjustments,

Does this really make a different for the last run?
Max Ev change was set to 32. Max changes on the end of the run where only 7 (calculated d_ev (n) - d_ev(n-1) or tv96(n) - tv96(n-1)). So far away from maximum.
Or a misunderstanding from my side?


The overexposure show really big jumps on the end of my last run.
I understand that adjust exposure can give unknown change in over exposure fraction.
I’m not sure, but it might be helpful, to give the overexposure a smooth before analyzing it.

From your documentation:
Quote
There are too many options, and their meanings are not clear.

I agree… specially for not experience user…
I still not try to play too much with all parameters.

On my first analog SLR I had a Switch for ‚Simple‘ and ‚Advanced‘ user.
Something like this could be helpful for beginners…
« Last Edit: 08 / July / 2015, 04:30:11 by c_joerg »
Logged
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd
*

Offline waterwingz

  • ******
  • 12139
    • Waterwingz
Re: proposal - script shooting hooks
« Reply #192 on: 08 / July / 2015, 07:09:25 »
Quote from: reyalp on 07 / July / 2015, 16:58:18
SD cards have their own CPU and do wear leveling and garbage collection so their performance can be quite complicated, especially if they are used in a different way than the designers expected.
For reference,  there was a discussion (with data) about this here.   It is a legitimate concern with "high shot rate" intervalometers.  Logging makes it worse.

Quote
I could make the log buffer to RAM for the whole run, but this would probably cause errors on long runs.
That's eventually what I did (with code from naccio) for the kap_uav.lua script, with the refinement of writing the buffer periodically to the SD card as it filled up.

Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline c_joerg

  • *****
  • 1248
Re: proposal - script shooting hooks
« Reply #193 on: 08 / July / 2015, 09:56:02 »
My understanding of rawopint is, there is only one write for a line after each picture.

Quote
of writing the buffer periodically to the SD card

What does ‘periodically’ mean? Collecting 100 Lines for 100 pictures and writing them together?
For time-lapse probably not helpful, because every 100th interval will be definitely wrong (maybe).

Quote
the log buffer to RAM for the whole run

May be an option for a fast and short run…
My longest run had 5000 pictures; I want to make a 24h run with 20000 pictures (4s Interval) or more…
What happens, when the external power supply gets disconnected….


Logged
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #194 on: 08 / July / 2015, 16:21:44 »
Quote from: c_joerg on 08 / July / 2015, 04:26:32
The run is definitely different as on in Reply #144. On Reply #144 you really see an oscillation on meter96.
This was not seen on the last run.
Overexposure on the end of the run only exceed on two pictures.
The flicker is really not much. This can be reduced easy with deflicker filter...
The oscillation in meter is small, but it's there. It's more obvious in over_frac and over_weight.  I think the oscillation just got more extreme in 144, it's gets larger toward the end of 188.

Quote
Does this really make a different for the last run?
Max Ev change was set to 32. Max changes on the end of the run where only 7 (calculated d_ev (n) - d_ev(n-1) or tv96(n) - tv96(n-1)). So far away from maximum.
Or a misunderstanding from my side?
The over/under protection always use the "max" value, scaled by the "weight", which depends on close (or far over) it is to the thresh value. So adjusting the max would change how much the exposure changes for a given amount of over exposure.

This again is because there is no way to know how much adjustment is needed. The "meter" calculates exactly how much it needs to get "correct" exposure, and then limits the change to the "max"

Quote
From your documentation:
Quote
There are too many options, and their meanings are not clear.

I agree… specially for not experience user…
I agree completely. I hope to simplify it eventually, but right now it's all experimental. I don't know what will work well in a given situation.

Although I hope to make it useful to users in the end, the I wrote the script because I needed a real world test case for the features I'm adding to the C code.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #195 on: 08 / July / 2015, 16:43:33 »
Quote from: waterwingz on 08 / July / 2015, 07:09:25
For reference,  there was a discussion (with data) about this here.   It is a legitimate concern with "high shot rate" intervalometers.  Logging makes it worse.
As we discussed before, in that case the log was being opened/closed for every line, which is not what rawopint does.
Quote
That's eventually what I did (with code from naccio) for the kap_uav.lua script, with the refinement of writing the buffer periodically to the SD card as it filled up.
Which is what the camera firmware does if you don't close or flush every line.

Quote from: c_joerg on 08 / July / 2015, 09:56:02
My understanding of rawopint is, there is only one write for a line after each picture.
It calls write() for every line, but the firmware physically writes the data to the file less often (probably, the details are mostly unspecified although I know there's a 32k buffer involved).
Quote
What does ‘periodically’ mean? Collecting 100 Lines for 100 pictures and writing them together?
For time-lapse probably not helpful, because every 100th interval will be definitely wrong (maybe).
IMO the actual problem probably isn't writing to the card per se, but that the card periodically takes much longer to write than normal, due to needing to GC erase cycles or something like that. Lots of small writes, or writes that don't line up with the cards underlying block structure probably make this worse.

There may be some optimal size that minimizes this. I went with OS buffering because it was easy, and there is at least some chance it's optimized to play well with typical cards.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #196 on: 12 / July / 2015, 21:19:25 »
Some updates in trunk 4191
Quote from: reyalp on 05 / July / 2015, 19:11:44
Error handling
Currently, the code doesn't check if you are in the raw hook, or if a valid raw buffer is available in the current mode (IE cameras that don't have raw in auto, or binned modes with low res raw).
All the functions that access the raw buffer will now error() if called outside the hook, or if CHDK knows the current shooting mode does not have raw (note that many ports probably don't do this correctly in all cases)

Quote
G1x blacklevel issue
I don't really want to change the code a lot to deal with quirks in the one camera, especially since the ISO issues aren't really resolved. However, we will probably run into other cameras with quirks later.
* values associated raw "neutral" needs to be re-calculated when black level changes.
* a callback on entry into the hook could be used to re-calculate the values.
I added code to update the blacklevel based values on hook entry.

Note that rawopint and the other scripts will need to be updated to correctly handle this. It also doesn't address the other ISO related weirdness on G1x.

This raises another question:
Should blacklevel and dependent values and functions (like raw_to_ev96) be accessible outside of the hook at all? In theory, they shouldn't, since any calculation done with them could become invalid before the next shot. However, for all (known) cameras except the G1x this is a needless inconvenience. I haven't decided which way to go.

Finally, I made a subtle change to the behavior of the hooks. Previously, if you didn't set the hook, it would become "active" momentarily when the hook was reached, without blocking the hooked task. I don't think this could have been detected reliably from script, bit in any case 4191 removes the ambiguity and hooks only become active if the hook has been set. If you just want to detect the hook being reached, you can check for hook.count() incrementing. I don't think this will affect any of my scripts.
Logged
Don't forget what the H stands for.
*

Offline c_joerg

  • *****
  • 1248
Re: proposal - script shooting hooks
« Reply #197 on: 13 / July / 2015, 02:37:57 »
Quote
Note that rawopint and the other scripts will need to be updated to correctly handle this.

So an update from my side to r4191 is not useful until rawopint (or isoinc) is updated?

Quote
It also doesn't address the other ISO related weirdness on G1x.

My understanding where, there are 2 things on G1X:
1) The black point at ISO320
2) The1/2 ISO kick at ISO400

R4191wil fix the black point but not the ½ ISO kick?
But R4191 together with the ISO Set mode Script on isoinc will work?
Logged
M100 100a, M3 121a, G9x II (1.00c), 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #198 on: 13 / July / 2015, 22:00:49 »
Quote from: c_joerg on 13 / July / 2015, 02:37:57
Quote
Note that rawopint and the other scripts will need to be updated to correctly handle this.

So an update from my side to r4191 is not useful until rawopint (or isoinc) is updated?
rawopint needs to modified to take advantage of this change.

I think isoinc should not need any changes.

Quote
My understanding where, there are 2 things on G1X:
1) The black point at ISO320
2) The1/2 ISO kick at ISO400
R4191wil fix the black point but not the ½ ISO kick?
Right, R4191 makes it possible to write scripts that use rawop and behave correctly when the blacklevel changes.
Quote
But R4191 together with the ISO Set mode Script on isoinc will work?
The details are confusing enough that I'm not sure, but it would be worth trying.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14082
Re: proposal - script shooting hooks
« Reply #199 on: 19 / July / 2015, 18:23:19 »
Quote from: reyalp on 05 / July / 2015, 19:11:44
* fb_info is not well suited to values changing on the fly. It's possible that other values could change at runtime in the future so this should probably be converted to individual calls. This may make it less convenient to write efficient lua code, but C calls shouldn't be worse than a table lookup.
In trunk r1493, I got rid of rawop.fb_info() and converted the members to individual function calls.

Most scripts that used rawop need to be updated for this change. This includes laulib/rawoputil.lua and the scripts under scripts/test so be sure to include those when you update you CHDK build.

Attached are updated versions of rawopint and contae. There are no changes to functionality, but they will see the correct blacklevel dependent values on G1x when it changes. I'm not sure if this will result in correct behavior, and it still doesn't account for the other ISO change at 400.

Quote
Should blacklevel and dependent values and functions (like raw_to_ev96) be accessible outside of the hook at all? In theory, they shouldn't, since any calculation done with them could become invalid before the next shot.
Still on the fence about this one. For now, I have not made them error if called outside the raw hook.
Logged
Don't forget what the H stands for.
Pages: 1 ... 15 16 17 18 19 [20] 21 22 23 24 25 ... 30   Go Up
« previous next »
 

Related Topics