proposal - script shooting hooks - page 4 - General Discussion and Assistance - CHDK Forum supplierdeeply
« previous next »
Pages: 1 2 3 [4] 5 6 7 8 9 ... 30   Go Down

proposal - script shooting hooks

  • 290 Replies
  • 107289 Views
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: proposal - script shooting hooks
« Reply #30 on: 28 / December / 2014, 15:07:31 »
Advertisements
Quote from: reyalp on 28 / December / 2014, 14:41:37
Here's and example hitting the over exposure limits
Wow - quite the amazing job of ignoring the sun and keeping everything else constant !
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #31 on: 29 / December / 2014, 01:06:21 »
An update which contains no end user improvements.

I mentioned trouble testing earlier... it's hard to get a scene that exercises the different scenarios, and if I see it do something bad it can be hard to reproduce and verify fixes.

Thinking about this last night, I realized if I refactored the metering code a bit, I could feed it fake meter and over/under values. Then I realized that the log contains enough information to get back out the scene brightness. So I added a simulation mode that takes the normal log output as input (named rawopsim.csv), and does all the exposure calculation without actually shooting. The log can also be edited to create specific scenarios.

Here's a sunset with the current version:
http://youtu.be/KJ3d-fk6ZFI?list=UUvBOkM9SkswQao0KN7Dj0jA

This behaved pretty much perfectly (aside from the camera getting bumped and the crud flapping in front of the lens :haha), but aesthetically it would have been better if the under exposure protection had been set to be more aggressive (this was 10% / neutral -5).
« Last Edit: 29 / December / 2014, 01:08:03 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: proposal - script shooting hooks
« Reply #32 on: 29 / December / 2014, 09:17:01 »
I noticed you are not managing the ND filter (you mentioned that originally I know).    When I was working on the Lua version of the yass script I ran into issues where the camera would insert the filter and ruin the exposure.    IIRC it was usually when the sun came out from behind the clouds and intervening tree branches. 

A set_nd_filter(2) at the start of the script solved that.
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #33 on: 29 / December / 2014, 12:55:00 »
Quote from: waterwingz on 29 / December / 2014, 09:17:01
When I was working on the Lua version of the yass script I ran into issues where the camera would insert the filter and ruin the exposure.    IIRC it was usually when the sun came out from behind the clouds and intervening tree branches. 
Since rawopint holds down halfshoot for the entire sequence, the Canon firmware won't change the ND (or focus, or aperture) after the first shot. It could still start with the ND in, if the sun happened to be in the FOV.

Setting the ND out for the first is probably a good idea. Originally, I was thinking about having try to maintain the initial Canon exposure value, but just using the neutral value (and perhaps an EV comp option later) seems fine. This does mean that it takes several frames for the metering to settle.

I would like to have proper ND support to be able to put the ND in over specific shutter speeds, but one sticking point is getting the exact value of the ND. If the Canon firmware puts it in, you can compare the Av propcase and the min av propcase (at least for cams that don't have both iris and ND), but with CHDK control, the propcases don't seem to be affected by ND.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #34 on: 29 / December / 2014, 16:07:46 »
Quote from: reyalp on 29 / December / 2014, 01:06:21
Then I realized that the log contains enough information to get back out the scene brightness. So I added a simulation mode that takes the normal log output as input (named rawopsim.csv), and does all the exposure calculation without actually shooting.
I was feeling pretty smart when I did this. Of course, now I realize the under / over pct from the log won't respond to any adjustment, so (with additional hacks) it only "works" in the normal metering regime. :-[
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #35 on: 30 / December / 2014, 14:20:11 »
Update:
I wrote a new exposure calculation based on weights, which should solve most of the flapping issues.

There is still a little bit due to the low precision of shot_histogram, e.g. if you set over exposure to limit to 3%, you can only "ramp in" the change over ~3 distinct values. This gives me motivation to work on the histogram code...

There are now options to limit how far under/over exposure protection can drive the value of the metered area.  These are the Meter high/low thresh and limit described in the readme.

I also made simulate do something useful and fixed number of other bugs.

Sunset:
http://youtu.be/zlQaTRbmn_M
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #36 on: 31 / December / 2014, 19:17:53 »
In trunk revs 3866 and 3866 I updated the C code and rawoplib.lua support module. Scripts for previous versions will be incompatible.

I moved the rgb rectangle function to C, which speeds up drawing > 10x. I also implemented the cfa aware get and set pixel functions in C.

I have renamed these functions to *_rgbg. Functions that write take four color arguments in r,g,b,g order with the second g optional. Getters return in the same order, so lua callers can just ignore the last g if they want.

A added a test script CHDK/SCRIPTS/TEST/rawdraw.lua which tests some of the drawing functions. This will be expanded later. drtest.lua is also update for the changes.

Updated rawopint.lua attached. Not many functional changes: ND filter is forced out per waterwingz suggestion, and the debug display is reworked.

Boring sunset showing the previously mentioned histogram precision issue
http://youtu.be/Moji_gNpm6o

Edit:
Updated scripts fixes off by one in the Underexp EV menu option, changed video to a slightly less boring one showing the same problem. This run did 4118 shots before low battery shutdown, starting on less than a full charge. Time was 1:49. Target was 1.3 sec/frame, but it got pushed over by shutter time around 3400 shots in.

I think toward the end we see another issue which IIRC lapser noted before, that ISO changes only take effect in some increment greater than a single APEX96 unit (3?) (edit: nope, see http://chdk.setepontos.com/index.php?topic=12165.0)


Happy new year!
« Last Edit: 14 / March / 2015, 23:25:11 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #37 on: 02 / January / 2015, 17:28:56 »
In trunk 3873, I added histogram functionality to rawop. I'm not sure how this will end up fitting with shot_histogram, right now it's something to play with so I can figure out how it really should work.

The following are added

edit: see following posts.

histo=rawop.new_histogram([shift])
Creates a new histogram object, with the number of slots optionally reduced by shift. All other histogram operations are done on the object, like histo:foo() rather than from rawop. The memory used by the histogram is tied to this object, and garbage collected like other lua values.

histo:update(top,left,width,height,xstep,ystep)
Builds a histogram of the specified area. You can use the CFA offsets to build R, G, B histograms.

frac=histo:range(min,max[,'count'])
return number of values in range, either as a fraction in parts per 1000, or total count

total=histo:total_pixels()
returns total number of pixels sampled


Comments / rationalizations / issues:
1) The way shift is specified isn't clear. It should probably specify an absolute number of bits.
2) I will probably change the bit depth to be specified in update, with the memory (re-)allocated if needed. This would also allow optimizing to use shorts if the number of samples is sufficiently low, and only holding on to the memory during the actual use.
3) Does it makes sense to allow multiple simultaneous histograms? IMO, it probably does. You can update the same one as many times as you like, but it may be useful to have R G B all active at the same time, or have an overall scene histogram and separate one for a meter area or something.
4) It's easy to get overflow if you work with raw counts on a significant fraction of the frame buffer. (e.g 0.5 MP, * 4095 white level will just about wrap negative)
5) Are raw counts useful at all? I think so, you can do things like finding peaks more efficiently.
6) Is it useful to allow different bit depths? I think it is, since it allows allows you to trade memory and precision. It also gives you a really quick way to bin down to say 8 levels.
7) If you could average the histogram, you could meter the histogram area without the cost of reading the frame buffer again. However, it's easy to run into overflow problems if you don't do it in fp or 64 bit.
8) Is there any reason to allow range() to produce a fraction other than parts per thousand? I don't immediately see a need for more precision, and getting to % in lua is trivial.

Updated version of rawopint is attached. This uses the new histogram, and the parts per thousand precision eliminates some of the bad over/under protection behavior.
« Last Edit: 21 / January / 2015, 00:15:33 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #38 on: 02 / January / 2015, 18:20:09 »
Video with the new code. I didn't get the sun close enough to the FOV to trigger overexposure. There's some drops of melting frost falling on the lens toward the end.

http://youtu.be/u1iaoaYVEvI

I'm still seeing occasional glitches where the exposure change without any change commanded by the script or apparent change in external lighting. E.g. around 2:14

The log looks like
Code: [Select]
sv sv96 tv tv96 meter meter96 d_ev
75 371 0.000676 1011 724 0 100 0
75 371 0.000676 1011 722 0 100 0
75 371 0.000676 1011 695 -6 100 3
75 371 0.000691 1008 734 2 100 0
75 371 0.000691 1008 739 3 100 -2
For the first two frames, the shutter and ISO are constant, and the script hasn't requested any changes, since the scene is within rounding error of neutral. On the third frame, the meter value drops 6 (1/16th stop), and the exposure code requests +3 to start compensating. Smoothing prevents it from requesting the whole 6. On the next frame, the meter is over by 2, meaning the scene brightness has returned to essentially where it was before.

I don't think this is a real lighting change, since it affects the whole scene (although maybe shadow -> less glare on lens?)

I mostly been using the D10 for testing, I need to see if this happens with my other cams.
« Last Edit: 02 / January / 2015, 20:34:49 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14080
Re: proposal - script shooting hooks
« Reply #39 on: 06 / January / 2015, 00:58:40 »
Another application I've had in mind for raw metering is a psuedo auto exposure in continuous mode. So I wrote a script called contae.lua which does this.

It borrows a bunch of code from rawopint, but instead of being an intervalometer, it's meant to be used while shooting bursts of shots manually. While the script is running, keys other than full shoot and menu are just passed to the canon firmware. So you can open the func menu, adjust mf etc without leaving alt or exiting the script. Menu is currently set as the exit key.

In continuous mode, shoot_full is basically passed through as well, making the camera shoot for as long as you have it held. If not in continuous mode, it simulates continuous with the "hold half press and click shoot" strategy for as long as you hold the shutter down.

In either case, shots after the first shot in a burst are fed though the the same exposure system used by rawopint, with smoothing disabled and a large default ev step.

This isn't the same as real auto exposure, since it only reacts to the previous frame, but it should do a lot better than fixed exposure if you track a subject from sunlight to shade or something like that.

The interface is still in proof of concept state, but it works well enough to use for basic shooting. It doesn't know anything about jog dials etc, and menu isn't a great choice for the exit key.

For both scripts, I added an EV shift option, which lets you set the target meter level instead of the default neutral value. I also added an option to use the exposure level from the first shot as the target. They can be combined to get e.g. +1 ev from whatever the Canon firmware used for the first shot.

Attached zip contains both contae and rawopint, along with readme files for each.

I have shot a bunch of test time lapses with the current code (see my youtube channel from the previous links if you haven't had enough ;)). I think it's working pretty well, but I still see the glitches mentioned earlier, both on d10 and a540.
Logged
Don't forget what the H stands for.
Pages: 1 2 3 [4] 5 6 7 8 9 ... 30   Go Up
« previous next »
 

Related Topics