proposal - script shooting hooks - page 2 - General Discussion and Assistance - CHDK Forum supplierdeeply

proposal - script shooting hooks

  • 290 Replies
  • 61980 Views
Re: proposal - script shooting hooks
« Reply #10 on: 15 / June / 2014, 08:52:21 »
Advertisements
Random thought - would it be possible to register a function name in the script that the hook code could then 'call'.
I think I asked at one time if the script engines are re-entrant?  I was thinking at the time about assigning individual scripts to camera buttons or events to make it easy to mod camera performance without a big honking master script.   The answer IIRC was "no" - things like the action stack come to mind immediately as a problem.

Would that not be a requirement for hook "callbacks" to work?
Ported :   A1200    SD940   G10    Powershot N    G16

*

Offline philmoz

  • *****
  • 3200
    • Photos
Re: proposal - script shooting hooks
« Reply #11 on: 15 / June / 2014, 09:05:38 »
Random thought - would it be possible to register a function name in the script that the hook code could then 'call'.
I think I asked at one time if the script engines are re-entrant?  I was thinking at the time about assigning individual scripts to camera buttons or events to make it easy to mod camera performance without a big honking master script.   The answer IIRC was "no" - things like the action stack come to mind immediately as a problem.

Would that not be a requirement for hook "callbacks" to work?

Yes, that would cause problems.
Calling script functions from a different task would most likely be problematic as well.

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline reyalp

  • ******
  • 12796
Re: proposal - script shooting hooks
« Reply #12 on: 15 / June / 2014, 16:43:12 »
Yes, that would cause problems.
Calling script functions from a different task would most likely be problematic as well.
Yes, calling from a different OS task would be bad. Calling a separate Lua "thread" from kbd_task might be possible in theory, but I think it would be counter productive.

What I do plan to do to simplify usage is to add a wait_ready() function that uses the action stack to wait until the hook is reached, with a timeout. This should be straightforward, I just haven't got around to do doing it, and the current code is useable.

Note there are cases (like chdkptp rsint) where you want script to continue doing stuff while it waits for the hook to be ready, so both options should be available. wait_ready could be done as a lua module without any C code, but it might be more convenient to have it standard.
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 12796
Re: proposal - script shooting hooks
« Reply #13 on: 22 / December / 2014, 02:16:37 »
Quote
wait_ready could be done as a lua module without any C code, but it might be more convenient to have it standard.
I somehow didn't end up posting here when I did it, I added a module called hookutil in 1.3 that includes wait_ready().

I have now started working on the next phase of this project, which is allowing lua to analyze and modify the contents of the raw buffer. With the shooting hooks, the approach is somewhat different than the old shot histogram or lapsers code.

Attached patch and test script includes some initial work. This is very preliminary, but in the spirit of keeping the dev branch unstable, I'll probably check it in if no one screams. Function names, parameters, code organization all subject to change.

The code is currently a C lua module called "rawop". This isn't dynamically loadable (yet).

rawop currently provides the following functions (as rawop.whatever)

fb_info
Returns a table containing information about the raw framebuffer, e.g for d10
Code: [Select]
={
 width=4104,
 height=3048,
 white_level=4095,
 black_level=127,
 cfa_pattern=16777729,
 bits_per_pixel=12,
 active_area={
  x1=0,
  y1=10,
  x2=4072,
  y2=3040,
 },
 jpeg_area={
  x1=36,
  y1=25,
  x2=4036,
  y2=3025,
 },
}

raw_to_ev96
converts a raw value to a pseudo APEX96 value. This is very primitive, it doesn't know anything about different color channels, but it does give a rough approximation of how many stops of exposure the value is from neutral.

raw_ev96 = log2((raw - blacklevel)/(raw_neutral-blacklevel))

where raw_neutral is based on the average value of a DNG shot uniform grey subject with auto exposure on my d10.

The remaining functions are only valid inside the raw hook, although this isn't currently enforced.

value=get_pixel(x,y)
set_pixel(x,y,value)
These should both be self explanatory

meter(x1,y1,xcount,ycount,xstep,ystep)
This takes the average raw value of the defined region.
x1 and y1 are the upper left corners
xcount and ycount are how many pixels to sample in each direction. They are NOT width and height unless the step is 1.

xstep and ystep are how many pixels to skip between in each direction, so width is xcount*xstep

To prevent overflow, the total number of pixels must less unsigned_max / white_level. For a 12 bpp camera, this is about 1 million. For 14 bpp, it's only 256k. You could of course meter multiple areas and average the results.

On my d10, it takes about 0.5 second to sample 1m pixels. Doing the same with get_pixel in a Lua for loop (optimized and using set_yield) takes about 11 seconds.

meter isn't directly aware of the cfa pattern, but you could sample particular channels choosing your start and step appropriately.

The test script is a simple rapid shooting intervalometer. It uses meter and tries to adjust the tv so that the next shot will be "neutral". If the "draw" option is enabled, it draws a box around the metered area. A real application would probably want a smoothing function for the exposure adjustments. I will try to come up with a more realistic script a bit later.

There's a bunch more I could talk about but this post is already long enough ;)
Don't forget what the H stands for.


Re: proposal - script shooting hooks
« Reply #14 on: 22 / December / 2014, 08:58:30 »
There's a bunch more I could talk about but this post is already long enough ;)
Really nice work.  Can't play with it (much) this week but its a big addition to 1.4.0.
Ported :   A1200    SD940   G10    Powershot N    G16

*

Offline philmoz

  • *****
  • 3200
    • Photos
Re: proposal - script shooting hooks
« Reply #15 on: 22 / December / 2014, 13:57:11 »
Very nice, great job :D

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline reyalp

  • ******
  • 12796
Re: proposal - script shooting hooks
« Reply #16 on: 22 / December / 2014, 15:45:03 »
I haven't decided how histogram fits into this yet. Some notes (mostly for my own reference, feel free to tl;dr)

As far as I understand it, the current shot_histogram operates as follows:
* The whole active area is sampled
* The step size of 31 ensures the all the colors are sampled, and keeps the total counts down
* Bit depths higher than 10 are shifted to give a 10 bpp value
* Counts are stored as unsigned shorts
* The histogram can only be queried as % pixels in a range.
* memory for the histogram is allocated for as long as the histogram is in use (included in the module in 1.4)

This is sufficient to get an idea if there are a lot of pixels near over or under exposure. By repeatedly querying, you can get an idea where the peak is and how much of the dynamic range is used.

Drawbacks:
* Can't set sampled area
* Sample density is fixed, and relatively sparse
* Can't sample specific color channels
* Using percentages reduces precision.
* Shifting to 10 bpp reduces precision
* 10 bpp values aren't directly compatible with the meter/get_pixel values, fb_info black level, whitelevel.
* get_histo_range is limited due to ubasic limitations. In lua it would be nice to be able to access the histogram as an array
* If the megapixel war goes over 60 MP, ushorts will be at risk of overflow :-[

Other:
Average of histogram value*count is equivalent to meter average, but a dedicated meter is faster and more memory efficient if you don't need histogram.
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 12796
Re: proposal - script shooting hooks
« Reply #17 on: 24 / December / 2014, 04:03:53 »
I've checked in the initial code, not changed too much from the patch a couple posts back. Still very much a work in progress.

I also added two lua files
LUALIB/rawoplib.lua
This has some helper functions for drawing and metering in RGB. Some of this will probably go in C code.

SCRIPTS/TEST/drtest.lua
A dynamic range tester. It takes the exposure selected by canon auto exposure, and then shoots a selectable number of stops over and under. This can be used to check the raw to ev approximation.

To use it, you should aim at a uniform, neutral color subject.

A detailed log is recorded in A/drtest.csv. The m, m96, r, r96 etc columns recorded the "metered" value and apex96 ev equivalents. For exposures that aren't clipping, the m96 value divided by 96 should give approximately the number of stops of over or under exposure.

"peak" is the peak value detected in shot_histogram. bl% and wl% are the percent of pixels near black level and white level respectively.

If you enable "draw" it draws a box around the metered area and bars showing the relative brightness of each channel. This mostly a test of the drawing stuff.
Don't forget what the H stands for.


Re: proposal - script shooting hooks
« Reply #18 on: 24 / December / 2014, 13:54:05 »
Looking at this,  it seems we might be close to being able to play with "in camera"  HDR algorithms now.  (scripting or embedded C)
Ported :   A1200    SD940   G10    Powershot N    G16

*

Offline reyalp

  • ******
  • 12796
Re: proposal - script shooting hooks
« Reply #19 on: 25 / December / 2014, 21:02:44 »
Looking at this,  it seems we might be close to being able to play with "in camera"  HDR algorithms now. 
I think you could actually do a fair job of this with just shot_histogram, but it would be inconvenient. I originally wrote the drtest script using only histogram, when I was trying to figure out the how to do the meter values.


Getting back to the project at hand:
I haven't been doing much with the C code, because I need a real application figure out how it should really work. So I've been working on an intervalometer script that uses the new feature.

obsolete see later posts

Current version attached. This is not completed, in particular, it doesn't handle aperture or ND filter adjustments yet, and the way metering is specified is not very clear.

The basic idea is the script keeps the camera in half_press and repeatedly clicks full shoot. The exposure for each subsequent shot is calculated from the raw of the previous.

The metering is done using an adjustable sized rectangle in the middle of the screen, and the script normally tries to keep this at the "neutral" value. Additionally, shot_histogram is used to limit the % of over and under exposed pixels to certain values.

I think this is working pretty well, but I encountered an oddity with D10. In one run, about 1 in 20 shots ends up with ~1/5 stop lower exposure even though no exposure parameters have changed.
I don't think this is a real lighting change (like a bird or airplane shadow), since it is apparent in the sky. According to the log, the tv and sv are unchanged. I've done a couple more runs without seeing it so  :-[


Script options
shots:
number of shots

Draw:
Draw metering information on the raw buffer. This consists of
1) a box around the meter area
2) A bar above the meter box, representing the metered value as a % of white level (log scale).
3) A bar under the meter box, representing the change in exposure as a as a fraction of the max EV change value described below. The origin is at the center, with longer exposure to the right, and shorter exposure to the left.
Drawing can take a significant amount of time, especially if the meter area is large  (e.g. 0.5 sec for a 1200 px on d10). The log contains this information and much more.

Interval sec/10:
time between shots, in 10ths of a second.

Meter size:
Size of metered area in pixels. This should be even and a multiple of step. Currently, it is always square and centered.

Meter step:
Ever step-th pixel is measured. To sample all colors, it should be odd.
To prevent overflow, the total number of pixels (size/step)2 must less unsigned_max / white_level. (~1M for 12 bpp, 250K for 14 bpp). Larger numbers of pixels will be slower.

Max ev change 1/x:
Maximum exposure change in a single step, in fractions of a stop. The actual exposure change is smoothed, so the max value will be only used if there is a sufficiently large, consistent trend in scene brightness.

Max Tv:
Longest shutter speed to use, in 1/100ths of a sec. If required exposure is longer, ISO will be used up to ISO limits.

Target ISO:
ISO value to use if shutter limits (described below) not reached. Should normally be lowest ISO

ISO adj TV Sec/100:
Shutter value (in 1/100ths of a sec) at which ISO will start to be increased. If the exposure time exceeds this value, exposure will changes are split 50/50 between shutter and ISO, subject to limits below

Max ISO:
Absolute max ISO to be used.

Av Pref:
Not implemented

Overexp thresh %:
The percent of pixels that must be near to white level to trigger over exposure protection. This uses shot histogram, so includes the whole frame not just the meter area. If this threshold is hit, the script will prevent further exposure increase from the meter. If it is exceeded, exposure will be reduced.

Underexp thresh %:
Similar to the above, but for pixels near black level. Underexposure is only checked if over is not triggered. Under exposure adjustment is also not triggered if it would push the metered exposure more than 1 stop over the "neutral" value.

Display:
Display power saving mode. On = always on, Off use set_lcd_display off. Bklt_Off use set_backlight every shot. Pressing set while the script is running will turn the display on for 30 sec.

Append log:
If unchecked, the logfile "A/rawopint.csv" is overwritten every time the script is run.

The is a CSV with a bunch of columns. The most relevant to the metering process are

sv, tv
exposure values for this shot (image number indicated in the exp column)

meter, meter96
the average value of the metered area. *96 indicates this as an APEX96 value where the target exposure is 0. Negative values indicate an underexposure, positive is overexposure.

over_pct, under_pct
shot histogram percentages in the over and under limits.

d_ev
exposure change requested for the next shot. Negative (confusingly) means increased exposure.

desc
various cases in the metering process trigger messages here, e.g if over exposure protection is triggered, shutter limits hit etc.

Umm.... if you made it this far, help yourself to another :xmas cookie and beverage   :haha

edit:
The "sleep" column indicates how long the script slept between shots. If it's negative, it means the script wasn't able to maintain the requested frame rate.

edit:
Might help if I attached the script
« Last Edit: 26 / December / 2014, 14:03:15 by reyalp »
Don't forget what the H stands for.

 

Related Topics