CHDK Forum
CHDK Development => General Discussion and Assistance => Topic started by: reyalp on 09 / September / 2018, 18:30:44
-
Working on the ND stuff and trying to figure out which cameras have a real aperture lead me down some other rabbit holes :-[
Some general observations:
Modern cameras (starting around G12, maybe) need CAM_AV_OVERRIDE_IRIS_FIX to control the aperture in continuous shooting. This calls MoveIrisWithAv to set the aperture rather than just setting propcases. However, the define is currently only set up to work in bracketing, so scripted aperture changes in continuous mode (ala rawopint etc) doesn't work.
This should probably be changed to apply to any set call while in shooting, but the set_now/set_later logic makes it somewhat complicated. Also, it doesn't appear to be required "quick" mode, but if we just check get_shooting it would still be applied there. This would probably be harmless.
It's likely that some cameras which need this define don't have it set, or vice versa.
Some current CHDK functions use GetCurrentAvValue. This function (also discussed in https://chdk.setepontos.com/index.php?topic=13508.0) returns the current value (not a half press value or next shot setting) so mixing it with propcase values that only update on half press seems suspect. The history of this usage is unclear to me, but it seems to have been around for a long time. It's used in
* shooting_get_real_aperture - used only for misc display and _min_real below
* shooting_get_min_real_aperture - used in shooting_get_subject_distance_
* shooting_update_dof_values
Attached avtest.lua script is similar to ndtest and isoinc for aperture.
By default, the script will step through aperture values from min to max in 1/3 stops, compensating with shutter to attempt to the keep the exposure constant. It has the usual single/quick/cont modes. If it prints "no warnings" aperture control probably works with the selected settings, otherwise there are possibly problems with the scene or bugs in the port.
You can also use it to run through the available range in 1 APEX*96 steps to measure the precision of aperture control.
Testing on g7x, control is quite good (see attached chart, "m96_err" is the difference between predicted meter and actual in APEX96. I suspect the biggest outliers are ambient light changes), so smoothly ramping through the AV range for DOF effects in timelapse should be totally viable with modest software de-flicker.
Test video
edit:
Youtube compression makes it harder to see what's happening, link to original file https://app.box.com/s/r1eg4is9dqrp3kkpp0r3x458wqrlnxix
edit 2018 09 11:
Updated script to handle ISO better
edit 2018 09 13:
Fix formatting of aperture values in log
edit 2018 10 06:
* Error now calculated from baseline shots at the starting aperture
* Log av96_err (Av compared to get_current_av96) and m96_err_cur (meter 96 error assuming get_current_av96 is correct)
-
Attached avtest.lua script is similar to ndtest and isoinc for aperture.
I missed the attachment…
You can also use it to run through the available range in 1 APEX*96 steps to measure the precision of aperture control.
That has always interested me…
so smoothly ramping through the AV range for DOF effects in timelapse should be totally viable with modest software de-flicker.
That's all very interesting…
I did not expect that you can change the aperture in 1 APEX steps because this is a mechanical setting. In the ISO setting, it is still to be understood because it is probably only a DA converter here.
Test video
In continuous mode or quick?
-
Some general observations:
Modern cameras (starting around G12, maybe) need CAM_AV_OVERRIDE_IRIS_FIX to control the aperture in continuous shooting. This calls MoveIrisWithAv to set the aperture rather than just setting propcases. However, the define is currently only set up to work in bracketing, so scripted aperture changes in continuous mode (ala rawopint etc) doesn't work.
This should probably be changed to apply to any set call while in shooting, but the set_now/set_later logic makes it somewhat complicated. Also, it doesn't appear to be required "quick" mode, but if we just check get_shooting it would still be applied there. This would probably be harmless.
It's likely that some cameras which need this define don't have it set, or vice versa.
I have a problem with AV bracketing in continuous mode on the sx410.
Exif is updated for each image as expected, but there are no differences in the images or in the histogram.
With CAM_AV_OVERRIDE_IRIS_FIX defined in platform_camera.h occurs a crash.
On the other hand, AV bracketing works well from a script, eg Ultra HDR v3.1 ,without defined CAM_AV_OVERRIDE_IRIS_FIX.
EDIT:ROMLOG attached.
-
I missed the attachment…
Ooops, forgot to attach it :-[ Added to the first post.
I did not expect that you can change the aperture in 1 APEX steps because this is a mechanical setting.
I wasn't sure what to expect. I suppose it doesn't require more precision then focus or zoom, but on the other hand, the stock firmware doesn't really seem to need 1 unit steps.
I will try to do a more controlled test later.
In continuous mode or quick?
Quick, it doesn't work in continuous on G7 X because of the CAM_AV_OVERRIDE_IRIS_FIX issue.
-
I have a problem with AV bracketing in continuous mode on the sx410.
Exif is updated for each image as expected, but there are no differences in the images or in the histogram.
With CAM_AV_OVERRIDE_IRIS_FIX defined in platform_camera.h occurs a crash.
On the other hand, AV bracketing works well from a script, eg Ultra HDR v3.1 ,without defined CAM_AV_OVERRIDE_IRIS_FIX.
Ultra HDR https://chdk.wikia.com/wiki/Ultra_HDR_Script uses shoot (a half shoot cycle every shot) so that makes sense.
I think the crash is because it needs a semaphore created for MoveIrisWithAv in init_required_fw_features. G7X boot.c has an example.
av_override_semaphore should probably be at 0xCF30 on sx410 100c
-
Quick, it doesn't work in continuous on G7 X because of the CAM_AV_OVERRIDE_IRIS_FIX issue.
Is this the same on G1x or M3?
What happens with aperture on G7x between the shoots in quick mode?
My M3 always opens the aperture in quick mode between the shoots. G1x aperture doesn’t change aperture in quick mode between the shoot. I would expect more m96_err on M3.
-
Is this the same on G1x or M3?
G1x almost certainly has the same problem as G7x. I don't know about M3.
What happens with aperture on G7x between the shoots in quick mode?
Like G1x, it does not open between shots.
-
Ultra HDR https://chdk.wikia.com/wiki/Ultra_HDR_Script uses shoot (a half shoot cycle every shot) so that makes sense.
I think the crash is because it needs a semaphore created for MoveIrisWithAv in init_required_fw_features. G7X boot.c has an example.
av_override_semaphore should probably be at 0xCF30 on sx410 100c
Thanks, now works well.
The patch is attached.
-
The patch is attached.
Thanks. Added in r5100.
-
Made just an avtest run with G1x. 16 shoots with quick and 16 with cont. As you said, cont mode doesn’t work.
I’m wondering why ISO increases every shoot in quick mode. Maybe it has something to do with the setting of ISO here (the difference of 1).
https://chdk.setepontos.com/index.php?topic=13191.msg137008#msg137008
But meter increases more than ISO…
-
I’m wondering why ISO increases every shoot in quick mode. Maybe it has something to do with the setting of ISO here (the difference of 1).
Yeah, looks like that plus sloppy coding on my part. I've updated the script in the first post to not touch the ISO if the camera is already in manual ISO mode, and when it does control ISO use the value from the initial half press rather than the last value read from the propcase.
-
Another run with G1x in Quick mode. Looks like G1x has more noise in m96_err as G7x…
-
Did a run with M3 in all 3 modes (Quick, Cont, Single). I used EF50mm 1:1.8 STM lens.
Get_min_av96 returns a nil value on M3 so I set values from 200 to 600 (may be max as well).
Locks like all 3 modes are working!
There is a big noise in m96_err and some warnings in log files. The warnings came from the big difference?
-
Did a run with M3 in all 3 modes (Quick, Cont, Single). I used EF50mm 1:1.8 STM lens.
Get_min_av96 returns a nil value on M3 so I set values from 200 to 600 (may be max as well).
Locks like all 3 modes are working!
Good to know. Perhaps available AV range can be found though other functions on EOS cameras.
There is a big noise in m96_err and some warnings in log files. The warnings came from the big difference?
Yes, all the warnings were from "high error" which is arbitrarily set to > 6 APEX*96 units.
Interestingly, your error does not appear to be random: The 4th, 7th, 10th and 11th all have high error, but across runs are within 2 units.
"error" is simply the meter96 difference from the previous shot compared to the expected value. With "constant exposure" shutter speed is adjusted to compensate for the Av change so the expected value is the previous value.
-
Interestingly, your error does not appear to be random: The 4th, 7th, 10th and 11th all have high error, but across runs are within 2 units.
It probably depends on how far the set value is away from a whole aperture. I have not tested smaller steps yet.
As you can see in this video, I would expect greater deviations in the adjustment of the aperture. Interestingly,
this does not seem to be the case with the G7x.
-
Why AV96=200 delivers f2.59 in the log file? f2.06 should be Ok
AV96=232 delivers f2.311 which looks OK
Looks like av96_to_aperture is limited from f2 to f16.
What delivers the G7x at f1.8?
It’s time to extend av96_to_aperture from f1.2 to f32….
I was looking at implementation of av96_to_aperture. Implementation in C is different as in hostlua functions. Why?
-
I made 4 runs in quick mode with 4 different lenses
(logging of focal length would help to identify them):
EF 50mm f1.8 USM
EF 24mm f/2.8 IS USM
EF-S 55-250mm f/4-5.6 IS STM at 55mm
EF-M 15-45mm f3.5-6.3 IS STM at 15mm
Only the native EF-M lens didn’t open the aperture between shots. All other lens always gets open between the shoots. I expected less noise in m96_err but I can’t see it.
-
Another 2 runs in quick mode with 2 different lenses. 50 shoots and step of 4.
EF 50mm f1.8 USM
EF-M 15-45mm f3.5-6.3 IS STM at 15mm
It works very well but I have definitely to deflicker a time-lapse scene…
-
Last run for today :)
G1x with step size 1. More noise at G7x but less than M3…
-
Why AV96=200 delivers f2.59 in the log file? f2.06 should be Ok
Oops, that's a formatting error on my part. It should be 2.05[9]. All the calculation and exposure control is done in APEX*96, so this should only be a log issue.
I was looking at implementation of av96_to_aperture. Implementation in C is different as in hostlua functions. Why?
Presumably someone didn't want to figure out how to do it in integer-only hostlua.
The CHDK C implementation should be correct, and should work for anything over f/1.0. We should probably support f/1.0 and below now that we have ILC cameras...
G7 X works correctly from f/1.8
It works very well but I have definitely to deflicker a time-lapse scene…
If the error has a repeatable component it would be relatively easy to come up with a correction table and apply the difference in Tv.
A better way to measure error might be to take several shots at a fixed value, and then compare subsequent to the average. In the current script, the accuracy of the previous shot can significantly affect the error reported.
-
Updated first post to fix logged non-APEX aperture values.
-
Presumably someone didn't want to figure out how to do it in integer-only hostlua.
Is it possible to use the CHDK C implementation in hostlua (like a library call)?
If the error has a repeatable component it would be relatively easy to come up with a correction table and apply the difference in Tv.
The value will not be constant at the aperture. I have already measured this at constant aperture in the past.
With a time lapse, I would change the aperture in one direction only. If, for example, you reach the maximum exposure time with a time-lapse, then I would continue to open the aperture. If it gets brighter again, then I would shorten the exposure time (with a hysteresis like the ND filter).
A better way to measure error might be to take several shots at a fixed value, and then compare subsequent to the average. In the current script, the accuracy of the previous shot can significantly affect the error reported.
Yes of course, but a certain systematics can be seen in meter96.
As next step, I will also create a time lapse with variable aperture. For moving objects, of course, it would be an advantage if the script ran at a fixed interval.
-
I did another test run with the M3 (cont mode).
25 pictures with AV96 = 560 and 25 pictures with AV96 = 576. As you can see well, only larger errors occur (at index 25) when the aperture is changed. At constant aperture, the m96_err is only + -1.
In another run (30 pictures) I fixed the aperture in which I twisted the lens after dimming (Canon Lens Twist trick). For the script to work like this I had to delete Set TV. Here you can see very well that no more error arises. So my light source does not seem to flicker.
-
Interestingly, your error does not appear to be random: The 4th, 7th, 10th and 11th all have high error, but across runs are within 2 units.
I think I know the cause now. I have now used as a increment 1. I suspect that the aperture really changed only every 12 steps, so not like the G7x at every step.
-
Another run with the EF 50mm from AV96=160 to 210 and with EFM 15-45 from 480 to 530.
Actually, I always see jumps to 12 APEX96 with the exception of the EF 50mm between AV96 = 160 and 192. Here the aperture seems to change by 1/3 EV.
But why does the aperture only change every 12 APEX96 values?
8, 16 or 32 I could understand yes ....
-
My first test video with M3 and EF 50mm from f1.8 until f16. 1/8 EV steps a still small enough to get smooth aperture ramping. I used deflicker CS in VirtualDub.
Really great stuff and great work from reyalp :)
Update:
Download link:
https://drive.google.com/file/d/1vKuuqLesvFZ6tEPQIFqqWypD4GDGmYT-/view?usp=sharing
-
My first test video with M3 and EF 50mm from f1.8 until f16. 1/8 EV steps a still small enough to get smooth aperture ramping. I used deflicker CS in VirtualDub.
Really great stuff and great work from reyalp :)
Nice work and thanks for all the test reports. It doesn't surprise me that the EOS M cameras are different, or that some work in 1/8th stop (12 unit) steps.
It would be nice for CHDK to be able to know which stops are valid.
There are quite a few iris related eventprocs that seem to be specific to the M cameras. Some I noticed
MoveIrisWithAvNotSupportAbsFNo
IsIrisSlowCtrlAvailable
EFLensCom.GetFNo
EFLensCom.GetFNoRange
EFLensCom.MoveIris_FNo
EFLensCom.MoveIris
edit:
Get_min_av96 returns a nil value on M3 so I set values from 200 to 600 (may be max as well).
Looking a bit more, this surprises me. The functions are found and appear to be real code that is used elsewhere in the firmware.
Edit:
Never mind, I see you are using CHDK build 4928. The min / max Av functions were added in 4998. The error you get is that the function itself is nil.
-
or that some work in 1/8th stop (12 unit) steps.
Looks like my entire lens has 12 unit step (with the exception of the beginning).
Here is another run with step size 1 with
EF 24mm f/2.8 IS USM
EF-S 55-250mm f/4-5.6 IS STM at 55mm
EF-S 10-18mm/f4.5-5.6 EF-S IS STM at 10mm
It would be nice for CHDK to be able to know which stops are valid.
Yes of course…
Edit:
Never mind, I see you are using CHDK build 4928. The min / max Av functions were added in 4998. The error you get is that the function itself is nil.
Ups, still used a test version from lapser ;) . Update done…
I tested my entire lens with the following Script:
press'shoot_half'
repeat sleep(10) until get_shooting()
local AV96=get_prop(props.AV)
release'shoot_half'
print_screen(get_day_seconds()/10)
local dof=get_dofinfo()
print("AV Values")
print(' FL=',dof.focal_length)
print(' AV96=',AV96,',f',fmt_av(AV96))
print('Min AV96=',get_min_av96(),',f',fmt_av(get_min_av96()))
print('Max AV96=',get_max_av96(),',f',fmt_av(get_max_av96()))
On some lenses I got smaller AV96 values with get_prop(props.AV) as with get_min_av96 like:
EF 24mm f/2.8 AV96=288 (what I expect) and Min AV96=300
or
EF-S 55-250mm f/4-5.6 IS STM at 55mm AV96= 384 (what I expect) and Min AV96= 396
Interesting on both lens:
Min AV96 on this lens is exactly 12 stops before the first real aperture change.
Here are complete results:
EF 24mm f/2.8 IS USM
AV96= 288 ,f 2.82
Min AV96= 300 ,f 2.95
Max AV96= 876 ,f 23.62
EF 50mm f1.8 USM
AV96= 168 ,f 1.83
Min AV96= 168 ,f 1.83
Max AV96= 864 ,f 22.62
EF-M 15-45mm f3.5-6.3 IS STM at 15mm
AV96= 348 ,f 3.51
Min AV96= 348 ,f 3.51
Max AV96= 864 ,f 22.62
EF-M 15-45mm f3.5-6.3 IS STM at 45mm
AV96= 516 ,f 6.44
Min AV96= 516 ,f 6.44
Max AV96= 1032 ,f -24.-3
EF-S 55-250mm f/4-5.6 IS STM at 55mm
AV96= 384 ,f 4.00
Min AV96= 396 ,f 4.17
Max AV96= 876 ,f 23.62
EF-S 55-250mm f/4-5.6 IS STM at 250mm
AV96= 480 ,f 5.65
Min AV96= 492 ,f 5.90
Max AV96= 972 ,f -32.-11
EF-S 10-18mm/f4.5-5.6 EF-S IS STM at 10mm
AV96= 420 ,f 4.55
Min AV96= 420 ,f 4.55
Max AV96= 864 ,f 22.62
EF-S 10-18mm/f4.5-5.6 EF-S IS STM at 18mm
AV96= 480 ,f 5.65
Min AV96= 480 ,f 5.65
Max AV96= 924 ,f 28.10
The CHDK C implementation should be correct, and should work for anything over f/1.0. We should probably support f/1.0 and below now that we have ILC cameras...
Looks like av96_to_aperture has problem with high AV96 values like 972. If I understand shooting_get_aperture_from_av96 correct, then the function returns an unsigned short and this is limited to +32.767.
972 deliver 33417.
-
Updated script in the first post.
I changed the error calculation to be based comparison with shots at the initial (widest) av value. This makes it easier to understand aperture behavior than the previous relative comparison, but is more sensitive to lighting changes over the duration of the test. The "baseline shots" setting controls how many shots are averaged for the comparison value.
I also added code to compare the current aperture value (reported by get_current_av96) with the set value. This requires trunk build 5104 or later.
New log columns are
av96_cur: value reported by get_current_av96, which should theoretically be the actual aperture value
av96_err: difference between the set value and current value = av96 - av96_cur
m96_err_cur: m96_err, if av96_cur is the actual used aperture.
a540, g7x and sx710 all sometimes show av96_err of 1 or 2.
A540 looks like it only does steps of 3 units for most of the aperture range: Setting 282, 283 and 284 all give av_cur = 282. However, this isn't totally consistent.
g7x and sx710 seem to sometimes lag behind by 1. That is set 481=481, 482=481, 483=482, 484=484.
It's not clear to me whether the cur value more accurately represents the actual exposure. On sx710, average m96_err_cur was lower, while g7x and a540 were higher.
I'm reading the av96_cur value at the remote hook, assuming the actual aperture will be set at that point.
-
Updated script in the first post.
#ui_av_end=0 "End value (0 = max)" [0 1920]
Where does the value of 1920 come from?
-
Where does the value of 1920 come from?
Arbitrary large value. I wanted to make the minimum at 0 and range syntax expects two values. If you have a lens that goes past f/1024, feel free to adjust it :haha
-
It is also very easy to determine with the script at which aperture a lens is the sharpest. In doing so, I simply created shots of a noisy picture. It is then sufficient to sort only the JPG images by size. The largest JPG is then the sharpest picture. I compared that to other methods. That's pretty good. The following plot shows the curves of a 50mm lens on the EOS M3. The first curve is the JPG size. Not sure if the Script could log the JPG size of the image.
More info
https://translate.google.de/translate?sl=de&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=https%3A%2F%2Ffotoschule.fotocommunity.de%2Ffoerderliche-blende-welche-blende-ist-meine-schaerfste%2F&edit-text=&act=url
Interestingly, the SX50 goes beyond aperture f8. The Canon Interfacce is only available until f8.
range=3.50-11.31 (347-672)
On high apartures the script delivers wrong values as I descript in Reply #27
range=6.44--24.-3 (516-1032)
-
Maybe still interesting:
I have now not only evaluated the entire image, but also sometimes parts, so 10% of the image area from the middle or on the edge. Again, almost everything is explainable. The edges at the wide angle were always significantly worse than the telephoto.
However, there is an inconsistency with the combination EOS M3 and the EF 50mm f1.8 USM (to a lesser extent also with the EF 24mm f / 2.8 IS USM). If I only look at the center, then the lens reaches a maximum at f2.8, then gets a minimum at f4.5 and then gets a new maximum at f9. At f4.5, the edges are slightly sharper than the center. It makes no difference if I focus again before each shot or just once at the beginning. It is probably a Fokusshift that arises when closing the aperture.
For the 6D and EF 50mm f1.8 USM this does not happen in LiveView. When focusing, the aperture is also set on the 6D. I have never noticed that. I always thought, it is focused with an open aperture. The break in the sharpness in the center is then no longer available. Maybe that's why the S110 does not focus on open aperture.
Presumably there is no way with CHDK to close the aperture when focusing?
-
It is also very easy to determine with the script at which aperture a lens is the sharpest. In doing so, I simply created shots of a noisy picture. It is then sufficient to sort only the JPG images by size. The largest JPG is then the sharpest picture. I compared that to other methods. That's pretty good. The following plot shows the curves of a 50mm lens on the EOS M3. The first curve is the JPG size.
Good idea.
Not sure if the Script could log the JPG size of the image.
You can get the file name:
string.format('%s/IMG_%04d.JPG',get_image_dir(),get_exp_count())
and then get the size with os.stat
Interestingly, the SX50 goes beyond aperture f8. The Canon Interfacce is only available until f8.
range=3.50-11.31 (347-672)
Yes, most powershots work to somewhat higher f numbers than the UI max, and much higher when zoomed.
On high apartures the script delivers wrong values as I descript in Reply #27
range=6.44--24.-3 (516-1032)
This should be fixed in trunk 5126. I don't think any powershots get close f/32 but maybe you could put something on the Ms that does.
For the 6D and EF 50mm f1.8 USM this does not happen in LiveView. When focusing, the aperture is also set on the 6D. I have never noticed that. I always thought, it is focused with an open aperture. The break in the sharpness in the center is then no longer available. Maybe that's why the S110 does not focus on open aperture.
I wonder if this depends on available light. With good light, then using the same aperture as the shot should be best, but at some point the SNR probably matters more?
Presumably there is no way with CHDK to close the aperture when focusing?
If setting CHDK overrides doesn't, then probably not. Hooking into the AF process could be useful, but I'm not aware of any work in that area. One thing I'd love is to be able to limit the range of the AF scan, so if you are shooting far away it can just sweep say 100m+ instead of the whole range.
-
This should be fixed in trunk 5126. I don't think any powershots get close f/32 but maybe you could put something on the Ms that does.
Thanks a lot I've just tested it works really well. Of course there will always be some f-stops on the M's but higher values do not really make sense anymore.
I wonder if this depends on available light.
It's hard to say how far that depends on the available light. The aperture on the S110 is already very small to determine that. It does not work on the real aperture but only fades something off.
It's the same with the 6d. It does not completely go to the selected aperture. It looks like the aperture is only a little bit closed, may be enough to avoid the focus shift. That seems to be so even in low light. In the case of a lot of light, it seems to dazzle much more in the live view and to be easily opened when focused.
With good light, then using the same aperture as the shot should be best, but at some point the SNR probably matters more?
From a certain aperture, the depth of field is probably so big that the camera no longer finds maximum
If setting CHDK overrides doesn't, then probably not.
In the M3, I have a button with which I can control the depth of field. When I press the button, the camera goes to working aperture. What could be behind it? Could I search for a function in the firmware?
-
Looks like my entire lens has 12 unit step (with the exception of the beginning).
I found out that on my EF 100mm 2.8 Macro USM it changed from f16 to 24 levels. May be the difference are to small.
A new video that shows how smoothly the aperture can be changed with CHDK. This time a 1: 1 macro.
-
Here is run with M3 + Sigma 24mm F1,4 DG Art.
It looks like also the Sigma Lens changes the aperture in steps of 12 (1/8Ev)
-
Here is a run with M3 + Sigma 24mm F1,4 DG Art from f1.4 until f16 in steps of 12.
I don't see anything unusual here at the opening of f1.4 like I saw it with the EFM32mm.
https://chdk.setepontos.com/index.php?topic=12758.msg142872#msg142872 (https://chdk.setepontos.com/index.php?topic=12758.msg142872#msg142872)
The flicker on this lens is slightly larger.
Overall, however, there is an increase of 1/3EV in the range up to f16.
-
But why does the aperture only change every 12 APEX96 values?
8, 16 or 32 I could understand yes ....
I think I now know why they chose the 1/8EV (12 APEX96 values) setting. I didn't know that Canon DSLR can change the aperture setting from 1/3EV to 1/2EV in the menu. Then the 1/8EV setting is a good compromise. Ok, they could have chosen 1/6EV.