Manual Shutter Control, Electronic vs Mechanical Shutter

jmac698

138

« on: 27 / July / 2016, 05:53:36 »

Advertisements

Hello,
I want to exactly control shutter speed/exposure time, but not in Tv values but by a timer. For an example I've studied fixedint by @reyalp

Note, there's really two shutters, an electronic shutter and a mechanical shutter. They both work, however the purpose of the mechanical shutter is to avoid a rolling shutter effect on still pictures, and also to make the picture look smoother somehow?

Also, this all applies to CMOS sensors.

As far as I can follow/guess, the sequence goes like this:

Code: [Select]

hook_shoot.set(10000)
hook_raw.set(10000)

-Enables the shoot and raw hooks. Hook here means 'upon using the hook_xxx command, pause the automatic shooting sequence and run script, until hook.continue()'. The 'shoot hook' pauses just before the shutter opens, the 'raw hook' pauses just before saving the file.

Code: [Select]

set_tv96_direct(tv)-This sets the exposure time

Code: [Select]

press('shoot_half')
repeat sleep(10) until get_shooting() == true

-This waits for the cam to set shooting parameters (exposure and focus). Since Tv was already set, I assume this only works if the shooting mode is 'P'. Otherwise the Tv could be overidden. So the set_tv line is a bit confusing at first unless you know the context. The mechanical shutter is open, and I assume the camera takes a pic with the electronic shutter, just to read out the histogram or scene recognition to set the shooting parameters.

Code: [Select]

if cont then
    press'shoot_full_only'
end

-If you want to use continuous mode, you hold down the shoot button here

Code: [Select]

for shot=1,ui_shots do-Start a loop to take a number of shots. Now what follows is the important part:

Code: [Select]

if not cont then
        press('shoot_full_only')
    end

-Press the shoot button (assuming it's not in continuous mode)
Close the mechanical shutter then complete and save the shot. Instead, we have:

Code: [Select]

hook_shoot.wait_ready()-Here we wait until just before the shutter opens *I'm going to say 'before the electronic shutter opens' which means before the pixels are reset (discharged to 0v) in preparation for accumulating charge from light

Code: [Select]

if ui_darks then
        close_shutter()
    end

-If you want to shoot a dark frame, close the mechanical shutter at this point. This only makes sense if it means to close the mechanical shutter, before opening the electronic shutter. It also implies that normally, the mechanical shutter is open at this point, which then implies that the sensor is reset globally and immediately, so there's no rolling shutter problem here.

Code: [Select]

if not cont then
        release('shoot_full_only')
    end

-Start to end the shot. Normally the exposure would finish and save automatically at this point, however:

Code: [Select]

sleep(interval - et)-Manually wait as part of the Tv. This delay would be in addition to the Tv value set before the shot.

Code: [Select]

hook_shoot.continue()-Finish the shot. I assume the mechanical shutter will be closed here (it may already be closed, if there was a dark frame, which means it tries to close an already closed shutter, which apparently doesn't cause any mechanical problems), then close the electronic shutter (which means to read the current charge serially by row of the pixels, which are still exposing until the moment they are read, which is the cause of the rolling shutter effect).

Code: [Select]

hook_raw.wait_ready()
    set_tv96_direct(tv)
    hook_raw.continue()

-Stop just before saving the pic, set the Tv, then finish saving the pic. I believe the purpose of this is because of the sequence of the loop; to prepare the Tv for the *next* shot (the first first shot's Tv was set before the loop). However if that's the case, I don't see why set_tv isn't at the very end.

Code: [Select]

end-Continue the loop

Logged

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #1 on: 27 / July / 2016, 07:00:06 »

Yep I didn't ask a question yet. But let me summarize even more what I think happens.

The Shooting Sequence

(for CMOS sensors)

Code: [Select]

hook_shoot.set(10000)-you need this to enable the shooting hook, to have control over exposure time

--- only if setting Tv each shot, such as HDR in continuous mode ---

Code: [Select]

hook_raw.set(10000)-you need this to enable the raw (saving) hook, it seems the only purpose is to set the Tv for the next shot (so this is optional)

--- optional, set whatever parameters you want here ---

Code: [Select]

set_tv96_direct(tv)-set exposure time for first shot

Code: [Select]

press('shoot_half')
repeat sleep(10) until get_shooting() == true

-let the camera set other values, based on Tv time (must be in 'P' mode)
-The mecha shutter is open, a pic is taken with electronic shutter, then the pic is analyzed to adjust exposure

Code: [Select]

for shot=1,ui_shots do-start your shooting loop

Code: [Select]

press('shoot_full_only') --Press the shoot button. Doesn't start accumulating light yet, due to shoot hook enabled.

--- only for dark frames ---

Code: [Select]

hook_shoot.wait_ready() -wait until just before the electronic shutter opens. You only need this for dark frames.

Code: [Select]

if dark_frame then close_shutter() -a routine to close the mechanical shutter, if dark frames, and before the electronic shutter opens
--- end of code only for dark frames ---

Code: [Select]

release('shoot_full_only') -Not sure why this has to go here, it seems you could release the button at any convenient time?
-Electronically reset the pixels and start accumulating light, wait for Tv

Code: [Select]

sleep(interval - et) -add in additional Tv here

Code: [Select]

hook_shoot.continue() -close the mecha shutter, then start the pixel readout (close the electronic shutter)

--- optional, only if you set Tv for next shot ---

Code: [Select]

  hook_raw.wait_ready()
  set_tv96_direct(tv)
  hook_raw.continue()

-set the Tv for the next shot here (optional), seems only necessary to be able to interrupt continuous mode
-Save the picture to the card, then open the mecha shutter again(?)
---end of optional code ---

end
-continue the loop

Questions:
-Do I need hook_shoot.wait_ready() if I'm not taking a dark frame?
-Does the mecha shutter have to be closed in the hook? I have my own script where it seems I can close the shutter before press("shoot_full")
-Is set_tv only in the raw hook, because that's the only way to change it during continuous mode?
-Does release('shoot_full_only') actually open the electronic shutter, even though it doesn't seem to make sense?
-Does there seem any way to do a double exposure? Being able to open/close the mecha shutter while the electronic shutter is open would be great, or flashing twice while shutter is open.
-After

Code: [Select]

release('shoot_full_only')you would run the flash twice (or once, if flash was already enabled?), or open close the shutter, then

Code: [Select]

hook_shoot.continue()

« Last Edit: 27 / July / 2016, 07:07:51 by jmac698 »

Logged

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #2 on: 27 / July / 2016, 13:25:20 »

Ok,
I'm going to answer my own question here, you can open/close the shutter at any time apparently, see http://chdk.wikia.com/wiki/Meteor_Intervalometer_with_Dark_Frame_Management for the routines.

There was some work on controlling the flash if you search. You have to charge it until it's ready, stop charging, then fire it.

The other questions still apply, as I can't be sure just from reading scripts.

Logged

reyalp

14118

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #3 on: 27 / July / 2016, 13:59:32 »

This is a good description, but a couple of points need clarification

Quote from: jmac698 on 27 / July / 2016, 05:53:36

Code: [Select]
hook_shoot.wait_ready()-Here we wait until just before the shutter opens *I'm going to say 'before the electronic shutter opens' which means before the pixels are reset (discharged to 0v) in preparation for accumulating charge from light

On any correctly implemented port, it will happen before the exposure starts, but the exact point where this happens in the shooting sequence is not really well defined, and might vary between ports. I would generally expect it to happen a few tens of milliseconds before the actual exposure start.

Quote

Code: [Select]
if ui_darks then close_shutter() end-If you want to shoot a dark frame, close the mechanical shutter at this point. This only makes sense if it means to close the mechanical shutter, before opening the electronic shutter. It also implies that normally, the mechanical shutter is open at this point, which then implies that the sensor is reset globally and immediately, so there's no rolling shutter problem here.

Right, from what I've observed, the mechanical shutter is normally open, it only closes and re-opens for readout. For dark frames, the script closes it early (before the shot starts) and lets the camera re-open it normally after.

Quote

-Start to end the shot. Normally the exposure would finish and save automatically at this point, however:
Code: [Select]
sleep(interval - et)-Manually wait as part of the Tv. This delay would be in addition to the Tv value set before the shot.

This is not correct. The actual exposure duration (for both mechanical and electronic shutter) is controlled entirely by the tv value. This wait exists only to maintain the specified shooting interval.

Quote

Code: [Select]
hook_shoot.continue()-Finish the shot.

No, this allows the shot to *begin*. The hook_shoot blocks execution of the shooting process at a point before the exposure starts.

Quote

Code: [Select]
hook_raw.wait_ready() set_tv96_direct(tv) hook_raw.continue()-Stop just before saving the pic, set the Tv, then finish saving the pic. I believe the purpose of this is because of the sequence of the loop; to prepare the Tv for the *next* shot (the first first shot's Tv was set before the loop). However if that's the case, I don't see why set_tv isn't at the very end.

You are correct this is to set the tv for the next shot (the initial tv is set outside the loop)
The reason it's done here is that it is at a relatively well defined point in the shooting process, which has been observed to work correctly on many cameras. If it was set outside of hook_raw, it wouldn't happen at a predictable point in the shooting process, because when shooting is not blocked in one of the hooks, the tasks that control shooting run independent of kbd_task where the script runs.

Logged

Don't forget what the H stands for.

reyalp

14118

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #4 on: 27 / July / 2016, 14:21:01 »

Quote from: jmac698 on 27 / July / 2016, 07:00:06

Code: [Select]
hook_shoot.set(10000)-you need this to enable the shooting hook, to have control over exposure time

No, you use this for either:
1) Control shooting rate in continuous mode. Normally, in continuous mode the camera just shoots as fast as it can. If you want a fixed interval, you need to be able to make it wait somewhere.
2) Control exposure start time with high precision. Normally, there is a substantial delay between pressing shoot_full_only and actual exposure start. hook_shoot waits after most of that delay is done.

Quote

-you need this to enable the raw (saving) hook, it seems the only purpose is to set the Tv for the next shot (so this is optional)

Right, it's mostly just a convenient place to put code that you want to run after you know the exposure is finished, before the next shot starts.

Quote

Code: [Select]
press('shoot_half') repeat sleep(10) until get_shooting() == true-let the camera set other values, based on Tv time (must be in 'P' mode)

More generally, let the camera do all it's normal pre-shoot setup.

Quote

-The mecha shutter is open, a pic is taken with electronic shutter, then the pic is analyzed to adjust exposure

The data presumably comes from something like a live view frame, which are being taken continuously up to shortly before the full frame shot starts.

Quote

Code: [Select]
press('shoot_full_only') --Press the shoot button. Doesn't start accumulating light yet, due to shoot hook enabled.

Correct.

Quote

--- only for dark frames ---
Code: [Select]
hook_shoot.wait_ready() -wait until just before the electronic shutter opens. You only need this for dark frames.

Not really, see earlier comment. For dark frames, you could do it earlier after get_shooting()

Quote

Code: [Select]
release('shoot_full_only') -Not sure why this has to go here, it seems you could release the button at any convenient time?

It's here because it is a convenient time

Specifically, once hook_shoot has been hit, you know for sure the button press has been recognized and the shot has been irreversibly started. If you click shoot full very quickly at the wrong point in the shooting process, it might not be detected by the firmware.

Quote

-Electronically reset the pixels and start accumulating light, wait for Tv

Code: [Select]
sleep(interval - et) -add in additional Tv here

Code: [Select]
hook_shoot.continue() -close the mecha shutter, then start the pixel readout (close the electronic shutter)

See earlier comment, the exposure doesn't *start* until after this point.

Quote

Questions:
-Do I need hook_shoot.wait_ready() if I'm not taking a dark frame?

I think this is explained above, let me know if it's not clear.

Quote

-Does the mecha shutter have to be closed in the hook? I have my own script where it seems I can close the shutter before press("shoot_full")

If you were doing single shots, you could close it after get_shooting() == true. If you are using continuous mode or holding down half shoot, it would be hard to find another well-defined point to do it. You *might* be able to do it in the raw hook instead.

Quote

-Is set_tv only in the raw hook, because that's the only way to change it during continuous mode?

I think the earlier comment covers this. It's a well defined point that seems work, for both continuous mode or holding half shoot.

Quote

-Does release('shoot_full_only') actually open the electronic shutter, even though it doesn't seem to make sense?

No, it doesn't. You could release it later (e.g. in the raw hook). Shutter opening/closing is controlled by canon firmware code based on the set tv value only.

Quote

-Does there seem any way to do a double exposure? Being able to open/close the mecha shutter while the electronic shutter is open would be great, or flashing twice while shutter is open.

You should be able to open / close the mecha shutter during a long exposure. However, be aware that script timing has only 10ms precision at best, so this would only be viable for quite long exposures. You would also have to guess a bit for the timing: the exposure starts some arbitrary (and probably variable) amount of time (on the order of tens of ms) after hook_shoot.continue(), and ends some time after that based on your tv value.

Logged

Don't forget what the H stands for.

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #5 on: 07 / August / 2016, 12:28:22 »

@reyalp

Thanks so much for a complete and clear answer. When I can get to this, I'd like to test out what I've learned and then write an article on the wiki, plus take some cool double exposure pics.

A few other investigations come to mind, like measuring the speed of script, calibrating Tv against tick(), and testing a script against two models of different digic (hopefully a bit of coverage), testing the effectiveness of the mecha against bright light.

Logged

reyalp

14118

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #6 on: 07 / August / 2016, 13:48:10 »

Quote from: jmac698 on 07 / August / 2016, 12:28:22

Thanks so much for a complete and clear answer. When I can get to this, I'd like to test out what I've learned and then write an article on the wiki, plus take some cool double exposure pics.

This would be very welcome

Quote

A few other investigations come to mind, like measuring the speed of script,

If you mean execution speed, some very basic measurements at: https://chdk.setepontos.com/index.php?topic=12451.msg123416#msg123416 and https://chdk.setepontos.com/index.php?topic=6729.msg71456#msg71456

Quote

calibrating Tv against tick(),

get_tick_count() is only 10ms precision at best, so it's pretty useless for normal hand held exposure times. It is possible to access higher precision counter values, see https://chdk.setepontos.com/index.php?topic=11342.20
I would still like to expose this in a more usable way someday...

It's also important to note that the actual exposure control occurs outside of the tasks where CHDK code runs, likely involving multiple Canon tasks, interrupts and high precision timers.

Logged

Don't forget what the H stands for.

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #7 on: 08 / August / 2016, 11:49:12 »

Calibrating Tv against get_tick_count()

Code: [Select]

--[[
@chdk_version 1.4.0
--]]
--test Tv timing

function init()
	require'hookutil'--import hook utils
	hook_shoot.set(10000)--enable hooks
	hook_raw.set(10000)
end

function time_shot(tv_direct,dark_sub)
	set_raw_nr(dark_sub)--enable or disable dark frame subtraction, which doubles time over 2sec
	set_tv96_direct(tv_direct)--set the time as a tv direct value, where 0=1sec
	press('shoot_half')--do preshot and wait until shooting settings are ready
	repeat sleep(10) until get_shooting() == true

	press('shoot_full_only')
	hook_shoot.wait_ready()
	release('shoot_full_only')--convenient place to put this
	t0=get_tick_count()--start timer just before shot
	hook_shoot.continue()--exposure starts here
	hook_raw.wait_ready()
	t1=get_tick_count()
	hook_raw.continue()
	return t0,t1
end

--attempt to find shutter time relative to ticks, and overhead
off=1
on=2
--set two different times here, which vary by 
--orders of magnitude more than 10 milliseconds
tv_sec_1=16--short shot time in seconds
tv_sec_2=32--long shot time in seconds
tv96_1=usec_to_tv96(tv_sec_1*1000000)
tv96_2=usec_to_tv96(tv_sec_2*1000000)

init()--prepare hooks
measured_start_1,measured_end_1=time_shot(0,off)--dummy shot due to weird bug
measured_start_1,measured_end_1=time_shot(tv96_1,off)--short shot
measured_start_3,measured_end_3=time_shot(tv96_1,on)--shoot short with dark sub
measured_start_2,measured_end_2=time_shot(tv96_2,off)--long shot

print("Measured timings")
print(measured_start_1,measured_end_1)
print(measured_start_2,measured_end_2)
print(measured_start_3,measured_end_3)

tick time vs Tv time
sd1200 16010/16000
sx50 16020/16000

overhead
sd1200 680
sx50 210

overhead with dark frame (about twice overhead)
sd1200 1380 (vs 1360=680*2)
sx50 940 (vs 420=210*2)

time between shots (includes half shoot, saving time)
sd1200 380,370
sx50 (raw+jpg) 2190,2190

time per shot:
sd1200 (Tv*1000+10)+680+380
sx50 (Tv*1000+20)+210+2190 (raw+jpg)

Does this mean the histogram is much faster with the new cpu?

Logged

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #8 on: 06 / September / 2016, 05:18:16 »

I poured over your answers and tried to write out a doc a bit clearer.

The Shooting Sequence

(for CMOS sensors)

hook_shoot.set(10000)
-you need this to enable the shooting hook, to have control over start time (potentially),
and also to synchronize the script and shooting task which run in parallel.
Hook here means 'upon using the hook_xxx command, pause the automatic shooting sequence
and run script, until hook.continue()'. The 'shoot hook' pauses just before the shutter
opens, the 'raw hook' pauses just before saving the file.

hook_raw.set(10000)
-you need this to enable the raw (saving) hook, the only purpose is to set the Tv for the next shot (so this is optional). You would use this for HDR in continuos mode for example.

set_tv96_direct(tv)
-set exposure time for first shot. Assumes you are in P mode, because otherwise the
next step could override this value, as it sets shooting parameters.

press('shoot_half')
repeat sleep(10) until get_shooting() == true
-let the camera set other values, based on Tv time (must be in 'P' mode)
-The mecha shutter is open, a pic is taken with electronic shutter, then the pic is
analyzed to adjust exposure
-the script task also comes into sync with the shooting task for this moment

for shot=1,ui_shots do
-start your shooting loop

press('shoot_full_only')
--Press the shoot button, holding down this button is for continuous mode.

hook_shoot.wait_ready()
-wait until just before the electronic shutter opens. The use this has, is to
synchronize the start of the shot to some exact time (like a remote release,
or certain time).

if dark_frame then close_shutter()
-a routine to close the mechanical shutter, if dark frames, and before the electronic
shutter opens. This should be done here, if using continuous mode, to ensure
it gets closed at the right moment.

release('shoot_full_only')
-a convenient place to ensure this button press gets registered.

hook_shoot.continue()
-start exposing, which means resetting the sensor then allowing charge to accumulate, from photons hitting the transistors. The reset is global and immediate, so the rolling shutter is not the issue at this point, but during the readout.
-wait the Tv value
-close the mecha shutter, then start the slower (and rolling) sensor readout. In video mode, this is where the rolling shutter problem comes, because the mecha shutter is always open in that case.
-do some after shot tasks like building a histogram, and converting to jpg

hook_raw.wait_ready()
-stop just before saving the file(s)

set_tv96_direct(tv)
-set the Tv for the next shot (if different, for example for an HDR effect in
continuous mode)
-it's put here, where the shooting process is stopped and only the script process is
running, to ensure it gets executed before the next shot

hook_raw.continue()
-finish saving the shot
end
-continue the loop

« Last Edit: 06 / September / 2016, 05:38:39 by jmac698 »

Logged

jmac698

138

Re: Manual Shutter Control, Electronic vs Mechanical Shutter

« Reply #9 on: 06 / September / 2016, 05:59:37 »

I tried to do double exposures on one of my cams, but wasn't able to get it working. I believe the reason is tied to problems with rawhooks on that port, and it may work on another camera. Also the theory of continuous mode operations and hooks above is probably correct, as the simple script I made based on it does work on other cameras.

« Last Edit: 18 / September / 2016, 14:07:06 by jmac698 »

Logged