Video Manual Control: ISO, APERTURE & SHUTTER SPEED CONTROL, PLEASE!!!!

[fudgey]

edit: since this is in the feature request section instead of the devel one, a disclaimer is probably in order: do not try the code below if you don't know what you're doing

Looks like these event procedures aren't registered by default on a570 (call_event_proc always returns -1).

ExpCtrlTool.* event proc table is at 0xffd1f1d4 for a570 100e, function that registers them is at 0xffd1f704. That one gets called from Capture.Create (among a whole lot of other stuff, hopefully nothing too bad).

SetAE_ShutterSpeed eventproc table is at fff345dc, but I couldn't figure out how to register that one so I called it by its a570 100e entry point directly:

Code: [Select]

buf=call_event_proc("AllocateMemory",4)
if buf > 0 then
    poke(buf,tv)
    --ret=call_event_proc("SetAE_ShutterSpeed",buf)
    ret=call_func_ptr(0xffe0b70c,buf) --SetAE_ShutterSpeed(&tv) on a570 100e
    print("AE_ShutterSpeed=" .. ret)
    call_event_proc("FreeMemory",buf)
end
and exposure changed. So far I've tried overriding to 1/256 s and 1/4 s, the latter crashed a few seconds later (ROMLOG shows assert in AviWriter.c line 606, tMovieReco task).

Haven't tried Av or ISO yet, but those eventprocs are similar to shutter speed and probably not registered either (MoveIrisToAv entry 0xffe05ba4, eventproc table fff34168 and SetAE_CdsGainValue entry 0xffe0b964 is actually in the same event proc table with SetAE_ShutterSpeed so highly likely not registered).

Also need to find current *v96 values for a starting point for adjustment (I haven't searched, propcases would be an obvious first look).

btw, http://chdk.setepontos.com/index.php/topic,1075.msg19070.html#msg19070

[fudgey]

Okay well I wrote a little Lua script VideoExp.lua to control Tv and Av during video record (or idle live view). it's here: VideoExp.lua
Please note that in it's current form it's strictly for a570is 100e only (and it tries to prevent you from running it on other cameras) because it uses hard coded entry points for the functions that set Av and Tv since I couldn't figure out how to register them using eventprocs only.

It appears that these adjustments aren't the most stable thing out there. It's probably Tv adjust that causes problems. My camera crashes every time I set Tv slower than the 30 Hz frame rate (even in 15 Hz mode), but Tv=1/32 s seems to work fine in the 60 Hz video mode. Worryingly the camera sometimes crashes after Tv adjustments in the faster range too. This doesn't mean Av adjustment would be bug free either, I haven't tested this enough to tell.

ND filter and ISO (Sv) are still missing from the script. ND is something I'm not going to be able to test at all.


Oh, I forgot to make the script test for rec mode. Running in play mode probably results in a crash, I did that at one point by accident.

edit: turns out 'prop=require "propcase"' crashes the camera during video record. Maybe other file access is a no-no during video too? So a new version (link updated) it now checks for rec mode too. It's possible that some crashes were due to file i/o instead of tv adjust...

[reyalp]

Nice.
<- should spend more time hacking, less http://xkcd.com/386/

http://chdk.wikia.com/wiki/User:ReyalP/EventProcNotes may help you find registration functions. As you mentioned earlier, some of them do unknown Other Stuff, so beware.

This is all from a540. Where I've noted something like
 RAM 0x9ED8 -> FFEFC3C8
It means that I found the event proc table in the data that gets copied to RAM. Presumably, if they are copied to RAM they some entry points could change before they are registered. The procedure for find the ram based tables is pretty simple. Find name,function pairs in the canon data segment in ROM, translate that to a RAM address (<table address> - <ROM data start> + 0x1900), then search for the resulting address as a literal value. So for example, in a570 100e, SetAE_ShutterSpeed is found in a table starting at FFF33FB8.  Data starts at FFF2D320  -> 0x8598. The only obvious code reference to this value is  sub_FFE04358. If you backtrack through the callers, you find it associated with InitializeAdjustmentFunction

Regarding the crashes, it is possible that adjusting some of these things are expected to be synchronized.

[fudgey]

Thanks reyalp, that got all three registered. New version available at http://pastebin.com/Nm5r9V7p.

This one is no longer camera model dependent, but it's possible that it will still only work (without modification) on a some cameras from the same era (FYI, the a570 is a Digic III vxworks camera from year 2008 or so). To debug you may want to print + sleep after each call_event_proc command to see if they return -1, which for this script quite likely means the event procedure was not found (doesn't mean it doesn't exist, it may just be unregistered).

I used SetCDSGain for ISO control since SetAE_CdsGainValue needed an input value to be placed in RAM (0x8ba4 for a570) with poke. Their effect seems to be the same, a range 0-ish to 1024-ish seems to cover the camera's ISO range (probably just the true ISO range, not artificial software extensions like ISO 1600 on a570, but I really don't know for sure...just a gut feeling that this number is connected to 10-bit PGA HW control).

The script is still not stable (and possibly never will be). Expect crashes every now and then. I didn't experience SD card corruption even though several video recordings got cut with battery removal.

Sv (ISO) adjust doesn't start from startup value but instead resets to zero, sorry.

[Stimpy]

Thanks Fudgey, this is a great start! - being able to lock in the ISO to a low value should improve video quality immensely. 
From what I gather, you'll need to "roll your own" CHDK to include the "native call" support, (required to run this LUA script).

I'm not overly confident in doing this, I'd like to know if anyone else has compiled CHDK with "Native Call Support" for a Canon G7?
Perhaps once we can verify this works as a LUA script on a number of camera's, it can be included within the main CHDK build?

Thanks again all for your amazing work!

[fudgey]

From what I gather, you'll need to "roll your own" CHDK to include the "native call" support, (required to run this LUA script).

I'm not overly confident in doing this, I'd like to know if anyone else has compiled CHDK with "Native Call Support" for a Canon G7?

Building CHDK with native calls enabled is very simple and doesn't really require any programming experience, especially not if you run Windows: Get http://chdk.wikia.com/wiki/CHDK-Shell and download latest trunk using it. Go to the "Compile options" menu and check OPT_LUA_CALL_NATIVE.

If you can't find it on the compile options screen (I don't run Windows so I can't check easily), you need to either edit the file trunk/buildconf.inc by removing '#' from start of the line that says #OPT_LUA_CALL_NATIVE=1 or create a file called localbuildconf.inc to the same folder that has a single line that reads
OPT_LUA_CALL_NATIVE=1
in it.

Then you just compile (you'll probably want to select your camera version first, maybe some other stuff too that I don't know about -- see CHDK shell documentation).


The danger of having native calls enabled lies pretty much entirely in that using native calls someone could write a malicious lua script for you and destroy your camera by fooling you into running their script. I assure you my script wasn't written to intentionally destroy your camera... In principle we could have a CHDK config option (e.g. in debug menu) to enable native calls, but the problem is that a malicious Lua script could modify your config using normal file I/O functions. Note that someone could quite as well create a malicious CHDK build that does the same and fool you into installing it. Scripts are just a tad easier to try out than builds.

[Stimpy]

Thanks for the detailed guide Fudgey! I'll give it a try over the weekend & see how I go. The G7 has very similar specs to the A570 so I would expect similar results.

[Stimpy]

I can confirm that this LUA script works perfectly on a Powershot G7!
Compiling a custom CHDK with the shell script was a piece of cake. In the tests I've done to date, I'm leaving the shutter, (Tv), value at the default of 480, (which I presume is 1/60th sec). I'm setting the ISO, (Sv), to 128. I'm seeing virtually zero noise in the dark areas of the image, which is exactly what I hoped to achieve.

A question, in which mode is the Canon MJPEG codec operating, 4:1:1, 4:2:0 or 4:2:2 color space?

Update: I'm now able to record at Quality=99, (around 47.5 mbit/sec or 6MB/Sec), whereas before the camera would crash at these data rates. It's possible that by lowering the ISO value, less noise is now being recorded in the video clip, therefore the video compression is working more efficiently.

[Stimpy]

Tested on a Canon IXUS 100IS or SD780, (DIGIC 4 processor).
- Camera crashes soon after loading the LUA script.
- It seems the "native calls" need to be updated to work on DIGIC 4 based cameras 

Fudgey - it appears you were spot on in your estimate that this would only work on DIGIC3 series camera's!

Unfortunately, Canon didn't make the switch to HD movie recording on the PowerShots until after the DIGIC4 chip was released, (2008), so we're stuck with 640x480 until we have a break through.

As ISO control has such a huge impact on video quality, it'd be great to see this incorporated in the main CHDK build...?

[reyalp]

Tested on a Canon IXUS 100IS or SD780, (DIGIC 4 processor).
- It seems the "native calls" need to be updated to work on DIGIC 4 based cameras  

Native calls work fine on digic 4. The particular functions used in the script may have changed.

s ISO control has such a huge impact on video quality, it'd be great to see this incorporated in the main CHDK build...?

Agreed.  That requires someone to do the grunt work to make it work reliably, or figure out which cameras it works on. fudgeys script is a good place to start experimenting.

This one is no longer camera model dependent, but it's possible that it will still only work (without modification) on a some cameras from the same era (FYI, the a570 is a Digic III vxworks camera from year 2008 or so). To debug you may want to print + sleep after each call_event_proc command to see if they return -1, which for this script quite likely means the event procedure was not found (doesn't mean it doesn't exist, it may just be unregistered).

Good advice. Even if the functions are registered, it may tell you which one is crashing.

You can also try getting a ROMLOG http://chdk.wikia.com/wiki/LUA/Scripts:Standard/Test/Romlog

[Stimpy]

Although the CHDK does run, I'm getting errors when building CHDK for an IXUS 100IS, (SD780) - which may be effecting my testing of the LUA script...? Here's the error report;

boot.c: In function 'taskCreateHook':
boot.c:36: warning: comparison between pointer and integer
boot.c:43: warning: assignment makes integer from pointer without a cast
boot.c: In function 'dumpCF90_SD7802':
boot.c:144: warning: implicit declaration of function 'Fopen_Fut'
boot.c:145: warning: implicit declaration of function 'Fwrite_Fut'
boot.c:146: warning: implicit declaration of function 'Fflush_Fut'
boot.c:147: warning: implicit declaration of function 'Fclose_Fut'
boot.c: In function 'task_blinker':
boot.c:191: warning: implicit declaration of function 'draw_txt_string'
wrappers.c: In function 'OpLog_Get':
wrappers.c:23: warning: implicit declaration of function '_OpLog_Get'
kbd.c: In function 'dumpProps':
kbd.c:216: warning: implicit declaration of function 'Fopen_Fut'
kbd.c:219: warning: implicit declaration of function 'sprintf'
kbd.c:219: warning: incompatible implicit declaration of built-in function 'sprintf'
kbd.c:220: warning: implicit declaration of function 'Fwrite_Fut'
kbd.c:220: warning: implicit declaration of function 'strlen'
kbd.c:220: warning: incompatible implicit declaration of built-in function 'strlen'
kbd.c:222: warning: implicit declaration of function 'Fflush_Fut'
kbd.c:223: warning: implicit declaration of function 'Fclose_Fut'
kbd.c: In function 'dump1900_SD780':
kbd.c:258: warning: incompatible implicit declaration of built-in function 'sprintf'
kbd.c:262: warning: incompatible implicit declaration of built-in function 'sprintf'
kbd.c:267: warning: incompatible implicit declaration of built-in function 'sprintf'
kbd.c: In function 'my_kbd_read_keys':
kbd.c:377: warning: implicit declaration of function 'draw_txt_string'

Note, when building CHDK for my G7, I don't get any errors at all, so I'm confident that this is a camera specific error. 
 

[reyalp]

These are warnings, not errors. Most of them look relatively harmless, probably from camera specific debugging / development code.

[Microfunguy]

Just delete functions 'dumpCF90_SD7802',  'task_blinker', 'dumpProps',  'dump1900_SD780'.

In boot.c, aHookList[] is declared as an array of 'long' but 'p' is a pointer to an integer value.

[Stimpy]

I've decided to ignore the compiler warnings & proceed, as it doesn't appear to effect the CHDK from functioning normally.
So we know the VidExp.lua script crashes the Digic4 based IXUS 100is, but at what point?

I broke the script down into various sections, remarking or deleting out lines of code until I could isolate the problem.
All of the initialization checks to see if the Camera supported native calls, ExpCtrlTool & ND tests all work fine.

However, the following lines of script would always crash the camera;

Line 95. if (call_event_proc("InitializeAdjustmentFunction") == -1) then
Line 96.    error("InitializeAdjustmentFunction failed")

Line 130. ret=call_event_proc("SetAE_ShutterSpeed",tv96)

Line 154. ret=call_event_proc("MoveIrisToAv",av96)

Line 177. ret=call_event_proc("SetCDSGain",sv96)

Are we to assume these particular event procedures have yet to be "enabled" or "mapped out" on the newer Digic4 cameras?
How to move forward? - Is there something a non-dev, such as myself, could investigate?

[fudgey]

Line 95. if (call_event_proc("InitializeAdjustmentFunction") == -1) then
Line 96.    error("InitializeAdjustmentFunction failed")

Line 130. ret=call_event_proc("SetAE_ShutterSpeed",tv96)

Line 154. ret=call_event_proc("MoveIrisToAv",av96)

Line 177. ret=call_event_proc("SetCDSGain",sv96)

Are we to assume these particular event procedures have yet to be "enabled" or "mapped out" on the newer Digic4 cameras?
How to move forward? - Is there something a non-dev, such as myself, could investigate?

Function entry points for ixus100_sd780 sub 100c are

SetAE_ShutterSpeed 0xffa96f60
MoveIrisToAv 0xffa92248
SetCDSGain 0xffa91e6c
InitializeAdjustmentFunction 0xff91d4fc

I suppose the problem is that InitializeAdjustmentFunction isn't registered. The other three funcs don't work because they are supposedly registered by InitializeAdjustmentFunction whose call failed.

It's name+entry can be found at a table that starts at 0xffb5581c but I don't know who registers it. But until someone finds out, if you're feeling brave you could start by replacing

Code: [Select]

if (call_event_proc("InitializeAdjustmentFunction") == -1) then
    error("InitializeAdjustmentFunction failed")
end
with something like

Code: [Select]

bi=get_buildinfo()
print("platform: ",bi.platform," ",bi.platsub)
if bi.platform~="ixus100_sd780" then
  error("wrong camera model")
elseif bi.platsub~="100c" then
  error("wrong firmware version")
end

ret=call_func_ptr(0xff91d4fc)
print("InitAdjFunc=",ret)
That effectively calls InitializeAdjustmentFunction directly (not as an event procedure), which should be just as good unless you wish to use the same script on other cameras. That buildinfo stuff should prevent accidentally running this on incorrect camera models.