The bigger difficulty is where things run: Lua code always runs in the kbd task at a specific point. The gui runs in spytask, and raw processing runs in the capt_seq task. To safely interface lua with the other tasks requires additional effort.
Where do I find more info about this? I see lots of functions (most of which I can grasp), but as to how everything works is quite a mystery to me thus far.
In the code of course, probably with some experimentation in addition to reading
IMO, this is the only way to really understand it.
Different tasks are different threads?
Tasks in the camera OSes is similar threads/processes in a desktop OS. Since the cameras have a flat, unprotected memory model (think DOS real mode, but with preemptive multitasking), the distinction is marginal.
A brief overview of how it works in CHDK
The camera has many native tasks. Most of them sit in loop either polling something and sleeping or waiting on some kind of synchronized object like a message queue.
CHDK modifies the OS startup, so it can replace some of these tasks with it's own modified tasks, and also create entirely new tasks. This happens starting in your cameras platform/<camera>/sub/<version>/boot.c (technically it starts in loader/<camera>/entry.S, but you probably don't have to worry about that), and for vxworks cameras, platform/generic/main.c
For what we are discussing, the the following are are the main areas of interest:
kbd_task: (aka PhySw)
This is a canon task that polls the MMIO associated with various buttons and switches etc, and updates a global variable (physw_status) from them, along with a few other things. It sits in a tight loop, polling the MMIOs at a 10ms interval. CHDK modifies this task, to intercept and possibly override the values. All CHDK script also runs in this task. See your platform kbd.c (handles hooking / overriding the hardware values, uses generic/kbd.c on some cameras) and core/kbd.c (handles alt mode input, runs scripts)
spy_task:
This is an entirely new task created by CHDK. It handles (almost) all the ui and most non-script processing. This runs in a loop, sleeping for 20ms and updating the gui every 3rd iteration (see core/main.c). It also handles raw saving.
capt_seq_task
This is a canon task (modified by CHDK) that sits in a loop waiting waiting on messages. The messages relate to various actions in the shooting process. CHDK hooks various points here to handle overrides, raw saving (in cooperation with spy_task), bracketing etc.
It should be noted that CHDK plays fast and loose with inter-task synchronization and communication. Rather than doing this the proper way with semaphores etc, it does various dangerous things with global variables.
How do you adjust the yields? C yields also? This have to do with tasks?
Lua yielding is done in core/luascript.c, via the lua_sethook function. You could quite easily make it so the yield interval was adjustable from lua. Essentially, set the hook to run at the maximum rate you'd ever want, and then inside the hook check a variable which can be set from lua.
The hook does a lua yield, meaning the script is left in a state where it can be resumed later from C code. This only works if the lua state isn't tangled up in the C stack, which is why it can only happen when not inside a C function that was itself called from lua (thus, the yield hook does not run inside pcall or other C functions called from lua. Various other functions that yield, such as wait_click suffer the same limitation) Yielding from lua returns control to the main kbd.task, which will eventually reach the end of it's iteration and yield with it's normal 10ms sleep.
What I want to do is this: Wait for a button press, run a bit of code for some buttons, pass the button press through for others - depending on which mode is selected on the dial (auto, P, etc) and if I'm in a menu of some sort what code is ran and what's just a button press (passed though) will change. This little bit doesn't sound that hard to implement by a script in alt mode, but I'm guessing modifying kbd.c is where I should do it.
The mode dial position is current something CHDK exposes. We can get the actual shooting mode, but if you override it, you can't recover the original dial position. On cameras with a physical dial, the values probably appears somewhere in physw_status, but because Canon has so many variations in their interface (some cameras have a slider, some modes are selected in menus etc), it's not practical for CHDK to expose it in a general way.
CHDK does not interact with the Canon menus at all, aside from trying to detect if the func or main menu is open.
Script currently takes over input almost completely, meaning that you can't really have a script that sits in the background intercepting specific key presses but leaving the rest of the Canon UI functioning normally. You could pass keys on using scripted key presses, but I suspect this would end up being pretty hokey without some additional work in the C code.