Lua Scripting Integration

[Velo]

I was thinking of not putting anything but scripts to the chdk/scripts folder. If (when) the path is hardcoded to chdk/scripts, would it be possible to load libs from e.g. chdk/lib? I think I tried
require "../lib/libname"
but no avail.

Hmm. Haven't tried it. But my guess would be that it should work. Seems like its unfortunately not the case.

[fudgey]

I would prefer the scripts folder to only have scripts and user created subfolders, because everything in chdk/scripts will result in more keypresses required in script selection.

I also think there should be a standard default absolute search path for the scripts (like chdk/lualib), so that a script wouldn't stop working if I move it around a bit. Otherwise (to use a library in several scripts) I'd need to either make multiple copies of libraries or edit scripts each time I move them.

But you all knew this already, keep up the good work.

[Jucifer]

What if libs were first searched from one location (chdk/lualib), and then if not found, from another (chdk/scripts)? Or the other way round...

Is this correct?

Code: C

1. static int loader_Lua (lua_State *L) {
2.   const char *filename;
3.   const char *name = luaL_checkstring(L, 1);
4.   lua_pushstring(L, "A/CHDK/LUALIB/");
5.   lua_pushvalue(L, 1);
6.   lua_pushstring(L, ".LUA");
7.   lua_concat(L, 3);
8.   filename = lua_tostring(L, -1);
9.   if (!readable(filename)) {
10.     lua_pushstring(L, "A/CHDK/SCRIPTS/");
11.     lua_pushvalue(L, 1);
12.     lua_pushstring(L, ".LUA");
13.     lua_concat(L, 3);
14.     filename = lua_tostring(L, -1);
15.     }
16.   if (!readable(filename))
17.     return 0;
18.   if (luaL_loadfile(L, filename) != 0)
19.     loaderror(L, filename);
20.   return 1;  /* library loaded successfully */
21. }
22.  

At least it works. :]

[fudgey]

What if libs were first searched from one location (chdk/lualib), and then if not found, from another (chdk/scripts)? Or the other way round...

Probably better the other way around i.e. a local copy in chdk/scripts would override the system default library in chdk/lualib. Altough...I can already foresee multiple search paths causing apparent user errors such as

Q: "I installed your new version of the yaddayadda Lua library but it didn't fetch my morning paper nor did it brew coffee like you promised it would!"
A: "Do you have old copies hanging around somewhere?"
Q: "No, I don't."
A: "Are you sure?"
Q: "Ooooooohhhh sorry never mind..."

But I can live with either. 

[Jucifer]

Ok, now they're the other way around. Building. I'll update my evil build tonight.

[fe50]

Just for information to all the people want's to do lua scripting:

I've started a wikia page for lua, and - surprise- you'll find it by name LUA 

You're all welcome to participate !

[reyalp]

On my camera (a540, using my own build of Jucifers branch), the script console disappears right away if there is a compile error. Since this happens before the script runs, turning on print screen won't help.

A quick workaround in lua is to do something like this:

Code: [Select]

function saferun(code)
local status
local f,msg=loadstring(code,"code")
if not f then
print("compile err")
print(msg)
sleep(5000)
else
print("compile ok")
sleep(2000)
status,msg = pcall(f)
if not status then
print("run err")
print(msg)
sleep(5000)
end
end
end
-- now put all your code in a big string
saferun([[
all
my
lua code
]])
You can of course stick the saferun function in a module.

The sleeps keep the console up long enough to see what happened.

That said, I think I'll buckle down and write a lua test harness that works on the PC, because pulling the card, rebooting the camera and trying to read the error off the tiny screen gets old real quick

I also got the lua string library working, but it's a bit messy with the required bits of libc hacked in here and there.

[fe50]

Hi reyalp,
there's the uBasic script debugger by zeno, written in Java, posted here - perhaps parts from this may be helpful for you...

[zeno]

My debugger doesn't support Lua. It probably could, but from a quick look at the Lua code, it  doesn't look easy.

[fe50]

Hi zeno,
thx, i gave this hint as a little inspiration to reyalp, perhaps he can reuse some code parts like the GUI or other things to save some time in his project...

Your debugger is a nice tool, very helpful with big basic scripts !

[fudgey]

What's the current status of execution speed of Lua? I like to know how fast or slow my scripts will run but currently I can only guess. For ubasic it's "all non-whitespace lines take 10 ms except for consecutive rem lines, who only take 10 ms to run per block of up to 100 lines", and something more complex for Lua, obviously.

From above I can see sleep() makes Lua wait for the next tick that loops may create automatic "sleeps" to prevent lockups (But when? At each iteration?). I somehow remember someone saying that some commands related to camera functionality do this as well (like shoot(), obviously) but couldn't find on a quick search.

This information should go to the wiki of course.

Another thing that should be documented is how large numbers we can use. I assume it's +-2^15 or +-2^31, one short from one (negative?) end (this one we're missing in our ubasic documentation as well).

[fe50]

...
Another thing that should be documented is how large numbers we can use. I assume it's +-2^15 or +-2^31, one short from one (negative?) end (this one we're missing in our ubasic documentation as well).

From the wikia, uBASIC_syntax#uBASIC_Syntax_short_manual:
Variables: lower latin letter from a to z. 32 bit signed integer (-2147483648 to +2147483647).

I wrote this with the informations from reyalp:

It should be a 32 bit int, +2147483647 -2147483648

Ubasic does use shorts in some places, but not for variables as far as I can tell.

[reyalp]

From above I can see sleep() makes Lua wait for the next tick that loops may create automatic "sleeps" to prevent lockups (But when? At each iteration?). I somehow remember someone saying that some commands related to camera functionality do this as well (like shoot(), obviously) but couldn't find on a quick search.

in ubasic, labels are treated similarly to rem lines.

It looks to me like the following yield (and thus wait for another 10ms cycle):
shoot
sleep (duh )
click, press, release
wait_click
md_detect_motion

Additionally, a "count hook" is used so that lua will yield every 1000 lua VM instructions (with some exceptions that I'm not totally clear on.) You can't easily tell how many VM instructions a bit of lua code turns into, but in general every keyword, variable reference, operator or function call will turn into one or more. Comments and whitespace are taken care of by the compiler, so you don't have to worry about them.

Note that you can check execution time using get_tick_count, although the resolution is limited to 10ms.

e.g.
t0=get_tick_count()
... do the stuff you want to time
print("time: ",get_tick_count()-t0)

One thought that has occurred to me, for both languages, is that we could just sleep based on execution time. Maybe the 10ms resolution of get_tick_count would be a problem ? I don't suppose ARM has anything like an x86 TSC ?

I'm glad you brought this up, because I should probably also make the file IO stuff sleep too.

Another thing that should be documented is how large numbers we can use. I assume it's +-2^15 or +-2^31, one short from one (negative?) end (this one we're missing in our ubasic documentation as well).

CHDK Lua numbers are 32 bit signed ints.

[Velo]

It looks to me like the following yield (and thus wait for another 10ms cycle):
shoot
sleep (duh )
click, press, release
wait_click
md_detect_motion

Exactly. This is because some event is scheduled in the internal queue. Lua resumes, when the queue is empty again.

Additionally, a "count hook" is used so that lua will yield every 1000 lua VM instructions (with some exceptions that I'm not totally clear on.) You can't easily tell how many VM instructions a bit of lua code turns into, but in general every keyword, variable reference, operator or function call will turn into one or more. Comments and whitespace are taken care of by the compiler, so you don't have to worry about them.

The count hook isn't strictly necessary. As long as you don't write loops that contain no yield-functions (see above). My first version of the patch hadn't this hook and it was easy to freeze the cam with scripts like:

Code: [Select]

while not is_pressed "up" do
end

If you use sleep(0) in this loops you don't need the count hook.

The exception you mentioned in the count hook has to do with yielding across c-calls. Yielding from the count-hook is more or less an undocumented feature. But it only works when there are no c-frames on the call stack.
This shouldn't be possible now, because you have to have a callstack like this Lua->C->Lua. But if some later patch introduces callbacks from C->lua everything should work fine.

[fudgey]

Thanks for the answers you all!

Additionally, a "count hook" is used so that lua will yield every 1000 lua VM instructions (with some exceptions that I'm not totally clear on.) You can't easily tell how many VM instructions a bit of lua code turns into, but in general every keyword,

I take this count is cleared each time sleep() or one of the other commands you listed are executed so this will never happen with code that has "enough" sleep creating commands in it?

One thought that has occurred to me, for both languages, is that we could just sleep based on execution time. Maybe the 10ms resolution of get_tick_count would be a problem ? I don't suppose ARM has anything like an x86 TSC ?

I once spent a little while looking for a timer that would update faster (working on the assumption that a faster clock has to be responsible of updating that 10 ms ticker be it via an interrupt or something else) and be very likely readable but couldn't find anything. But that's very much probably just because I really didn't know where and how to look.

I'm no expert in ARM so please enlighten me if I'm wrong, but I've understood that ARM cores are pretty much just cores, and as such any peripherals like timers are pretty much implementation specific and handled as co-processors, meaning that it's hard to know how to access the Digic processor peripherals unless they are following some industry convention unknown to me. Of course we have disassemblies to dig into for all that...

A timer interrupt would have no debug strings to slow it down making it harder to find unless ARM has restrictions for placing them (looks like wdt things are pretty early in our dumps for instance)?