uBASIC speed optimizations (topic split from another thread) - General Discussion and Assistance - CHDK Forum supplierdeeply
« previous next »
Pages: [1] 2 3 4   Go Down

uBASIC speed optimizations (topic split from another thread)

  • 36 Replies
  • 12237 Views
*

Offline tsvstar

  • ***
  • 231
uBASIC speed optimizations (topic split from another thread)
« on: 05 / November / 2011, 10:16:29 »
Advertisements
Ubasic optimization.

What is done:

- Greatly increase ubasic execution speed. From tens (print flow) to thousands (raw calculation cycle) times.
  * Execution flow keep inside ubasic most part of time (avoid huge transactions cost for each statement and greatly increase execution speed)
  * Parser optimized to be 4-6 times faster on first pass and 30-40 times faster(runtime compilation used) on next passes
  * Control flow operators are now zero-cost (do not parse script from begin each time). So now size and complexity of script has no matter
- Some fixes:
  * Fixed bug: hanging-up if unknown token found in some places (in print statement for example)
  * Fixed bug: error "No label" if jump to the end
- PC-version updated to last version, added support simple parsing check (not always execution match all branches


Known limitations: 32Kb script size, 250 symbols per string
Changes was heavily tested: by parser benchmark, common and special test execution script, tokens flow reverting to strings
« Last Edit: 06 / November / 2011, 11:20:18 by acseven »
Logged
*

Offline waterwingz

  • ******
  • 12121
    • Waterwingz
Re: Re: Adding new cameras, applying patches into trunk (with source code prepared)
« Reply #1 on: 06 / November / 2011, 10:37:40 »
Quote from: tsvstar on 05 / November / 2011, 10:16:29
- Greatly increase ubasic execution speed. From tens (print flow) to thousands (raw calculation cycle) times.
  * Execution flow keep inside ubasic most part of time (avoid huge transactions cost for each statement and greatly increase execution speed)
From recent PM's, it appears that the current developers are busy on other non-CHDK things right now so responses to changes this significant might take a little while.

I took a look myself and am impressed. Looks like some nice optimizations.  FWIW - currently the standard is TortoiseSVN to produce patch files but I was able to rename your file from .txt to .patch and it worked.

My biggest concern right now might be the change to execution flow that allows uBASIC to take up more of the available camera processing time.  Your comment in the code says "This is good balance between script speed and chdk reponsivness" but its not just chdk responsiveness you need to worry about.  The entire camera waits while uBASIC runs (i.e. CHDK tasks run in cooperation with CANON's tasks).   I guess it will take testing on older (slower) cameras as well as new ones to see how this works out.

I'll try it on my G10 and SD940 and see what happens.

We might want to get these posts moved into their own thread too .. this discussion does not belong here.





Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline tsvstar

  • ***
  • 231
Re: Re: Adding new cameras, applying patches into trunk (with source code prepared)
« Reply #2 on: 06 / November / 2011, 10:52:39 »
Regarding to moving discussion - sure. Please move this posts if you know how to do this.

Regarding to execution flow - yes, it is really only possible unsafe change but it make most impact to execution speed. Other optimization are also have influent but on big scripts. I finish them before I found reason why ubasic is so slow in CHDK.

Regarding to my phrase about balance. I do not investigate all nuances of chdk execution flow. It is too big and entagleous. So I'm not sure about possible impact to shooting postprocessing or interruption for example. I decide to do this optimization in safest way which shouldn't broke anything independent on how CHDK is (or will be) organized. Then I do test on my S95 and see no difference in scripts behaviour.
« Last Edit: 06 / November / 2011, 11:10:36 by tsvstar »
Logged
*

Offline waterwingz

  • ******
  • 12121
    • Waterwingz
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #3 on: 06 / November / 2011, 11:24:59 »
Quote from: tsvstar on 06 / November / 2011, 10:52:39
Then I do test on my S95 and see no difference in scripts behaviour.
I assume you meant to say that you tested on your S95 and saw no  difference in script behaviour except the speed ?

I think we will have to wait for reyalp to weigh in on this. I can't speak for him but I expect he will ask why take a risk when the existing method is fast enough for what its used for ?   But while he is not a fan of uBASIC, this same speed-up change would work for LUA too I expect.   

We will need testing on older & slower cameras before committing to this - your S95 is a pretty high end machine.
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline tsvstar

  • ***
  • 231
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #4 on: 06 / November / 2011, 12:06:35 »
Quote from: waterwingz on 06 / November / 2011, 11:24:59
I assume you meant to say that you tested on your S95 and saw no  difference in script behaviour except the speed ?
Exactly.

Quote from: waterwingz
I think we will have to wait for reyalp to weigh in on this. I can't speak for him but I expect he will ask why take a risk when the existing method is fast enough for what its used for ?   But while he is not a fan of uBASIC, this same speed-up change would work for LUA too I expect.   
My primary goal is optimize ubasic for futher use it as logic/controling extension. I did plan use it for some kind of switcheable "modes"(like Landscape, Indoor, ..) and switcheable profiles (Me-My blondie wife), which could be scripted by anyone in his personal preferences.
I need good enough perfomance (after optimization it is near to LUA) and stability (I have some negative expirience with LUA when camera was hanged up or script crashed possible because memory; even on my highend S95 :) )

Regarding to LUA, this will not work. Because it already use this. Details are in the end of message
 
Quote from: waterwingz
We will need testing on older & slower cameras before committing to this - your S95 is a pretty high end machine.
Sure.
But I think If any we could decrease interval to 10 msec. Then nothing will be missed surely.
Unfortunatelly this will slowdown back ubasic sometimes more than twice. (althrough it will be still surely much faster than before).


PS. I do investigate now why LUA is so fast.. I do not understand how LUA workflow processed, so I just do debug output to file ticks of enter and leave LUA points. I see that on my heavy calculation and printing test script sometimes it return almost in zero times, but sometime (in dump prop cycle for example) it will keep execution flow inside up to 650 msec. So this is nothing in compare with 50msec guaranteed response (and 0msec on raise action) for modified ubasic.
I suspect that  LUA return flow each 1000 operator and on actions or something like that.
« Last Edit: 06 / November / 2011, 12:19:30 by tsvstar »
Logged
*

Offline reyalp

  • ******
  • 13624
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #5 on: 10 / November / 2011, 00:38:45 »
Quote from: tsvstar on 06 / November / 2011, 12:06:35
PS. I do investigate now why LUA is so fast.. I do not understand how LUA workflow processed, so I just do debug output to file ticks of enter and leave LUA points.
ubasic (current version without your path) only runs one line per kbd_task loop iteration, except for REMs and labels (and the latter was an optimization I added some time ago). Lua runs 1000 lua vm instructions between yields.

In both cases, this is needed to keep kbd task running at a reasonable rate.

I think making ubasic run more than one line per kbd_task iteration is a good thing, as long as you make sure to yield for all the things that need to do it right away (which a quick glance at your patch suggests it does)

If you are going to do it based on time, I'd say 10ms (the smallest increment get_tick_count measures) would be a better choice, otherwise kbd_task iterations will take much longer than the camera expects and this could have side effects. Doing some fixed number of cycles like lua would be another option.

If I was going to merge this, I'd rather not have unrelated ubasic optimizations in the same patch. Just more than one line per cycles will make it orders of magnitude faster. Note that even running 1000 instructions per kbd task, the yield time is by far the limiting factor in lua performance: http://chdk.setepontos.com/index.php?topic=6729.msg71456#msg71456  but this simply isn't a problem for most scripting tasks.

I'm not personally not inclined to expend much effort on ubasic, Lua is a far better language, and the effort to bring ubasic to anything close to that standard would be huge. Not performance , but things like multi-letter variable names, local variables, strings, tables, functions, file io...

That doesn't mean I won't apply improvements for ubasic, but if they take me a lot of work to understand and apply, that certainly won't count in their favor.

If you have problems with lua running out of memory or crashing from other things, I'd say the effort would be much better spent debugging those.
Logged
Don't forget what the H stands for.
*

Offline tsvstar

  • ***
  • 231
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #6 on: 10 / November / 2011, 01:44:20 »
Sure. I easily could separate required changes and decrease tick it to 10ms.

All other changes have no side-effects and isolated inside ubasic. So they are much safer. And as I say they heavily tested on different examples.
This other two parts also have reasons:
- Parser optimization+runtime compilation allow to really quickly do goto to label
- Zerocost control flow is just easy to implement and it resolve possible problem which I found in some ubasic script ("Please do not write remarks below to increase perfomance" :)).

I clearly its limitation (everything that you list) and have no plan to resolve them. But for my far meta-goal ubasic is enough.
As I say I plan to base on scripts extendend configurable logic. Something that will quickly reconfigure chdk&camera to different scenes/tasks. And something smart enough to give warnings (like wrong dial, etc) raised on event.

Base language should have basic controls, have minimal memory footprint (ubasic is both in chdk and in heap) and could quickly response on external goto/function_call(this is target of my patch).
Logged
*

Offline tsvstar

  • ***
  • 231
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #7 on: 10 / November / 2011, 02:13:10 »
In attachment isolated patch to hold execution flow inside ubasic.

BTW: as I wrote before - LUA on test sample keep execution flow sometimes up to 600msec. I surely realize that this was ultimate example and it shouldn't happen on real scripts. :)
« Last Edit: 10 / November / 2011, 02:20:30 by tsvstar »
Logged
*

Offline zeno

  • *****
  • 889
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #8 on: 10 / November / 2011, 04:45:24 »
Some time back (nearly three years ago), I modified ubasic to execute more than one instruction per tick - see this thread http://chdk.setepontos.com/index.php?topic=2933.0. While nobody seemed to want it added to the build process, it did get added to SDM.

I also modified ubasic to support strings ( see http://www.zenoshrdlu.com/kapstuff/zsubasic.html) as a first step to making ubasic more powerful but again there wasn't any interest in adding it to the build process.

I hope you have more success!

Logged
A570, S100, Ixus 127
Author of ASSIST, STICK, WASP, ACID, SDMInst, LICKS, WICKS, MacBoot, UBDB, CFGEdit
*

Offline tsvstar

  • ***
  • 231
Re: uBASIC speed optimizations (topic split from another thread)
« Reply #9 on: 10 / November / 2011, 05:00:17 »
Thanks for ubasic with string, zeno. Probably I will use in my own home branch CHDK.

I also hope to including my work into CHDK. I could do my changes for me, but I would like to share them with community.
Are any other useful for main brunch changes like ubasic_w_string exists in SDM? And is anywhere exists SVN of SDM?
Logged
Pages: [1] 2 3 4   Go Up
« previous next »
 

Related Topics