supplierdeeply

Source code re-org for modules?

  • 36 Replies
  • 3733 Views
*

Offline philmoz

  • *****
  • 3041
    • Photos
Re: Source code re-org for modules?
« Reply #10 on: 12 / December / 2012, 06:44:22 »
Advertisements
Latest version in philmoz-reorg includes conversion of ubasic and Lua to modules.

Appropriate module is loaded when script is run (from gui or ptp). Module is not unloaded unless script language changes.

This is a work in progress and there is still a bit to do (and may not actually work correctly in all cases), it also has Lua native calls enabled so take care when using.

This gives a fair bit more memory when running ubasic scripts; but due to the current module overhead it actually loses a couple of K for Lua scripts. Tying to improve this.

Edit: Revision 2375 reduces the overhead of compiling Lua as a module. Loading Lua as a module now uses about 1.2K less memory than the current trunk version with both ubasic and Lua compiled into the core.

Edit2: With Lua and ubasic as modules the libraries only need to be built once so batch rebuild time for all cameras/firmware versions is now just under 15 minutes.

Phil.
« Last Edit: 13 / December / 2012, 07:25:11 by philmoz »
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)

*

Offline dvip

  • ****
  • 451
Re: Source code re-org for modules?
« Reply #11 on: 12 / December / 2012, 12:55:21 »
@Phil,
The only thing is, that I get this ubasic.o error using when building [email protected]

C:\chdk\gcc451\bin\gmake.exe[1]: *** No rule to make target `ubasic.o', needed by `ubasic.elf'.  Stop.
gmake: *** [all-recursive] Error 1

Thanks for your work, especially with the SX40HS-100i which I just got.


*

Offline philmoz

  • *****
  • 3041
    • Photos
Re: Source code re-org for modules?
« Reply #12 on: 12 / December / 2012, 14:03:43 »
@Phil,
The only thing is, that I get this ubasic.o error using when building [email protected]

C:\chdk\gcc451\bin\gmake.exe[1]: *** No rule to make target `ubasic.o', needed by `ubasic.elf'.  Stop.
gmake: *** [all-recursive] Error 1


I've added the file I missed in the previous commit.

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)

*

Offline dvip

  • ****
  • 451
Re: Source code re-org for modules?
« Reply #13 on: 12 / December / 2012, 17:02:11 »
@Phil,
It is fine now, I only get these warnings:

C:\chdk\gcc4\bin\gmake.exe[1]: *** Warning: File `ubasic.c' has modification time in the future (1355360568 > 1355349344)
C:\chdk\gcc4\bin\gmake.exe[1]: *** Warning:  Clock skew detected.  Your build may be incomplete.




*

Offline philmoz

  • *****
  • 3041
    • Photos
Re: Source code re-org for modules?
« Reply #14 on: 14 / December / 2012, 07:02:44 »
Some more improvements for Lua and uBasic as modules in revision 2377:

- Reduced the code size for the Lua module a bit more. Lua scripts have about 4K more free memory than the current trunk version.

- Native calls in Lua are disabled by default again; but can be enabled with a menu option in the Miscellaneous menu. A warning pops up if native calls are enabled. Also the code that allows scripts to change config items blocks changing the native call setting - native calls have to be user enabled.

- Standardised the interface between the core script engine and the Lua and uBasic modules. The script engine now doesn't care which language is loaded because the calls to the module are the same. Handling of loading/unloading the correct module moved to modules.c. In theory a completely new script language could be dropped in with little effort - other than writing the language handler of course :)

CHDK core size (G1X) is down to 136K in this version (trunk is currently 220K).

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)

*

Offline srsa_4c

  • ******
  • 3464
Re: Source code re-org for modules?
« Reply #15 on: 15 / December / 2012, 11:53:31 »
- Native calls in Lua are disabled by default again; but can be enabled with a menu option in the Miscellaneous menu. A warning pops up if native calls are enabled. Also the code that allows scripts to change config items blocks changing the native call setting - native calls have to be user enabled.
This change is useful for most of the users. However, there can be (isolated  :-[) cases where the camera buttons are not easily accessible (and browsing the CHDK menu is not possible via PTP/scripts). Can you include a build time option which permanently enables native calls? It might be as simple as changing the default (disabled) setting, although that doesn't cover the case where there's already a config file with disabled native calls.

edit: making the following code optional (build time option)
Quote
the code that allows scripts to change config items blocks changing the native call setting
could also help.
« Last Edit: 15 / December / 2012, 12:01:39 by srsa_4c »

*

Offline philmoz

  • *****
  • 3041
    • Photos
Re: Source code re-org for modules?
« Reply #16 on: 15 / December / 2012, 20:42:52 »
Revision 2385 adds the compile option to force Lua native calls on (as requested).
It also includes building a some of the core code as a library that only needs to be compiled once. Only code that is platform independent is built in this library. This reduces the total batch build time for all cameras to 11.5 minutes on my system.

This code re-organisation took a different turn to what I was originally planning; but I've learned a lot along the way :)

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)

Re: Source code re-org for modules?
« Reply #17 on: 15 / December / 2012, 21:32:39 »
This code re-organisation took a different turn to what I was originally planning; but I've learned a lot along the way :)
What doesn't kill you makes you stronger ?

What doesn't kill you makes you stronger ?
Ported : A1200  SD940  G10  Powershot N  G16*


*

Offline philmoz

  • *****
  • 3041
    • Photos
Re: Source code re-org for modules?
« Reply #18 on: 22 / December / 2012, 18:25:44 »
Here's a quick summary of the changes in my philmoz-reorg branch to date:

Batch Build
  • Move /core/modules to /modules
  • Move /core/games to /modules/games
  • Move module related code files to /modules directory
  • Batch build compiles module code only once
  • Batch build compiles libraries (/lib/*) only once.
  • Add a new library (/lib/core) to compile platform independant 'core' code only once. This has the added bonus that code can be compiled into the library always; but will be excluded by the linker if not used (e.g. gps_math.c). The /lib/core directory does not contain any source files, the code is still stored in /core.

Platform independance
  • Split most of platform.h into seperate include files by function. Each include file has a warning not to put platform dependant things inside.
  • Fix platform dependant I/O stuff in stdlib.h. All platform dependancy moved to wrappers.c
  • The platform.h and camera.h files will generate an error if included in module or library code files.
  • Code cleanup to use the camera_info structures instead of #ifdef logic.

Lua and uBasic as modules
  • Convert Lua and uBasic to modules.
  • Standardise the interface from the script engine to the Lua/uBasic modules. Remove the OPT_LUA and OPT_UBASIC compile time options.
  • Removed OPT_DBG_DUMP compile time option. The dbg_dump.c file is compiled always; but excluded by the linker unless OPT_DBG_LUA_ASSERT is defined. Note the default build does not set OPT_DBG_LUA_ASSERT; but a custom build can be created to enable Lua debugging.
  • Remove the OPT_LUA_CALL_NATIVE compile time option. Lua native calls can be enabled in the CHDK menu (disabled by default). Added OPT_FORCE_LUA_CALL_NATIVE compile time option to force Lua native calls to be enabled in the build.
  • Rewrite of the script and action_stack code to simplify the module interface and cleanup the code (26/12).

Build options
  • Removed OPT_TEXTREADER and OPT_CALENDAR compile time options. These are modules and only require one menu entry in the core code so the space saving is neglible.
  • Removed OPT_SCRIPTING option. With Lua and uBasic as modules the savings from removing the remaining script code is small.
  • Removed CAM_DETECT_SCREEN_ERASE option - always enabled for all cameras.

Other
  • Cleanup bracketing code.
  • Added rudi's fast integer math library to Lua.
  • Merge all other trunk changes.

Phil.
« Last Edit: 25 / December / 2012, 20:25:05 by philmoz »
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)

*

Offline dvip

  • ****
  • 451
Re: Source code re-org for modules?
« Reply #19 on: 24 / December / 2012, 16:40:43 »
Phil,
While testing Override Tv [Ev Step] Ev Step Value,
I noticed that no matter what Ev Value  I choose, I always
get an image with 1/2000. I tried this with the A590is-101b and [email protected]


 

Related Topics