Source code re-org for modules?

philmoz

3450

Re: Source code re-org for modules?

« Reply #10 on: 12 / December / 2012, 06:44:22 »

Advertisements

Latest version in philmoz-reorg includes conversion of ubasic and Lua to modules.

Appropriate module is loaded when script is run (from gui or ptp). Module is not unloaded unless script language changes.

This is a work in progress and there is still a bit to do (and may not actually work correctly in all cases), it also has Lua native calls enabled so take care when using.

This gives a fair bit more memory when running ubasic scripts; but due to the current module overhead it actually loses a couple of K for Lua scripts. Tying to improve this.

Edit: Revision 2375 reduces the overhead of compiling Lua as a module. Loading Lua as a module now uses about 1.2K less memory than the current trunk version with both ubasic and Lua compiled into the core.

Edit2: With Lua and ubasic as modules the libraries only need to be built once so batch rebuild time for all cameras/firmware versions is now just under 15 minutes.

Phil.

« Last Edit: 13 / December / 2012, 07:25:11 by philmoz »

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

dvip

451

Re: Source code re-org for modules?

« Reply #11 on: 12 / December / 2012, 12:55:21 »

@Phil,
The only thing is, that I get this ubasic.o error using when building philmoz-reorg@2372.

C:\chdk\gcc451\bin\gmake.exe[1]: *** No rule to make target `ubasic.o', needed by `ubasic.elf'. Stop.
gmake: *** [all-recursive] Error 1

Thanks for your work, especially with the SX40HS-100i which I just got.

Logged

philmoz

3450

Re: Source code re-org for modules?

« Reply #12 on: 12 / December / 2012, 14:03:43 »

Quote from: dvip on 12 / December / 2012, 12:55:21

@Phil,
The only thing is, that I get this ubasic.o error using when building philmoz-reorg@2372.

C:\chdk\gcc451\bin\gmake.exe[1]: *** No rule to make target `ubasic.o', needed by `ubasic.elf'. Stop.
gmake: *** [all-recursive] Error 1

I've added the file I missed in the previous commit.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

dvip

451

Re: Source code re-org for modules?

« Reply #13 on: 12 / December / 2012, 17:02:11 »

@Phil,
It is fine now, I only get these warnings:

C:\chdk\gcc4\bin\gmake.exe[1]: *** Warning: File `ubasic.c' has modification time in the future (1355360568 > 1355349344)
C:\chdk\gcc4\bin\gmake.exe[1]: *** Warning: Clock skew detected. Your build may be incomplete.

Logged

philmoz

3450

Re: Source code re-org for modules?

« Reply #14 on: 14 / December / 2012, 07:02:44 »

Some more improvements for Lua and uBasic as modules in revision 2377:

- Reduced the code size for the Lua module a bit more. Lua scripts have about 4K more free memory than the current trunk version.

- Native calls in Lua are disabled by default again; but can be enabled with a menu option in the Miscellaneous menu. A warning pops up if native calls are enabled. Also the code that allows scripts to change config items blocks changing the native call setting - native calls have to be user enabled.

- Standardised the interface between the core script engine and the Lua and uBasic modules. The script engine now doesn't care which language is loaded because the calls to the module are the same. Handling of loading/unloading the correct module moved to modules.c. In theory a completely new script language could be dropped in with little effort - other than writing the language handler of course

CHDK core size (G1X) is down to 136K in this version (trunk is currently 220K).

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

srsa_4c

4451

Re: Source code re-org for modules?

« Reply #15 on: 15 / December / 2012, 11:53:31 »

Quote from: philmoz on 14 / December / 2012, 07:02:44

- Native calls in Lua are disabled by default again; but can be enabled with a menu option in the Miscellaneous menu. A warning pops up if native calls are enabled. Also the code that allows scripts to change config items blocks changing the native call setting - native calls have to be user enabled.

This change is useful for most of the users. However, there can be (isolated

) cases where the camera buttons are not easily accessible (and browsing the CHDK menu is not possible via PTP/scripts). Can you include a build time option which permanently enables native calls? It might be as simple as changing the default (disabled) setting, although that doesn't cover the case where there's already a config file with disabled native calls.

edit: making the following code optional (build time option)

Quote

the code that allows scripts to change config items blocks changing the native call setting

could also help.

« Last Edit: 15 / December / 2012, 12:01:39 by srsa_4c »

Logged

philmoz

3450

Re: Source code re-org for modules?

« Reply #16 on: 15 / December / 2012, 20:42:52 »

Revision 2385 adds the compile option to force Lua native calls on (as requested).
It also includes building a some of the core code as a library that only needs to be compiled once. Only code that is platform independent is built in this library. This reduces the total batch build time for all cameras to 11.5 minutes on my system.

This code re-organisation took a different turn to what I was originally planning; but I've learned a lot along the way

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

waterwingz

12140

Re: Source code re-org for modules?

« Reply #17 on: 15 / December / 2012, 21:32:39 »

Quote from: philmoz on 15 / December / 2012, 20:42:52

This code re-organisation took a different turn to what I was originally planning; but I've learned a lot along the way

What doesn't kill you makes you stronger ?

What doesn't kill you makes you stronger ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

philmoz

3450

Re: Source code re-org for modules?

« Reply #18 on: 22 / December / 2012, 18:25:44 »

Here's a quick summary of the changes in my philmoz-reorg branch to date:

Batch Build

Move /core/modules to /modules
Move /core/games to /modules/games
Move module related code files to /modules directory
Batch build compiles module code only once
Batch build compiles libraries (/lib/*) only once.
Add a new library (/lib/core) to compile platform independant 'core' code only once. This has the added bonus that code can be compiled into the library always; but will be excluded by the linker if not used (e.g. gps_math.c). The /lib/core directory does not contain any source files, the code is still stored in /core.

Platform independance

Split most of platform.h into seperate include files by function. Each include file has a warning not to put platform dependant things inside.
Fix platform dependant I/O stuff in stdlib.h. All platform dependancy moved to wrappers.c
The platform.h and camera.h files will generate an error if included in module or library code files.
Code cleanup to use the camera_info structures instead of #ifdef logic.

Lua and uBasic as modules

Convert Lua and uBasic to modules.
Standardise the interface from the script engine to the Lua/uBasic modules. Remove the OPT_LUA and OPT_UBASIC compile time options.
Removed OPT_DBG_DUMP compile time option. The dbg_dump.c file is compiled always; but excluded by the linker unless OPT_DBG_LUA_ASSERT is defined. Note the default build does not set OPT_DBG_LUA_ASSERT; but a custom build can be created to enable Lua debugging.
Remove the OPT_LUA_CALL_NATIVE compile time option. Lua native calls can be enabled in the CHDK menu (disabled by default). Added OPT_FORCE_LUA_CALL_NATIVE compile time option to force Lua native calls to be enabled in the build.
Rewrite of the script and action_stack code to simplify the module interface and cleanup the code (26/12).

Build options

Removed OPT_TEXTREADER and OPT_CALENDAR compile time options. These are modules and only require one menu entry in the core code so the space saving is neglible.
Removed OPT_SCRIPTING option. With Lua and uBasic as modules the savings from removing the remaining script code is small.
Removed CAM_DETECT_SCREEN_ERASE option - always enabled for all cameras.

Other

Cleanup bracketing code.
Added rudi's fast integer math library to Lua.
Merge all other trunk changes.

Phil.

« Last Edit: 25 / December / 2012, 20:25:05 by philmoz »

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

dvip

451

Re: Source code re-org for modules?

« Reply #19 on: 24 / December / 2012, 16:40:43 »

Phil,
While testing Override Tv [Ev Step] Ev Step Value,
I noticed that no matter what Ev Value I choose, I always
get an image with 1/2000. I tried this with the A590is-101b and philmoz-reorg@2404.

Logged