Automating the batch build in the Makefile - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: [1]   Go Down

Automating the batch build in the Makefile

  • 8 Replies
  • 4938 Views
*

Offline philmoz

  • *****
  • 3450
    • Photos
Automating the batch build in the Makefile
« on: 13 / August / 2011, 06:01:12 »
Advertisements
Currently the top level Makefile has individual lines for each camera & firmware version in three targets - 'batch-zip', 'batch-zip-complete' and 'clean'.

With the ever growing list of ports this is getting a bit unwieldy.

Attached is a patch for review that automates this by scanning the directories under the platform and then for each camera found scans for firmware versions in the sub directory.

For this to work some changes & additions are used:
- A new (optional) include file called 'platform_conf.inc' can be placed in each platform/CAMERA directory. This file can contain any camera specific definitions that are currently duplicated in each sub/xxx/makefile.inc file. It is also used in this case to contain a definition for a STATE variable that defines cameras currently listed as _BETA. The patch includes this file for all cameras currently classified as BETA. The $(STATE) value is then added to the .zip file names during the build.
- For firmware versions that are copies of other versions a new definition is added to the base firmware version makefile.inc file. For example in the d10 100a makefile.inc file the line 'COPY_TO=100b' is added. When the 100a firmware .zip files are built this will be used to copy the 100a files to 100b version files.
- To stop a camera from being built automatically add a file called NO_AUTOBUILD to the platform/CAMERA directory. To stop a specific firmware version from being built; but allow other firmware versions for a camera, add the NO_AUTOBUILD file to the platform/CAMERA/sub/FIRMWARE directory.

Comments & suggestions appreciated.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: Automating the batch build in the Makefile
« Reply #1 on: 13 / August / 2011, 09:25:44 »
Can't really look at the patch file until I get home but I really like the description - especially the ease of adding things to or hiding them from the build.    Did you think about cleaning up the makefile.inc in the trunk while you were doing this ?
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: Automating the batch build in the Makefile
« Reply #2 on: 16 / August / 2011, 20:13:35 »
Downloaded the patch file and applied it to a newly downloaded virgin trunk1293.  I moved my G10 loader and platform sub directories over but made no other changes.  Fired up CHDK-Shell and it dies right away with the following error message, the GUI window never even appears :



Deleted the G10 stuff - same error.

What did I miss ?
« Last Edit: 16 / August / 2011, 20:16:47 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Automating the batch build in the Makefile
« Reply #3 on: 16 / August / 2011, 20:23:37 »
Quote from: waterwingz on 16 / August / 2011, 20:13:35
Downloaded the patch file and applied it to a newly downloaded virgin trunk1293.  I moved my G10 loader and platform sub directories over but made no other changes.  Fired up CHDK-Shell and it dies right away with the following error message, the GUI window never even appears :

From memory CHDK-Shell parses the Makefile to get the list of cameras (skipping comments).
It assumes the batch targets are in a certain format so this change will need an update to CHDK-Shell.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: Automating the batch build in the Makefile
« Reply #4 on: 16 / August / 2011, 20:27:12 »
Quote from: philmoz on 16 / August / 2011, 20:23:37
From memory CHDK-Shell parses the Makefile to get the list of cameras (skipping comments).
It assumes the batch targets are in a certain format so this change will need an update to CHDK-Shell.

Thanks Phil.

Ping whim .....
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Automating the batch build in the Makefile
« Reply #5 on: 11 / September / 2011, 07:43:40 »
I've updated my development branch with the latest version of this revised makefile build process.

Removed the need for the NO_AUTOBUILD and platform_conf.inc files to control the build and BETA status.

BETA status is managed using variables in the main Makefile to define the beta cameras and firmware versions.

Disabling auto build for cameras is controlled in makefile.cam and for a specific camera firmware version in makefile.fw. See the comments in the various files.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline reyalp

  • ******
  • 14125
Re: Automating the batch build in the Makefile
« Reply #6 on: 11 / September / 2011, 15:39:18 »
I think this is a big improvement. The only outstanding issue I'd think about before putting it in the trunk is chdk shell compatibility. If CHDK shell really needs a parseable a list (I don't know if it does, better check with whim first), a possible solution would be to generate it with another batch build command, which just echos the appropriate firmware/sub values to a file. Just a thought.
Logged
Don't forget what the H stands for.
*

Offline whim

  • ******
  • 2046
  • A495/590/620/630 ixus70/115/220/230/300/870 S95
Re: Automating the batch build in the Makefile
« Reply #7 on: 14 / September / 2011, 11:08:41 »
Thanks philmoz / reyalp / waterwingz !

CHDK-Shell 3.35 has just been released, it has been tested on philmoz1336 branch release.

cheers,

wim

PS I'm using the presence of the line ".PHONY: fir"  in Makefile as signal that the 'new Makefile system' is active;
if this line is not found it assumes the current Makefile system is active.
   
     
Logged
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Automating the batch build in the Makefile
« Reply #8 on: 09 / October / 2011, 02:59:37 »
After some more thought and trying to speed up the creation of the 'camindex.csv' file for CHDK-Shell, I've come to the conclusion that a simpler approach might be better.

My dev branch (branches/philmoz) now has this new approach:
- This version no longer scans all the directories and sub-directories in platform.
- The info that specifies which cameras & firmware versions to build and the beta status is stored in the camera_list.csv file.
- a new shell script auto_build.sh uses this file to perform the batch build actions
- all the PLATFORM & PLATFORMSUB entries have been removed from makefile.inc. Make will print an error message if these aren't defined.

The 'camera_list.csv' file becomes part of the build and would be maintained manually as new cameras and versions are added or beta status changes.

This is simpler and faster than the directory scanning approach and the camera_list.csv file can also be used by CHDK-Shell.
All of the relevant information is now in one place instead of being spread out across multiple files, and there is also only one copy of the information for each camera (instead of the current mess in Makefile).

Comments and suggestions appreciated.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
Pages: [1]   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal