1.4 development planning thread

srsa_4c

4451

Re: 1.4 development planning thread

« Reply #120 on: 04 / May / 2015, 19:45:28 »

Advertisements

Quote from: philmoz on 04 / March / 2015, 19:51:57

I would not worry about auto_build_parallel.sh - I should probably remove this.
It is Windows only, and no longer works on 8.0 or 8.1 - seems Windows now refuses to run hundreds of command prompts simultaneously.

I think I found a possible solution to this.
Changes needed:

Code: [Select]

Index: Makefile
===================================================================
--- Makefile	(revision 4153)
+++ Makefile	(working copy)
@@ -260,7 +260,7 @@
 # Note:- Windows only, this will use all available CPU and a fair amount of memory
 #        but will rebuild much faster on a machine with many CPU cores
 batch-rebuild-stubs-parallel:
-	sh tools/auto_build_parallel.sh $(MAKE) rebuild-stubs $(CAMERA_LIST) -noskip
+	sh tools/auto_build_parallel.sh $(MAKE) rebuild-stubs $(CAMERA_LIST) -noskip | parallel
 
 batch-clean:
 	$(MAKE) -C tools clean

and

Code: [Select]

Index: tools/auto_build_parallel.sh
===================================================================
--- tools/auto_build_parallel.sh	(revision 4153)
+++ tools/auto_build_parallel.sh	(working copy)
@@ -5,11 +5,10 @@
 # make program ($1)
 # ($4) = -noskip disable SKIP_AUTOBUILD
 # - also see main Makefile
-# parallel version - starts each build in a seperate session (Windows only)
 while IFS=, read cam fw state copy skip
 do
   if [ ${cam} != "CAMERA" ] && ( [ "$4" = "-noskip" ] || [ "${skip}" = "" ] ); then \
     if [ "${state}" != "" ]; then state=_${state}; fi; \
-    start $1 -s --no-print-directory PLATFORM=${cam} PLATFORMSUB=${fw} STATE=${state} COPY_TO=${copy} SKIP_AUTOBUILD=${skip} $2 || exit 1; \
+    echo $1 -s --no-print-directory PLATFORM=${cam} PLATFORMSUB=${fw} STATE=${state} COPY_TO=${copy} SKIP_AUTOBUILD=${skip} $2 || exit 1; \
   fi;
 done < $3

'parallel' is GNU Parallel, it runs one command per CPU core by default. It's a Perl script that should also work on Windows (did not try though).
This method might even work for other batch targets (once issues like the one with the 2 kinds of modules are solved).
The above is only for information ATM, will be part of the DIGIC6 support once it's ready...

Logged

waterwingz

12140

Re: 1.4 development planning thread

« Reply #121 on: 04 / May / 2015, 19:59:28 »

Quote from: reyalp on 24 / January / 2015, 15:33:33

My thinking has been along the lines of
Development continues through February
Stabilization start March / April
Release candidate April / May
Release in June / July
This is all off the top of my head, and the stabilization and RC phases could be compressed if things go well.

Bump ....

The forum has been pretty quiet recently - not a lot of new CHDK core projects in process from what I can tell? But all those tasty script upgrade are still waiting for stable release. Not to mention the RAW shooting hooks.

And .. well .. it's now May.

Logged

Ported : A1200 SD940 G10 Powershot N G16

srsa_4c

4451

Re: 1.4 development planning thread

« Reply #122 on: 04 / May / 2015, 20:21:09 »

Quote from: waterwingz on 04 / May / 2015, 19:59:28

The forum has been pretty quiet recently - not a lot of new CHDK core projects in process from what I can tell? But all those tasty script upgrade are still waiting for stable release. Not to mention the RAW shooting hooks.

And .. well .. it's now May.

We did not get a lot of help from users, so we don't know for sure if all ports function correctly with the rewritten keyboard code. (Well, the ixus240/elph320 surely doesn't work as desired and I don't know if that can be fixed 'blindly'...).

Logged

waterwingz

12140

Re: 1.4 development planning thread

« Reply #123 on: 04 / May / 2015, 20:50:34 »

Quote from: srsa_4c on 04 / May / 2015, 20:21:09

We did not get a lot of help from users, so we don't know for sure if all ports function correctly with the rewritten keyboard code. (Well, the ixus240/elph320 surely doesn't work as desired and I don't know if that can be fixed 'blindly'...).

I understand that. But if substantial completion of that testing is to be the release criteria then 1.4.0 will likely become the permanent unstable and CHDK will stall out at 1.3.0.

One way around that is to bump to 1.4.0 and fix concerns as they come along.

I'm open to a better choice than those two options.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14118

Re: 1.4 development planning thread

« Reply #124 on: 05 / May / 2015, 00:46:36 »

Quote from: waterwingz on 04 / May / 2015, 19:59:28

Bump ....

The forum has been pretty quiet recently - not a lot of new CHDK core projects in process from what I can tell? But all those tasty script upgrade are still waiting for stable release. Not to mention the RAW shooting hooks.

And .. well .. it's now May.

This has been on my mind too. As a great man once said "I love deadlines. I love the whooshing noise they make as they go by."

I think we are pretty much ready to go to stabilization / rc phase. My plan is to highlight the 1.4 and testing need on the wiki front page and downloads page at that point, which will hopefully get a few more of the ports tested before release.

Outstanding things I want to deal with before that
* fix the **** ixus240 so it's at least not more broken than 1.3
* integrate the digic 6 code and makefile changes
* tidy up some things in the raw hook

The last two weekends I've had time for CHDK stuff, I've been stuck on the first item :/

Logged

Don't forget what the H stands for.

andrew.stephens.754365

461

Re: 1.4 development planning thread

« Reply #125 on: 05 / May / 2015, 08:36:41 »

Quote from: srsa_4c on 04 / May / 2015, 20:21:09

We did not get a lot of help from users

I only just noticed this thread - normally anything with "development" in the Subject would scare me to death!

Anyway, i'm currently running sx150is-100a-1.4.0-r4153 but not really aware of what may not work due to the faily limited stuff i've been doing with it so far.

I'll try and read through those threads mentioned on the "Testing Needed" wiki, but -
is there any chance a concise list of tests could be posted? and users reply for their model in that single thread?
and
is there any further way that could be set as a priority thread , say right at the top of the http://chdk.setepontos.com/index.php page ?

Just a couple of (perhaps worthless) suggestions.

Logged

reyalp

14118

Re: 1.4 development planning thread

« Reply #126 on: 08 / August / 2015, 20:26:51 »

Time to bump this again

(thanks to gentle prodding by waterwingz)

I think the rawop code is good enough for now. The kbd.c rework isn't tested on every port, but given the results so far I'm confident that not too many ports are broken. That leaves digic 6 support as the only major outstanding item.

This raises the question: Put d6 support in before the 1.4 release, or leave it for 1.5? My feeling is still that it would be better to have it in 1.4, assuming it doesn't cause problems for the existing ports. Reasons
* have it in the official source before anyone starts other d6 ports.
* not have the existing d6 ports tied to a feature-unstable development branch.

When I last looked at the impact on the core or older ports, it didn't seem too terrible. Note that I'm not concerned if the d6 ports are in the autobuild for the 1.4 release, or how complete they are.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: 1.4 development planning thread

« Reply #127 on: 09 / August / 2015, 16:28:51 »

Quote from: reyalp on 08 / August / 2015, 20:28:16

Is this in your svn somewhere? I'd like to compare against the current trunk.

It is, although I'm maintaining an up-to-date local copy, based on your integration patch. I'm attaching a diff based on the latter.
It includes this little change too (which I think is harmless).
I'm not against including D6 support, although some parts of the code will need improvements , and that means backporting.

d6_trunk_150809_.diff (292.81 kB - downloaded 15 times.)

Logged

reyalp

14118

Re: 1.4 development planning thread

« Reply #128 on: 10 / August / 2015, 23:43:39 »

Quote from: srsa_4c on 09 / August / 2015, 16:28:51

Quote from: reyalp on 08 / August / 2015, 20:28:16
Is this in your svn somewhere? I'd like to compare against the current trunk.
It is, although I'm maintaining an up-to-date local copy, based on your integration patch. I'm attaching a diff based on the latter.

Thanks.

Quote

It includes this little change too (which I think is harmless).

Agree.

Quote

I'm not against including D6 support, although some parts of the code will need improvements , and that means backporting.

I think that's OK, the impact on the core seems fairly small, and backporting platform changes is usually not a big deal. Even if we stop backporting at some point, it's no worse for users than if we'd only put it in 1.5.

I'll make some more detailed notes and test my cameras with this tree later. I tried to look at the impact on chdkshell, but I haven't been able to get it running with an unmodified trunk yet.

So for now my plan is to get this in 1.4 and get to release candidate status in the next few weeks.

Logged

Don't forget what the H stands for.

reyalp

14118

Re: 1.4 development planning thread

« Reply #129 on: 17 / August / 2015, 01:44:51 »

I spent some time looking at the digic 6 patch and testing.

Overall, I don't see any problem with checking it in, I think the risk of breaking the autobuild or existing ports built with the arm-elf toolchain is small. chdkshell might break, but it's already broken. So unless there are objections, I'm for putting it in the trunk essentially as-is, and doing any required cleanup after.

Build process
I tried a full batch build with the arm-elf tool chain, to see what would happen to the autobuild if this was checked in. Seems fine. The .flt files are binary identical with the trunk except benchmark, which has code changes. The core isn't, because some of the changes affect all cams.

Also did a batch build with the eabi toolchain. On windows, this took ~1/2 as long as the arm-elf.

I tested the arm-elf and eabi builds on my cameras, using the test scripts and general CHDK options. Except for the a540 eabi issues, I didn't see any problems. The a540 problem needs to be understood before we use eabi for the autobuild, but this applies to the current trunk too.

Some other general notes
* loader/sx280/check_compat.c looks like it shouldn't be sx280 specific. It would be good to enable the bin compat checks for other cameras in 1.4 if we can.
* benchmark module change is unrelated to d6/sx280. It seems fine, no reason not to have it in trunk.
* is the d6 longjump code tested? errors in lua should exercise it, in something like chdkptp =return pcall(function() error('test') end
* update_screen_dimensions in mode_get seems like it leaves a window where CHDK could try to draw after the dimensions have changed, but before it has noticed the change. Could be OK, but maybe a risk of writing over things you shouldn't. Not clear how you'd avoid this completely if it were really a problem.
* multiple definitions of KEY_ZOOM_* for the three different speeds doesn't seem ideal. There were additional keys defined for for SX30, which should let you script them or detect them individually.
* the problem on elph130 the comments on get_target_dir_name refer to was fixed by changing the CAM_DATE_FOLDER_NAMING value, IIRC.
* There's some no longer used defines in sx280 platform_camera.h CAM_STARTUP_CRASH_FILE_OPEN_FIX, DNG_VERT_RLE_BADPIXELS, CAM_USE_SUNRISE(?)

Logged

Don't forget what the H stands for.