1.4 development planning thread

reyalp

14111

Re: 1.4 development planning thread

« Reply #170 on: 26 / September / 2015, 15:26:33 »

Advertisements

Quote from: srsa_4c on 21 / September / 2015, 14:51:38

That would probably be nice but hand-editing lots of files would risk breaking some ports.

Agreed.

Quote

For the parallel stubs-rebuild:
...
Perhaps better than making a copy of that script.

That seems fine to me (without actually trying it

)

Quote

Finally managed to try GNU parallel on Windows (Vista x86, on a Cygwin based install), with success. Little problem is that recent Cygwin no longer includes stricmp in strings.h, so the sigfinder source needs a trivial fix (and the makefiles don't recognize Cygwin either, so that also needed a fix).

I have no objection to adding cygwin fixes if they don't break anything else. I wouldn't want cygwin to be required.

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: 1.4 development planning thread

« Reply #171 on: 26 / September / 2015, 19:59:46 »

Quote from: reyalp on 26 / September / 2015, 15:26:33

Quote
Finally managed to try GNU parallel on Windows (Vista x86, on a Cygwin based install), with success. Little problem is that recent Cygwin no longer includes stricmp in strings.h, so the sigfinder source needs a trivial fix (and the makefiles don't recognize Cygwin either, so that also needed a fix).
I have no objection to adding cygwin fixes if they don't break anything else. I wouldn't want cygwin to be required.

That was meant to be more like a hint in case someone would like to try. I did not test that setup beyond this very specific trial.

I committed the parallel patch. Although I have not tried (I don't dare to) the old Windows specific method, I believe that it should still work on Win versions it used to.

Logged

philmoz

3450

Re: 1.4 development planning thread

« Reply #172 on: 26 / September / 2015, 20:48:43 »

Quote from: srsa_4c on 26 / September / 2015, 19:59:46

Although I have not tried (I don't dare to) the old Windows specific method, I believe that it should still work on Win versions it used to.

It hasn't worked properly since Window 8 was released.

Prior to 8, Windows would happily create hundreds of command processes, each running the finsig program. They would all eventually finish, and the entire process would use 100% CPU across all cores for the duration. The machine was pretty much useless for anything else while it was running.

In Windows 8 and 10 it seems that once a limit is reached Windows silently ignores the request to create a new process (or kills them). The result is that not all instances run and you don't get a full build. I never bothered to track down the root cause, or try and find a fix. I only use Windows now when I absolutely have to.

I have no objection to this getting removed from the build.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14111

Re: 1.4 development planning thread

« Reply #173 on: 27 / September / 2015, 18:27:47 »

After some thought and testing, I've asked hacki to switch the trunk autobuild server to use an eabi toolchain from https://launchpad.net/gcc-arm-embedded/+download

Reasoning:
1) Switching now allows us to add digic 6 cams to the "stable" autobuild if they get to an autobuild-worthy state, without having to switch compilers.
2) Since the compiler change may have unexpected impacts (like http://chdk.setepontos.com/index.php?topic=12512.0) it's better to switch before the release.
3) If we switch now, the autobuild will only need one toolchain after the 1.4 release. If we kept 1.4 on the old toolchain, we'd probably need to switch the 1.5 trunk at some point.
4) Switching compiler / ABI after release also goes against the philosophy of the "stable" branch, especially considering the modules aren't compatible.
5) The newer compilers generate substantially smaller code, which is a nice bonus.

I'm not sure exactly when the switch will happen, hacki said he will try for next weekend. It's possible the autobuild will be broken in the process.

Logged

Don't forget what the H stands for.

reyalp

14111

Re: 1.4 development planning thread

« Reply #174 on: 03 / October / 2015, 19:03:52 »

Continuing the slow crawl to release, I've spent time looking for loose ends

Script parameters changes: http://chdk.setepontos.com/index.php?topic=12121.0

Quote from: philmoz

- There is no error checking when parsing the script parameters (the current system does not have any either). This should probably be fixed.

This would be valuable, especially with the more sophisticated options now available, I've had errors that weren't obvious in the rawop scripts. I'm not sure what the appropriate response to errors would be though: Add a warning message like the compatibility one, put something in the console?

Long file name thread support http://chdk.setepontos.com/index.php?topic=10833.0
There was discussion about the menu option to enabled / disable CHDK LFN support: Should it stay in release? Should script have a way of selecting long vs short?
There is also some unresolved (?) issues with length limits.

Another thing I'd like to do is document compatibility:
Script
* Should be generally compatible, with @chdk_version handling most API changes
* 1.4 Lua API is different. Return values changed for get_raw get_video_button, get_focus_ok, save_config_file, load_config_file, set_mf. Some other functions also accept either boolean or number when they previously expected one or the other, but this shouldn't affect most scripts.
* Mode map changes (http://chdk.setepontos.com/index.php?topic=12163.msg119906#msg119906) mean some CHDK mode numbers have changed, and some names have been removed. Shouldn't affect common ones like M, P, AUTO
* LFN support affects Lua file / directory functions.
* slight change in shoot hook behavior, only become "active" is hook was set, previously would have done so for an very short time if not set.

CFG files
* Generally compatible, (? AFAIK)
* We didn't move to new cfg filenames, 1.4 will use existing 1.3.
* conf.disable_lfn_parser is the only conf setting I'm aware of being added, conf.script_vars removed.
* set_config_value backward compatibility (even to 1.2 numbers) should still work.
* palette changes might impact color settings?

Modules
1.3 modules should be assumed to be incompatible with 1.4. Also modules now come in multiple ABI and CPU target flavors, which must match the core.

PTP
No protocol changes.
edit:
However, PTP scripts don't have @chdk_version and default to 1.4 behavior, so may need to be updated. I need to check if chdkptp is affected.

« Last Edit: 03 / October / 2015, 20:45:41 by reyalp »

Logged

Don't forget what the H stands for.

srsa_4c

4451

Re: 1.4 development planning thread

« Reply #175 on: 03 / October / 2015, 20:55:43 »

Quote from: reyalp on 03 / October / 2015, 19:03:52

Long file name thread support http://chdk.setepontos.com/index.php?topic=10833.0
There was discussion about the menu option to enabled / disable CHDK LFN support: Should it stay in release? Should script have a way of selecting long vs short?
There is also some unresolved (?) issues with length limits.

I'd vote for keeping the menu option.
Potential (requested) changes include:
- LFN switch for script use (if it is needed), this only affects the wrapper functions
- do not return hidden files
- do not return long filenames if filename + path longer than ...
This is probably the most important one and it's also complicated: how do we determine the limits? I don't think I've seen any complaints about (for example) metronome.lua being inaccessible on certain cameras. We also can't really expect user feedback.
LFN discussion should IMHO continue in its own thread.

Quote from: reyalp on 03 / October / 2015, 19:03:52

PTP scripts don't have @chdk_version and default to 1.4 behavior, so may need to be updated. I need to check if chdkptp is affected.

Other CHDK-compatible PTP client software might also be affected by changes to scripting, although @chdk_version might be the only impact at the moment.

Logged

msl

1280
A720 IS, SX220 HS 1.01a

Re: 1.4 development planning thread

« Reply #176 on: 04 / October / 2015, 08:04:14 »

There are still a number of alpha and beta entries without actual porting work. That should be cleaned up before a new version is released. Which is important for normal CHDK users.

The script functions set-zoom() and set_focus() are not yet fully stable.
- lens distortion in JPGs for some cameras
- crashes by set_zoom() in MF mode (probably are affected cameras with CAM_USE_ALT_SET_ZOOM_POINT)
- setting subject distance > 65535, needed e.g. for hyperfocal distance with focal length >= 40 mm

msl

Logged

CHDK-DE: CHDK-DE links

reyalp

14111

Re: 1.4 development planning thread

« Reply #177 on: 04 / October / 2015, 14:00:18 »

Quote from: msl on 04 / October / 2015, 08:04:14

There are still a number of alpha and beta entries without actual porting work. That should be cleaned up before a new version is released.

I'm not sure what you mean. Individual ports being, alpha, beta etc shouldn't be connected to the release, and in any case "finishing" ports or determining whether the alpha/beta status should be removed depends on people with the individual cameras.

Some time back we removed the "alpha" status from ports just because they were old, but this turned out to be unhelpful, since some of them were quite broken.

Quote

The script functions set-zoom() and set_focus() are not yet fully stable.
- lens distortion in JPGs for some cameras
- crashes by set_zoom() in MF mode (probably are affected cameras with CAM_USE_ALT_SET_ZOOM_POINT)
- setting subject distance > 65535, needed e.g. for hyperfocal distance with focal length >= 40 mm

These are long standing CHDK issues, and again need development per-port if they are fixable at all.

If you can identify specific ports that have these problems and have testers to test them, maybe some progress can be made, but I don't think this needs to be tied to the 1.4 release. Fixing port specific bugs in the stable branch is fine.

What I'm trying to do for 1.4 is make sure that the new stuff we added isn't broken in a way that's going to cause us big headaches later on, and the 1.4 isn't more broken than 1.3 any significant ways.

Logged

Don't forget what the H stands for.

msl

1280
A720 IS, SX220 HS 1.01a

Re: 1.4 development planning thread

« Reply #178 on: 04 / October / 2015, 18:07:39 »

Quote from: reyalp on 04 / October / 2015, 14:00:18

I'm not sure what you mean. Individual ports being, alpha, beta etc shouldn't be connected to the release, and in any case "finishing" ports or determining whether the alpha/beta status should be removed depends on people with the individual cameras.

And I'm not sure that I understand correctly. for me it makes no sense to keep for years an alpha state for a camera, although there are no negative reports.
The end user checks a new release especially for new cameras and their status and not necessarily because of new functions.

Quote from: reyalp on 04 / October / 2015, 14:00:18

These are long standing CHDK issues, and again need development per-port if they are fixable at all.

This is not all dependent of single camera ports. E.g., the subject distance limits are a common topic. But no one wants to talk about it. Especially photo technical themes should not be neglected.

But your release plans are fine. Make it.

msl

Logged

CHDK-DE: CHDK-DE links

reyalp

14111

Re: 1.4 development planning thread

« Reply #179 on: 04 / October / 2015, 18:36:01 »

Quote from: msl on 04 / October / 2015, 18:07:39

And I'm not sure that I understand correctly. for me it makes no sense to keep for years an alpha state for a camera, although there are no negative reports.

I agree. OTOH, we changed some like this a while back, and later users found at least one was very broken (ixus300 I think).

I don't have a good solution

Ideally we'd have a comprehensive test suite ordinary users could run, but this would take a lot of work to implement.

Quote from: reyalp on 04 / October / 2015, 14:00:18

This is not all dependent of single camera ports. E.g., the subject distance limits are a common topic. But no one wants to talk about it. Especially photo technical themes should not be neglected.

I thought for subject distance, you can set > 64k if the port has the correct defines? If there's a thread about this somewhere, please feel free to bump it, I know I lose track of things. I do think photographic features should be a priority.

Logged

Don't forget what the H stands for.