1.4 script compatiblity

waterwingz

12138

1.4 script compatiblity

« on: 21 / December / 2014, 11:36:54 »

Advertisements

Fundamental 1.4.0 question : Is this where we finally make a break and "fix" all the Lua functions that return 0/1 rather than false/true like they probably should ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14034

Re: 1.4 script compatiblity

« Reply #1 on: 21 / December / 2014, 17:16:11 »

Quote from: waterwingz on 21 / December / 2014, 11:36:54

Fundamental 1.4.0 question : Is this where we finally make a break and "fix" all the Lua functions that return 0/1 rather than false/true like they probably should ?

I think this is a good point to give the script API a non-backward compatible cleanup.

For things like the return values, it would be possible to to write a compatibility wrapper module in Lua, which would allow old scripts to run unchanged. This module could be automatically "require"ed for old style script. Old style scripts could be recognized either by using the old @param syntax, or by adding some kind of version indicator to the script params.

An "expected chdk version" indicator might be useful in the long run, but we'd have to keep it simple enough to actually be usable.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: 1.4 script compatiblity

« Reply #2 on: 21 / December / 2014, 17:26:07 »

Quote from: reyalp on 21 / December / 2014, 17:16:11

An "expected chdk version" indicator might be useful in the long run, but we'd have to keep it simple enough to actually be usable.

It would be nice to be able to remove this from my scripts :

Code: [Select]

bi=get_buildinfo()
chdk_version= tonumber(string.sub(bi.build_number,1,1))*100 + tonumber(string.sub(bi.build_number,3,3))*10 + tonumber(string.sub(bi.build_number,5,5))
if ( tonumber(bi.build_revision) > 0 ) then
    build = tonumber(bi.build_revision)
else
    build = tonumber(string.match(bi.build_number,'-(%d+)$'))
end

if ((chdk_version<120) or ((chdk_version==120)and(build<3276)) or ((chdk_version==130)and(build<3383))) then
    printf("CHDK 1.2.0 build 3276 or higher required")
else

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14034

Re: 1.4 script compatiblity

« Reply #3 on: 21 / December / 2014, 18:45:45 »

I'm open to suggestions for how this should work. I suspect it will take some back and forth and actually trying to use it to get right.

I've been thinking about this for a while, but don't have a fully formed solution. My basic idea is to add a script parameter like:

@version 1.4

If loaded in 1.4, everything operates as now. If loaded in an 1.3 (pretending 1.3 supported this), the user gets a warning "this script was written for a newer version of CHDK (1.4) and may not work in 1.3"

If loaded in 1.5, the behavior depends on the scope of the changes between 1.4 and 1.5. If we didn't make any incompatible script changes, then it just runs. If we made incompatible changes, but included a compatibility wrapper like a posted earlier, it automatically loads the compatibility wrapper, and perhaps displays a warning suggesting the look for an updated script. If we made incompatible changes and didn't include a compatibility wrapper, or the build is so old we've retried compatibility completely, then they get a stronger warning.

Some questions / issues
* What happens if there is no version specifier? This applies to all existing scripts, presumably targeted at 1.3 or earlier. So we could implicitly set the version to 1.3. This might be confusing for people who didn't know to put the version statement in, but the warning could be adjusted "missing @version, assuming CHDK 1.3"

* What happens if a script author wants to make a script that actually works on multiple versions, taking care of compatibility themselves. I'd be hesitant to put complicated version matching statements, but perhaps something indicating it should never warn about old versions would be useful.

* How should the warnings / messages above appear? To minimize annoyance, I'd prefer making them text that appears in the script menu, rather than a dialog you need to confirm.

* How are dev branches handles? Dev branches can always be incompatible from version to version, so I think it should be up to script authors to do build number checks if they want. Dev versions should attempt to behave correctly if an earlier version is specified, but that might be broken from time to time.

* How granular should this be? In the example above I assumed that that it only covers releases, but it would be possible to allow a full version and even build. However, the behavior would need to be different for the other components.

** We don't currently use the 3rd digit (e.g. 1.4.0) but we could use it to indicate feature additions. If these are presumed to be backward compatible, it could be treated as a minimum, so @version 1.4.0 would work on 1.4.0, 1.4.1 without warning, but @version 1.4.1 would produce a warning if run on 1.4.0

** The same could be applied to build numbers, so if a script depends on a bug fix introduced in
r4567, @version 1.4.0.4567 would warn if used on a lower build.

My fear in all of this is making something so complicated it just causes more trouble and confusion.

I've ignored ubasic above. Obviously there isn't the capability to have compatibility modules (although we could theoretically choose different flt modules) but the warning behavior could be similar.

I split this into a new thread...

« Last Edit: 21 / December / 2014, 18:49:25 by reyalp »

Logged

Don't forget what the H stands for.

msl

1280
A720 IS, SX220 HS 1.01a

Re: 1.4 script compatiblity

« Reply #4 on: 22 / December / 2014, 07:49:20 »

Quote from: reyalp on 21 / December / 2014, 18:45:45

My fear in all of this is making something so complicated it just causes more trouble and confusion.

That's also my opinion.

A function or module for a version/revision check should be enough. An update of CHDK needs some minutes.

Code: [Select]

function ckeck_version(version, revision)
...
    if not needed_version then
        print("You need a recent CHDK version")
        return false
    else
        return true
    end
end

Btw., it's not the best time to discuss about essential developments on so many fronts (changeover 1.3 => 1.4, long file name, new script header, script compatiblity, pallette system, touchscreen UI, GPS, shooting hooks).

msl

Logged

CHDK-DE: CHDK-DE links

waterwingz

12138

Re: 1.4 script compatiblity

« Reply #5 on: 22 / December / 2014, 09:09:19 »

Quote from: msl on 22 / December / 2014, 07:49:20

Quote from: reyalp on 21 / December / 2014, 18:45:45
My fear in all of this is making something so complicated it just causes more trouble and confusion.
That's also my opinion.

When I first read this through, I was also worried about that. But on reflection, something that (at a minumum) allows us to make a major incompatible update (like changing Lua function Boolean return values) so that the user is warned about the older script potentially not working should be enough? We don't have to go to the effort of wrappers for forward/backward compatability - we could live with something that warns the user "this script may not work with this version of CHDK". After that they can sort it out with an update of their CHDK version or (forum assisted) changes to the script.

Quote from: msl on 22 / December / 2014, 07:49:20

Btw., it's not the best time to discuss about essential developments on so many fronts (changeover 1.3 => 1.4, long file name, new script header, script compatibly, palette system, touchscreen UI, GPS, shooting hooks).

This might be the time of year when some people have a little more time to work on new things? If so, then I believe that's welcome. Especially as the (renewed) philosophy is that the unstable trunk is allowed to change frequently. Let's get the proposed changes rolled in and if they don't work for the group, take them out or fix them. We are not locked into keeping backwards compatibility within the dev branch anymore.

« Last Edit: 22 / December / 2014, 09:24:40 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14034

Re: 1.4 script compatiblity

« Reply #6 on: 22 / December / 2014, 14:17:51 »

Quote from: msl on 22 / December / 2014, 07:49:20

A function or module for a version/revision check should be enough. An update of CHDK needs some minutes.
Code: [Select]
function ckeck_version(version, revision) ... if not needed_version then print("You need a recent CHDK version") ...

This handles running scripts written for newer builds on old builds. E.g. if you write your script for 1.4, it's easy to detect in the script that you are running on 1.3 and throw an error or do some special workarounds.

The problem is the reverse case: A user upgrades CHDK and their scripts potentially stop working. When you write a script for 1.3, you can't know in advance if it will work for 1.4. At the moment, CHDK cannot tell what version the script was written for.
So we have some choices:
1) Maintain backward compatibility forever
2) Just warn users in the documentation that their scripts may break when they change CHDK versions, and break compatibility if we need to.
3) Come up with a way of indicating which version the script was intended for.

From a development perspective, #1 is not very good. Being able to fix mistakes in the API (like the numbers vs booleans thing) would make it easier to understand and maintain.

#2 is not very good for users, because the breakage may happen in confusing ways, and there are a lot of scripts floating.

Quote

Btw., it's not the best time to discuss about essential developments on so many fronts (changeover 1.3 => 1.4, long file name, new script header, script compatiblity, pallette system, touchscreen UI, GPS, shooting hooks).

I'm not sure what you mean? Doing all these things at the same time, or because of the holidays? I think it's OK to start the major projects for 1.4 and let the dev branch be unstable for a while. I don't expect the big questions to all be resolved immediately.

Logged

Don't forget what the H stands for.

reyalp

14034

Re: 1.4 script compatiblity

« Reply #7 on: 22 / December / 2014, 16:13:24 »

Moved discussion of general development process to the 1.4 planning thread: http://chdk.setepontos.com/index.php?topic=12133.msg119277#msg119277

Logged

Don't forget what the H stands for.

philmoz

3450

Re: 1.4 script compatiblity

« Reply #8 on: 22 / December / 2014, 16:21:20 »

Quote from: reyalp on 22 / December / 2014, 14:17:51

So we have some choices:
1) Maintain backward compatibility forever
2) Just warn users in the documentation that their scripts may break when they change CHDK versions, and break compatibility if we need to.
3) Come up with a way of indicating which version the script was intended for.

#1 would be a bad idea, and would significantly limit future improvements.

Instead of checking the version requirements in the script code why not add this to the header where we set the title and parameters.

E.G.

Code: [Select]

@requires 1.4 3838

The requirements can be checked when the header is parsed and an error dialog shown if they are not met. It would also support uBasic if needed.

This could be added easily to both 1.3 & 1.4 so anyone using either build (after the change) is protected.

The only problem then is people running older versions of 1.3 or 1.2 - in this case the documentation for the script should let people know they need to upgrade to run the latest version of the script.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14034

Re: 1.4 script compatiblity

« Reply #9 on: 22 / December / 2014, 16:32:05 »

Quote from: philmoz on 22 / December / 2014, 16:21:20

E.G.
Code: [Select]
@requires 1.4 3838The requirements can be checked when the header is parsed and an error dialog shown if they are not met. It would also support uBasic if needed.

This is essentially my suggestion in the 3rd post, right? Not that I blame you for tl;dr'ing a bit

Logged

Don't forget what the H stands for.