Long filenames - a file browser modification for Select Script File menu - page 6 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 2 3 4 5 [6] 7 8 9 10 11   Go Down

Long filenames - a file browser modification for Select Script File menu

  • 103 Replies
  • 48404 Views
*

Offline reyalp

  • ******
  • 14126
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #50 on: 20 / December / 2014, 19:13:27 »
Advertisements
Quote from: philmoz on 20 / December / 2014, 18:23:09
It's only if we want to allow the script to control what it sees (force 8.3 names, don't return hidden files, etc), that changes may be needed.
I would say that Lua should be able to see all files. I'm not sure what the default should be.

FWIW, vxworks appears to return hidden files no matter where they are.

Not sure we need control over 8.3 vs lfn. The current behavior is already mixed due to some DryOS showing LFNs in the root and VxWorks returning them everywhere.

Assuming we get rid of the LFN on/off menu option before release, I my feeling is that returning LFNs all the time would be fine. I would say what listdir returns should not depend on a menu setting.

Currently os.listdir / idir have a "showall" option, which exists because the low level vxworks function returns stuff like deleted entries. It also filters out the "." and ".." entries since they are almost never useful to scripts, though in hindsight combining the two might not have been wise.

Rather than a separate function to control the behavior, I'd probably go with an option, like

os.listdir('A/foo',{lfn=true,hidden=false})

Since the existing parameter is a boolean, it could be distinguished from the new style option and handled in a backward compatible way.

Having a separate function to control behavior would make lua libraries awkward.

edit:
While we're on the subject of filesystem quirks, you can stat volume labels stat('A/CANON_DC') on vxworks but they don't appear in directory listings. DryOS r31 doesn't let you stat them
« Last Edit: 20 / December / 2014, 19:23:25 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #51 on: 21 / December / 2014, 12:20:36 »
The length limit used for file renaming and directory creation will need to be revised. To make things more complicated, the limit may need to be camera generation dependent (and therefore it will have to be exported by core)... Unless someone has a better idea.
Logged
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #52 on: 21 / December / 2014, 13:50:33 »
Quote from: srsa_4c on 21 / December / 2014, 12:20:36
The length limit used for file renaming and directory creation will need to be revised. To make things more complicated, the limit may need to be camera generation dependent (and therefore it will have to be exported by core)... Unless someone has a better idea.

Not ideal; but I was going to suggest we add a warning to the file browser to tell users if they go over 32 characters total length.

A popup with a message that the result may crash the camera and allow them to continue or cancel.

For cameras that we know can handle > 32, we could have a value in the camera_info struct to disable the warning.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline reyalp

  • ******
  • 14126
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #53 on: 04 / October / 2015, 17:36:43 »
Continuing from the 1.4 planning thread
Quote from: srsa_4c on 03 / October / 2015, 20:55:43
I'd vote for keeping the menu option.
I agree, given the camera dependent uncertainties, allowing users to go back to the previous behavior is a good thing.
Quote
Potential (requested) changes include:
- LFN switch for script use (if it is needed), this only affects the wrapper functions
I think this would be desirable. Script behavior changing depending on user settings makes things more difficult for script authors, and using set_config_value to override user settings risks changing them if the script doesn't end normally.

This leads to the question: should it be an option to os.listdir and os.idir, or should or a separate  "set_directory_lfn_mode" function?

My vote is for an option, otherwise modular code would need to get, set and restore it for every call anyway.

In either case, this probably means the underlying opendir should take an option (we could probably use global flags, but this seems a bit hacky even by our standards). I think this is a reasonable way of handling it, but note it will make our opendir nonstandard http://pubs.opengroup.org/onlinepubs/007908799/xsh/opendir.html

Another wrinkle is that the option will have no effect on vxworks, since it already returns LFNs subject to it's own limits/quirks.

Quote
- do not return hidden files
I would suggest that script at least should be able to see hidden files. I would lean toward the file browser showing them as well: IMO the point of CHDK is giving you control.

In my testing DryOS R31 and R52 native readdir appear to include hidden files (attr 0x2 set), even in sub directories. Phil earlier said it DryOS didn't return hidden files in sub directories, but this may vary depending on OS, filesystem, or perhaps specifics of the file.

If we add options (presumably a bitmask) to opendir() including/excluding hidden could be another one, but if it only works with the CHDK parser or requires stat-ing each result, I'm not sure it's a good idea.

Quote
- do not return long filenames if filename + path longer than ...
  This is probably the most important one and it's also complicated: how do we determine the limits? I don't think I've seen any complaints about (for example) metronome.lua being inaccessible on certain cameras. We also can't really expect user feedback.
This is indeed the tricky one. Especially since it depends on some combination of Camera OS and filesystem.

Going back over previous posts, a least common denominator approach would limit total path length to 32, and file name length to 18. This is quite restrictive.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #54 on: 04 / October / 2015, 18:56:14 »
Quote from: reyalp on 04 / October / 2015, 17:36:43
Quote
Potential (requested) changes include:
- LFN switch for script use (if it is needed), this only affects the wrapper functions
I think this would be desirable. Script behavior changing depending on user settings makes things more difficult for script authors, and using set_config_value to override user settings risks changing them if the script doesn't end normally.

This leads to the question: should it be an option to os.listdir and os.idir, or should or a separate  "set_directory_lfn_mode" function?

My vote is for an option, otherwise modular code would need to get, set and restore it for every call anyway.
Option is fine with me (I don't think I'll ever use it, so I'm not "voting" for either).

Quote
In either case, this probably means the underlying opendir should take an option (we could probably use global flags, but this seems a bit hacky even by our standards). I think this is a reasonable way of handling it, but note it will make our opendir nonstandard http://pubs.opengroup.org/onlinepubs/007908799/xsh/opendir.html
I think we can live with being nonstandard (adding another function argument is not that bad).

Quote
I would suggest that script at least should be able to see hidden files. I would lean toward the file browser showing them as well: IMO the point of CHDK is giving you control.

In my testing DryOS R31 and R52 native readdir appear to include hidden files (attr 0x2 set), even in sub directories. Phil earlier said it DryOS didn't return hidden files in sub directories, but this may vary depending on OS, filesystem, or perhaps specifics of the file.

If we add options (presumably a bitmask) to opendir() including/excluding hidden could be another one, but if it only works with the CHDK parser or requires stat-ing each result, I'm not sure it's a good idea.
The problem is: there's just too many problems.
This option would not work on
- VxWorks cameras
- exFAT
In addition, firmware routines are inconsistent.
Do we really want to add control over this?
Quote
This is indeed the tricky one. Especially since it depends on some combination of Camera OS and filesystem.

Going back over previous posts, a least common denominator approach would limit total path length to 32, and file name length to 18. This is quite restrictive.
I really wouldn't want to limit cameras that can handle bigger path/name lengths well.
Should we add #defines to restricted models one-by-one, or should we make assumptions based on DryOS revision or something else?
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #55 on: 04 / October / 2015, 18:59:43 »
Here's a preliminary patch to move LFN parser control to a flag passed to opendir.

The gui option now just controls the file browser behavior. It should be renamed and possibly moved to a different menu if we use this approach.

Lua os.listdir ond os.idir now behave as follows
* By default, they use the CHDK LFNs (I'm open to arguments it should be the other way around)
* If the second argument is a boolean (or not present) it is treated as before, controlling the "showall" behavior.
* If the second argument is a table, it is checked for two options "showall" and "chdklfn". Setting chdklfn to false disables the CHDK LFN parser. showall is treated like the boolean second arg would be.

This approach is backward compatible (unless someone was using a table as the "true" value for showall), and allows us to add additional options easily in a way that is readable.

Other uses of opendir are set to disable the CHDK LFN parser. I didn't see any obvious benefit to using LFNs for these.
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14126
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #56 on: 04 / October / 2015, 19:04:40 »
Quote from: srsa_4c on 04 / October / 2015, 18:56:14
Should we add #defines to restricted models one-by-one, or should we make assumptions based on DryOS revision or something else?
I suspect DryOS revision is probably good enough. If we find out later that it isn't, we should be able to default based on DryOS revision, and allow ports to override if needed.

Do we also need to vary depending on the filesystem?

It would be nice to have a test script to determine the limits, although this may be difficult since hitting some causes crashes.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #57 on: 05 / October / 2015, 19:59:44 »
Quote from: reyalp on 04 / October / 2015, 19:04:40
Do we also need to vary depending on the filesystem?
If you mean "are the name/path length limits different on FAT 12/16/32 ?", I have no idea.
Therefore,
Quote
It would be nice to have a test script to determine the limits,
+1
Quote
although this may be difficult since hitting some causes crashes.
We already have at least one such script (MFtest). And, crashing in playback mode is not that bad.
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #58 on: 24 / October / 2015, 16:54:08 »
I checked in the changes from the earlier patch with some minor modifications in trunk 4277:
I made the function that takes the flags be opendir_chdk. opendir() now just calls opendir_chdk() with lfns off. This keeps opendir() portable and makes code that doesn't care (like the module menu population) simpler.
I renamed the conf variable to conf.disable_lfn_parser_ui to make it clearer it only affects the UI. Actually it only affects the file browser, but if we expose filenames elsewhere it should be controlled by the same option.
We should probably update the menu text, but it's hard to describe what it does in a menu string length.

I'll try to do some work on the limits and a test script later. One thing I my old notes reminded me is that there are cases you can't test with camera script, because other devices can create filenames that can't be created using camera functions.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #59 on: 25 / October / 2015, 15:07:13 »
Quote from: reyalp on 24 / October / 2015, 16:54:08
I'll try to do some work on the limits
A suggestion (I don't have a patch ATM):
This check could be extended with a check for name length (not computation efficient, but simple).
We could add a new member to the myDIR_s struct (name length limit), and set its value in CHDKOpenDir(), based on the camera's name and path size limits.
Logged
Pages: 1 2 3 4 5 [6] 7 8 9 10 11   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal