Long filenames - a file browser modification for Select Script File menu - page 4 - General Discussion and Assistance - CHDK Forum supplierdeeply

Long filenames - a file browser modification for Select Script File menu

  • 103 Replies
  • 45509 Views
*

Offline reyalp

  • ******
  • 14098
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #30 on: 15 / December / 2014, 00:34:56 »
Advertisements
If there are indeed such limitations, do they affect FAT/exFAT users the same way (exFAT only supports long filenames)? Should opendir check for path length?
I don't remember the exact details. The symptom was when we had
A/CHDK/SCRIPTS/EXAMPLES
a script name that pushed the total path length over 31 chars would fail, e.g. http://chdk.setepontos.com/index.php?topic=7415.0

I did some testing on D10 (DryOS R31, without the LFN patch) on a FAT16 formatted card.

You cannot use open to create a file where the total length of the filename is > 18 chars, including the dot. Same goes for mkdir.
So using ptp to upload TEST14AAAAAAAA.TXT is OK, TEST15AAAAAAAAA.TXT is not. This is true in the root directory and sub-directories. The result is open() returning an error.

If you create a >18 char filename using a card reader, you cannot access it by the long file name. The short name works.

If the total length of the path (including A/, not including the null terminator) is longer than about 59 chars things start to break. Attempting to create longer paths crashes, although the file/directory seems to get created. stat, directory listing  and reading on longer paths seems to work. I have not tested to see
So for example you can create a file
A/test/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/1234567
but
A/test/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/12345678
crashes.

I think this 59 char limit is equivalent to the 31 char limit on newer dryos, but I haven't tested yet.

It's not immediately clear how much of this matters for the LFN patch. Some thoughts
1) If the new functions returned LFNs longer than 18 chars, they wouldn't open on dryos R31.
2) You can't assume that LFNs that work on other OS's will work
3) Figuring out the exact behavior of each OS variant would be a lot of work.
4) 8.3 and short paths is still the safest bet
Don't forget what the H stands for.

*

Offline reyalp

  • ******
  • 14098
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #31 on: 15 / December / 2014, 01:47:07 »
Elph130 (droys R52) seems much more robust.
Individual path components up to 255 chars seem to work, and you can put them deep in a subdirectory tree.

Trying to create a file with a name longer than 255 chars causes open to return an error.

Once you've create a file like this, you can stat it by the long name.

This allows you to create file paths that the chdk file browser can't handle, like
Code: [Select]
A/test/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/TEST128AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.TXT
Don't forget what the H stands for.

*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #32 on: 15 / December / 2014, 02:22:40 »
This allows you to create file paths that the chdk file browser can't handle, like
Code: [Select]
A/test/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/AAAAAAAA/TEST128AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.TXT

Yet :)

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #33 on: 15 / December / 2014, 16:32:56 »
1) If the new functions returned LFNs longer than 18 chars, they wouldn't open on dryos R31.
There could be a camera-specific #define for the maximal file name length, and another for maximum path length.
Quote
2) You can't assume that LFNs that work on other OS's will work
(I assume you mean the camera OS.)
There's no reason for introducing deep paths and too long filenames in the CHDK distribution archive.
Can those unfortunate models even use scripts such as a/chdk/scripts/exam/metronome.lua? That name+path is 33 characters long (including the terminating 0) when accessed by its short name.
Quote
3) Figuring out the exact behavior of each OS variant would be a lot of work.
4) 8.3 and short paths is still the safest bet
That sounds like you're suggesting LFN support should be disabled by default? That's OK for me.

BTW, files that really need longer name support are IMHO scripts and fonts.

edit:
The crash that happens when exceeding that ~59(?) character path+name limit is a 0xc exception in the FsIoNotify task. Could be stack corruption, I guess.
« Last Edit: 15 / December / 2014, 16:38:37 by srsa_4c »


*

Offline reyalp

  • ******
  • 14098
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #34 on: 15 / December / 2014, 21:30:59 »
There could be a camera-specific #define for the maximal file name length, and another for maximum path length.
This would probably be desirable. Working out the exact limits isn't really straightforward.
Quote
Quote
2) You can't assume that LFNs that work on other OS's will work
(I assume you mean the camera OS.)
I actually meant PC OSes, but yes it appears different cameras have different limits too.
Quote
There's no reason for introducing deep paths and too long filenames in the CHDK distribution archive.
Agreed.
Quote
Can those unfortunate models even use scripts such as a/chdk/scripts/exam/metronome.lua? That name+path is 33 characters long (including the terminating 0) when accessed by its short name.
I thought it was tested with a full 8.3 name when the change was made, but now I'm not sure. I don't have one of these cameras.

Quote
That sounds like you're suggesting LFN support should be disabled by default? That's OK for me.
I'd prefer not to have it off by default, and indeed not even an option if it works reliably. If we can't make it work reliably across the supported cameras, then it should be off by default.

Quote
edit:
The crash that happens when exceeding that ~59(?) character path+name limit is a 0xc exception in the FsIoNotify task. Could be stack corruption, I guess.
I didn't save the romlog from every crash, but one I got on D10 was
Code: [Select]
ASSERT!! CobjMngr.c Line 357
Occured Time  2014:12:14 20:10:40
Task ID: 15204384
Task name: FsIoNotifyTa0
SP: 0x0030E740
Don't forget what the H stands for.

*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #35 on: 15 / December / 2014, 21:50:38 »
Quote
edit:
The crash that happens when exceeding that ~59(?) character path+name limit is a 0xc exception in the FsIoNotify task. Could be stack corruption, I guess.
I didn't save the romlog from every crash, but one I got on D10 was
Code: [Select]
ASSERT!! CobjMngr.c Line 357
Occured Time  2014:12:14 20:10:40
Task ID: 15204384
Task name: FsIoNotifyTa0
SP: 0x0030E740

This error can also be triggered by an existing (but rare) issue with the file browser code.
There are global memory buffers that are used by both the GUI and KBD task code.

While cleaning up the file browser code I managed to make this happen fairly often - I'm currently looking at ways to remove the issue (or minimize it further).

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline reyalp

  • ******
  • 14098
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #36 on: 15 / December / 2014, 23:09:04 »
From discussion with waterwingz in IRC, it appears the R39 max file length and max total path length are both 32 chars (excluding the null)
Open returns an error for longer names.

So the EXAM scripts are fine with an 8.3 name, but a long name in that directory would not work.

Files with names longer than 32 chars in the root directory (created in a card reader) are returned by the os read dir functions, so it will list files by names that can't be used to open.

This above different from DryOS R31, which appears to always list short names only (on a 4GB FAT16 anyway)
Don't forget what the H stands for.

*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #37 on: 16 / December / 2014, 07:11:26 »
Another potential benefit of this patch.

On the G12, with the current file system, a directory with a long name and not in the root directory can't be deleted using the 8.3 short version of the name. The call to remove fails.

With srsa_4c's patch the call to remove() works with the long name.

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)


*

Offline srsa_4c

  • ******
  • 4451
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #38 on: 16 / December / 2014, 19:01:09 »
Quote
Quote
2) You can't assume that LFNs that work on other OS's will work
(I assume you mean the camera OS.)
I actually meant PC OSes, but yes it appears different cameras have different limits too.
I'm trying to address compatibility issues by heavily restricting the set of characters allowed in the long names. If any unallowed characters show up in the LFN, the routine will return the short name instead (short names are not currently checked for "safe" chars, so the returned short name may still be unusable for the camera - but, as you can see in some earlier posts, the camera routines also choke on those names).

On the G12, with the current file system, a directory with a long name and not in the root directory can't be deleted using the 8.3 short version of the name. The call to remove fails.

With srsa_4c's patch the call to remove() works with the long name.
Interesting... Looks like those fw routines were not tested for conditions that were not meant to happen.

BTW (if I did not mention this yet), the switch to new "ReadFDir" routines happened between r31 and r39 (the sx120 seems to be the only r39 model that doesn't have the new routines, it has "OldReadFDir.c" instead of "ReadFDir.c"). I think all models with the new routines support exFAT, even if SDXC support is not stated in the manual.

*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Long filenames - a file browser modification for Select Script File menu
« Reply #39 on: 19 / December / 2014, 04:35:44 »
Updated patch against 1.4 revision 3820.

I think this should be added to SVN for 1.4 - any objections?

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

 

Related Topics


SimplePortal 2.3.6 © 2008-2014, SimplePortal