Long filenames - a file browser modification for Select Script File menu - General Discussion and Assistance

waterwingz

12138

Long filenames - a file browser modification for Select Script File menu

« on: 22 / October / 2013, 21:36:16 »

Advertisements

Before I spend any more time looking at this, I'd like to gauge support for changing the file browser "Select Script File" window.

My pet peeve here is that filenames in that window are truncated to 12 character (the old DOS 8.3 name format). This makes it hard to select between scripts with similar names (especially if, like me, you are in the habit of adding version numbers to names of scripts being tested).

So would it be worth changing the browser to use long file names? Having a file date and size displayed for each file might be handy in the generic file browser window but those fields seems completely useless when trying to load a script file. So what's the consensus on just displaying the long version of the script filename (with extension of course) to the maximum width of the window with no date or filesize info?

(All this assumes there is some way to get the long non-8.3 file name from the SD card directory structure.)

Of course, you could also open each file and display the @title's too / instead. But that gets more complicated.

Note : the regular File Browser window would of course retain those other fields.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14037

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #1 on: 22 / October / 2013, 22:59:44 »

Quote from: waterwingz on 22 / October / 2013, 21:36:16

Before I spend any more time looking at this, I'd like to gauge support for changing the file browser "Select Script File" window.

My pet peeve here is that filenames in that window are truncated to 12 character (the old DOS 8.3 name format). This makes it hard to select between scripts with similar names (especially if, like me, you are in the habit of adding version numbers to names of scripts being tested).

The problem is that DryOS doesn't fully support long file names. The functions we use to list directories only return short names.

VxWorks does, and IIRC CHDK displays them adequately on VxWorks cameras.

I suspect this difference is due to Canon not wanting to pay the LFN patent tax to MS http://en.wikipedia.org/wiki/File_Allocation_Table#Patent_infringement_lawsuits

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #2 on: 22 / October / 2013, 23:16:54 »

Quote from: reyalp on 22 / October / 2013, 22:59:44

The problem is that DryOS doesn't fully support long file names. The functions we use to list directories only return short names.

Thanks. Spent two happy hours reading, tinkering with, and understanding the gui_fselect.c code. That was enough - I won't look any farther.

Darn.

Logged

Ported : A1200 SD940 G10 Powershot N G16

srsa_4c

4451

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #3 on: 20 / November / 2014, 18:10:21 »

This isn't completely this thread's subject, but I didn't want to open another.

All below findings are valid for the a3200 (DryOS r47).

OpenFastDir enables returning LFN in the following cases:
- FAT12, FAT16: only in the root directory
- FAT32: never
- exFAT: always (according to the wiki it doesn't know short names, so this isn't a surprise)

When LFN is enabled, an additional 0xa000 byte buffer is allocated (standard buffer is 0x8000 bytes).
With the check overridden, ReadFastDir retrieves long filenames even on FAT32. Unfortunately, that requires copying and modifying the firmware's OpenFastDir function...
The rest of file related APIs seem to handle long filenames correctly.

edit:
Cameras with DryOS r43 or newer have the above mentioned exFAT-aware FastDir functions. The FastDir routines of earlier DryOS models IMHO ignore LFN entries. However, it's possible to make an LFN-aware opendir/readdir implementation since the details are known (and these cams don't know exFAT anyway).

So, for pre-r43 cams: own implementation, for newer cams: hacked OpenFastDir is the way to go.

« Last Edit: 21 / November / 2014, 11:27:28 by srsa_4c »

Logged

srsa_4c

4451

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #4 on: 28 / November / 2014, 18:01:05 »

In addition to the above post, some cameras (r39?) which in theory don't support exFAT do appear to have the new style FastDir functions.
Below is an excerpt from the a3200 (100d) firmware. If the branch instruction in the "exfat: lfn always" line is modified to be unconditional, then ReadFastDir will return long file names on all filesystems. This requires some additional memory as already mentioned.
The test drive was working fine on this camera. Since this requires modification to the port, it can't be applied automatically to all ports.

Code: [Select]

sub_FFA90A98:                      // OpenFastDir
          STMFD   SP!, {R3-R9,LR}
          MOV     R7, R0
          LDRB    R0, [R0]
          BL      sub_FF86E98C     // returns drive number (0 for A)
          MOV     R8, R0
          BL      sub_FF86FD1C     // returns filesystem type (1 fat12, 2 fat16, 3 fat32, 4 exfat?)
          MOVS    R5, R0
          BNE     loc_FFA90AC0
loc_FFA90AB8:
          MOV     R0, #0
          LDMFD   SP!, {R3-R9,PC}
loc_FFA90AC0:
          MOV     R0, #0x44
          BL      sub_FF83A5C4     // wrapped malloc
          MOVS    R4, R0
          MOVEQ   R1, #0x1B0
          LDREQ   R0, =0xFFA90D4C  // "ReadFDir.c"
          BLEQ    sub_FF81EC88     // DebugAssert
          CMP     R5, #4           // exFAT(?) ?
          MOV     R6, #0
          STREQ   R6, [R4, #4]
          STREQ   R6, [R4, #8]
          BEQ     loc_FFA90B0C     // exFAT, skips allocating 0x8000 bytes
          MOV     R0, #0x8000
          BL      sub_FF83A678     // AllocateUncacheableMemory
          MOV     R1, #0x8000
          STMIB   R4, {R0,R1}
          CMP     R0, #0
          LDREQ   R1, =0x1BA
          LDREQ   R0, =0xFFA90D4C  // "ReadFDir.c"
          BLEQ    sub_FF81EC88     // DebugAssert
loc_FFA90B0C:
          MVN     R0, #0
          STR     R0, [R4]
          STR     R6, [R4, #0xC]
          STR     R6, [R4, #0x10]
          STR     R6, [R4, #0x14]
          STRH    R6, [R4, #0x1A]
          CMP     R5, #4           // exfat?
          STR     R6, [R4, #0x1C]
          BEQ     loc_FFA90B48     // exfat: lfn always
          CMP     R5, #1           // fat12: lfn for root dir only
          CMPNE   R5, #2           // fat16: lfn for root dir only
          BNE     loc_FFA90BB0     // fat32: no lfn
          LDRB    R0, [R7, #2]
          CMP     R0, #0
          BNE     loc_FFA90BB0     // not root dir
loc_FFA90B48:
          MOV     R0, #0xA000
          BL      sub_FF83A678     // AllocateUncacheableMemory
...

I have also created a basic FAT/FAT32 LFN-aware opendir/readdir/closedir implementation (~700 bytes when compiled as thumb) that only requires a #define in platform_camera.h. It's tested working on the g10 and sx100. It can be used on all DryOS models which have the "old style" ReadFDir functions, newer cameras (at least the a3200) don't allow open()-ing the root directory of the card. The current version returns long names only when they consist of a rather restricted set of letters (English alphabet, numbers plus these: @.-_()$&). Falls back to short names otherwise.

I know it's risky to get opinions here these days, but: is this worth implementing?

Logged

waterwingz

12138

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #5 on: 28 / November / 2014, 18:35:51 »

Quote from: srsa_4c on 28 / November / 2014, 18:01:05

I know it's risky to get opinions here these days, but: is this worth implementing?

Definitely fits into the "would be nice" category for heavy script users. When you start getting a lot of scripts stored on your SD card, the 8.3 limitations (or the mangled LFN) make it hard to quickly determine which script is which.

Or course, once you go down this route the file browser window needs to get reworked. Frankly, I can live without file size , date , and time information when I'm looking for a particular script file. YMMV though ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

srsa_4c

4451

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #6 on: 28 / November / 2014, 18:43:02 »

Quote from: waterwingz on 28 / November / 2014, 18:35:51

Quote from: srsa_4c on 28 / November / 2014, 18:01:05
I know it's risky to get opinions here these days, but: is this worth implementing?
Definitely fits into the "would be nice" category for heavy script users. When you start getting a lot of scripts stored on your SD card, the 8.3 limitations (or the mangled LFN) make it hard to quickly determine which script is which.

Agree, the other annoying one is when you try to select an rbf font (or even some of the language files).

Quote

Or course, once you go down this route the file browser window needs to get reworked. Frankly, I can live without file size , date , and time information when I'm looking for a particular script file. YMMV though ?

I was thinking about adding a row for the full filename somewhere at the bottom. Or, alternatively, the currently selected file could use full width for the file name?

Logged

philmoz

3450

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #7 on: 28 / November / 2014, 18:52:29 »

Quote from: srsa_4c on 28 / November / 2014, 18:01:05

In addition to the above post, some cameras (r39?) which in theory don't support exFAT do appear to have the new style FastDir functions.
Below is an excerpt from the a3200 (100d) firmware. If the branch instruction in the "exfat: lfn always" line is modified to be unconditional, then ReadFastDir will return long file names on all filesystems. This requires some additional memory as already mentioned.
The test drive was working fine on this camera. Since this requires modification to the port, it can't be applied automatically to all ports.
Code: [Select]
sub_FFA90A98: // OpenFastDir STMFD SP!, {R3-R9,LR} MOV R7, R0 LDRB R0, [R0] BL sub_FF86E98C // returns drive number (0 for A) MOV R8, R0 BL sub_FF86FD1C // returns filesystem type (1 fat12, 2 fat16, 3 fat32, 4 exfat?) MOVS R5, R0 BNE loc_FFA90AC0 loc_FFA90AB8: MOV R0, #0 LDMFD SP!, {R3-R9,PC} loc_FFA90AC0: MOV R0, #0x44 BL sub_FF83A5C4 // wrapped malloc MOVS R4, R0 MOVEQ R1, #0x1B0 LDREQ R0, =0xFFA90D4C // "ReadFDir.c" BLEQ sub_FF81EC88 // DebugAssert CMP R5, #4 // exFAT(?) ? MOV R6, #0 STREQ R6, [R4, #4] STREQ R6, [R4, #8] BEQ loc_FFA90B0C // exFAT, skips allocating 0x8000 bytes MOV R0, #0x8000 BL sub_FF83A678 // AllocateUncacheableMemory MOV R1, #0x8000 STMIB R4, {R0,R1} CMP R0, #0 LDREQ R1, =0x1BA LDREQ R0, =0xFFA90D4C // "ReadFDir.c" BLEQ sub_FF81EC88 // DebugAssert loc_FFA90B0C: MVN R0, #0 STR R0, [R4] STR R6, [R4, #0xC] STR R6, [R4, #0x10] STR R6, [R4, #0x14] STRH R6, [R4, #0x1A] CMP R5, #4 // exfat? STR R6, [R4, #0x1C] BEQ loc_FFA90B48 // exfat: lfn always CMP R5, #1 // fat12: lfn for root dir only CMPNE R5, #2 // fat16: lfn for root dir only BNE loc_FFA90BB0 // fat32: no lfn LDRB R0, [R7, #2] CMP R0, #0 BNE loc_FFA90BB0 // not root dir loc_FFA90B48: MOV R0, #0xA000 BL sub_FF83A678 // AllocateUncacheableMemory ...I have also created a basic FAT/FAT32 LFN-aware opendir/readdir/closedir implementation (~700 bytes when compiled as thumb) that only requires a #define in platform_camera.h. It's tested working on the g10 and sx100. It can be used on all DryOS models which have the "old style" ReadFDir functions, newer cameras (at least the a3200) don't allow open()-ing the root directory of the card. The current version returns long names only when they consist of a rather restricted set of letters (English alphabet, numbers plus these: @.-_()$&). Falls back to short names otherwise.

I know it's risky to get opinions here these days, but: is this worth implementing?

Nice work - definitely something worth doing I would think.

Can you post a patch, or if the code changes are large create a test branch in SVN.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

waterwingz

12138

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #8 on: 28 / November / 2014, 18:53:13 »

Quote from: srsa_4c on 28 / November / 2014, 18:43:02

Agree, the other annoying one is when you try to select an rbf font (or even some of the language files).

I find the rbf fonts pretty ugly so never change from the default. But I understand how people with bad eyesight (or old glasses) could struggle with the default and want something larger. Which just makes the filename problem worse of course. Especially on older cameras - I was shocked at how bad the LCD is on my A560 when I bought it for $20 so I would have a VxWorks cameras.

Quote

I was thinking about adding a row for the full filename somewhere at the bottom. Or, alternatively, the currently selected file could use full width for the file name?

Either would be great. I was leaning towards getting rid of the other columns and just having one, or alternatively switching from three to one column mode with the DISP key when the file select menu was active. Of course, the lack of a DISP key on my treasured S100 is getting to be more and more of a pain (yes... it mostly works with the video button as an alternate. But it gets annoying deleting all those accidental three second videos from my SD card).

Logged

Ported : A1200 SD940 G10 Powershot N G16

srsa_4c

4451

Re: Long filenames - a file browser modification for Select Script File menu

« Reply #9 on: 28 / November / 2014, 20:00:49 »

Quote from: philmoz on 28 / November / 2014, 18:52:29

Can you post a patch, or if the code changes are large create a test branch in SVN.

Patch attached, it's only the "for old DryOS" part. Adding LFN support for newer cameras is a porting decision as seen above. A modified version of this work could be used on all DryOS models, but:
Cons:
- root dir is only reachable by the firmware ReadFDir functions on new cams (could fall back to system routines)
- it can't handle exFAT, could fall back to system routines if we could somehow detect the filesystem
- adds ~700 bytes to CHDK core
Pros:
- allocates less memory
- no modification is required for ports

edit:

Quote from: waterwingz on 28 / November / 2014, 18:53:13

I find the rbf fonts pretty ugly so never change from the default. But I understand how people with bad eyesight (or old glasses) could struggle with the default and want something larger. Which just makes the filename problem worse of course. Especially on older cameras - I was shocked at how bad the LCD is on my A560 when I bought it for $20 so I would have a VxWorks cameras.

Problem reappeared for DIGIC 6: overlay is 640x480, the default font is tiny.

Quote

or alternatively switching from three to one column mode with the DISP key when the file select menu was active. Of course, the lack of a DISP key on my treasured S100 is getting to be more and more of a pain (yes... it mostly works with the video button as an alternate.

This isn't a bad idea. We have other unused keys too, for example halfshoot...

dir_functions_for_early_dryos_cameras_v1.diff (14.4 kB - downloaded 40 times.)

« Last Edit: 28 / November / 2014, 20:08:06 by srsa_4c »

Logged