Long filenames, how?

  • 4 Replies
  • 4489 Views
*

Offline jeff666

  • ****
  • 181
  • A720IS
Long filenames, how?
« on: 28 / January / 2008, 17:34:04 »
Advertisements
Right now filenames in dryos are limited to the old 8+3 DOS-limits. That's mainly, because readdir used to return 100 byte long filenames on VxWorks but DryOS doesn't and nobody tried to fetch the missing information (yet :D).

From back in the old days 8) I remember long filenames in FAT are implemented as several distinct directory entries. Did anyone check if readdir returns those follow-up-entries? If it does, we should be able to stitch several entries to one filename.

Ewavr, you found readdir. Any comments?

Cheers.

*

Offline RyeBrye

  • **
  • 73
  • SD-870
Re: Long filenames, how?
« Reply #1 on: 28 / January / 2008, 17:55:09 »
This wont directly answer your question, but is some additional background information...

Here is some reference information on the LFN implementation in FAT:

(source: http://en.wikipedia.org/wiki/File_Allocation_Table#Long_file_names )

Byte Offset     Length    Description
0x001Sequence Number
0x0110Name characters (five UTF-16 characters)
0x0b1Attributes (always 0x0F)
0x0c1Reserved (always 0x00)
0x0d1Checksum of DOS file name
0x0e12Name characters (six UTF-16 characters)
0x1a2First cluster (always 0x0000)
0x1c4 Name characters (two UTF-16 characters)

*

Offline ewavr

  • ****
  • 1057
  • A710IS
Re: Long filenames, how?
« Reply #2 on: 28 / January / 2008, 18:32:20 »
Ewavr, you found readdir. Any comments?

Not readdir(), but ReadFastDir() (also exists in VxWorks cameras). ReadFastDir returns only file and directory names, nothing else (volume label, deleted files entries, long filenames).
 We can:
* continue readdir() search (I have no hope that we can find it)
* search shortfilename->longfilename function
* modify ReadFastDir(). For example (a720,  part of original ReadFastDir):
Code: [Select]
ROM:FFDC1BC0                 LDRB    R1, [R0,#0xB]  ;  read attributes byte
ROM:FFDC1BC4                 CMP     R1, #0xF          ;  is long filename?
ROM:FFDC1BC8                 BEQ     loc_FFDC1BF8  
ROM:FFDC1BCC                 TST     R1, #8             ;  is volume label ?
ROM:FFDC1BD0                 BNE     loc_FFDC1BF8  
ROM:FFDC1BD4                 LDRB    R1, [R0]          
ROM:FFDC1BD8                 CMP     R1, #0xE5        ;  is deleted file entry ?
ROM:FFDC1BDC                 BEQ     loc_FFDC1C08  

Function is small, not uses global variables and only one subroutine and, I think, can be rewritten fully in C language.

upd: It seems that OpenFastDir()  calls Open(directory_name, 0, 0x124), ReadFastDir reads entries from this file... If this is true, we don't need any xxxFastDir(), we can make own  :)
« Last Edit: 28 / January / 2008, 19:00:37 by ewavr »

*

Offline jeff666

  • ****
  • 181
  • A720IS
Re: Long filenames, how?
« Reply #3 on: 28 / January / 2008, 19:52:23 »
Quote from: ewavr
upd: It seems that OpenFastDir()  calls Open(directory_name, 0, 0x124), ReadFastDir reads entries from this file... If this is true, we don't need any xxxFastDir(), we can make own  :)

I support this idea, but it's to late to do any investigations - I need some sleep.

I'll do some tests tomorrow after work. You will certainly check if VxWorks behaves identical, I assume ;)

update:

I did some tests and successfully found out that I'm not a C programmer  :haha

This goes into include/stdlib.h
Code: [Select]
typedef struct {
    char                fname[8];
    char                fext[3];
    char                attr;           // is there a numeric one-byte-datatype?
    char                reserved[10];
    short               time;
    short               date;
    short               start_cluster;
    int                 fsize;
} fat_de;

typedef struct {
    char                seqno;          // numeric
    char                fname_p1[10];
    char                attr;           // numeric
    char                reserved;
    char                chksum;
    char                fname_p2[12];
    short               start_cluster;  // always 0 for lfn-entries
    char                fname_p3[4];
} fat_lfn;

And this goes into spytask
Code: [Select]
if (cnt == 100) {
        char *de_char;     // one directory entry
        fat_de *de;        // one directory entry
        fat_lfn *lfn;      // long filename entry

        char cur_de[50];   // text-output of current entry
        char lfn_part[255];// long filename

        int dh = open("A/CHDK", 0, 0444);

        de_char = malloc(32);
        de = (fat_de*) de_char;
        lfn = (fat_lfn*) de_char;

        if (dh > 0) {
                int lnr = 0;
                while (read(dh, de_char, 32) > 0) { // get next entry

                        if (lnr > 15)           { continue; } // can't display more
                        if (de_char[0] == 0)    { continue; } // empty entry
                        if (de_char[0] == 0xe5 ){ continue; } // deleted entry

                        if ((int)de->attr == 0x0f) { // lfn
                                char tmp[12];

                                // new lfn, truncate string
                                if ((int) lfn->seqno == 1) { lfn_part[0] = 0; }

                                // add pieces of filename from lfn-entry to previous entry
                                strncpy(tmp, lfn->fname_p1, 10);   
                                strcat(lfn_part, tmp);
                                strncpy(tmp, lfn->fname_p2, 12);
                                strcat(lfn_part, tmp);
                                strncpy(tmp, lfn->fname_p3, 4);
                                strcat(lfn_part, tmp);

                                // this doesn't work (displays unusual characters)
                                sprintf(cur_de, "%s :%x", lfn_part, de->attr);
                                draw_txt_string(0, lnr++, cur_de, 0xff);
                        }
                        else {  // regular filename
                                sprintf(cur_de, "%s :%x", de->fname, de->attr);
                                draw_txt_string(0, lnr++, cur_de, 0xff);
                        }
                }
        } else {
                strcpy(cur_de, "dopen failed.");
                draw_txt_string(4, 4, cur_de, 0xff);
        }

        close(dh);
        free(de_char);
}

We do indeed get raw directory entries and can stitch together long filenames but my code doesn't show this very well. Someone who knows more about this computer-thingies is invited to fix my code.

Anyway, the basic idea of reading the directories ourselves does work and we get exactly what's in the FAT specs.

Cheers.
« Last Edit: 29 / January / 2008, 18:27:49 by jeff666 »


*

Offline RyeBrye

  • **
  • 73
  • SD-870
Re: Long filenames, how?
« Reply #4 on: 29 / January / 2008, 20:19:45 »
I just found this page that has a ton more FAT16 information - most of the stuff towards the beginning deals with the boot sector, the FAT16 system in general - towards the bottom it talks about the directory entries...

http://www.beginningtoseethelight.org/fat16/

Quote
the name is stored in unicode format. a double byte character system designed to handle all possible foreign and scientific characters. the orginal ascii characters are the same except they are two bytes in size, the second byte is a null; 00. check http://www.unicode.org for a more in depth explaination. there can be up to 13 unicode characters per 32 byte section. if however the long file name does not fill the slot exactly a unicode null (00,00) will be added to the end, followed by ff,ff's until the section is filled. note that entries do not have to have a lfn.

Does CHDK only support FAT16 file systems? If we are going to support FAT32 as well as FAT16 - things will obviously get a bit more complicated.

I'll take a stab at that code in a few minutes...

Haven't done much on the code, but found a bunch of open-source fat16 lfn implementations... One of them has some good definitions of structures and constants -

// http://www.krugle.org/kse/files/archive/archive.sourceforge.net/sleuthkit/task-1.60/src/fstools/fatfs.h
Code: [Select]

/* flags for attributes field */
#define FATFS_ATTR_NORMAL    0x00 /* normal file */
#define FATFS_ATTR_READONLY  0x01 /* file is readonly */
#define FATFS_ATTR_HIDDEN    0x02 /* file is hidden */
#define FATFS_ATTR_SYSTEM    0x04 /* file is a system file */
#define FATFS_ATTR_VOLUME    0x08 /* entry is a volume label */
#define FATFS_ATTR_DIRECTORY 0x10 /* entry is a directory name */
#define FATFS_ATTR_ARCHIVE   0x20 /* file is new or modified */
#define FATFS_ATTR_LFN       0x0f /* A long file name entry */
#define FATFS_ATTR_ALL       0x3f /* all flags set */

/* constants for first byte of name[] */
#define FATFS_SLOT_EMPTY 0x00
#define FATFS_SLOT_E5 0x05 /* actual value is 0xe5 */
#define FATFS_SLOT_DELETED 0xe5


typedef unsigned char u_int8_t;
typedef unsigned int u_int32_t;

/* directory entry short name structure */
typedef struct {
u_int8_t name[8];
u_int8_t ext[3];
u_int8_t attrib;
u_int8_t lowercase;
u_int8_t ctimeten; /* create times */
u_int8_t ctime[2];
u_int8_t cdate[2];
u_int8_t adate[2]; /* access time */
u_int8_t highclust[2];
u_int8_t wtime[2]; /* last write time */
u_int8_t wdate[2];
u_int8_t startclust[2];
u_int8_t size[4];
} fatfs_dentry;

typedef struct {
u_int8_t seq;
u_int8_t part1[10];
u_int8_t attributes;
u_int8_t reserved1;
u_int8_t chksum;
u_int8_t part2[12];
u_int8_t reserved2[2];
u_int8_t part3[4];
} fatfs_dentry_lfn;


Given the set of FAT16 constants, it's pretty obvious that there are only a few ways that the function in the firmware can read LFN's out of FAT16 - so there are only a few possible things to search for to find a function that reads directories... That being said - I'm not good enough at IDA to do it efficiently yet :)

At some point in any function, it's going to have to check two bytes  it will have to see if the attribute is set to 0x0F and then it will have to check the sequence bit...  It will also have to grab the pieces of the name in the set chunk sizes...

Would the DryOS compiler define a data structure for this the lfn? (i.e. make data structures in the compiled version that match structs the programmer wrote them in) Perhaps looking for a data structure that matched the known data structure of the fat16 lfn might help...

I'm leafing through my firmware looking and going over ARM9 as I do it. I searched for all occurrences of 0x0f - of which, there are many... but not an infinite number.

I found a few more helpful ARM things - like this quick reference:
http://www.zap.org.au/elec2041-cdrom/reference/arm-instructions-quickref.pdf

Or this paper on ARM performance that talks about the use of the semaphore in an RTOS:
 http://www.nit.gsu.unibel.by/resources/conferences%5Cesc_2004%5CSan_Francisco%5Cesc_470.pdf
« Last Edit: 30 / January / 2008, 13:19:13 by RyeBrye »

 

Related Topics