Camera object lists (image list) - DryOS Development - CHDK Forum  

Camera object lists (image list)

  • 7 Replies
  • 5823 Views
Camera object lists (image list)
« on: 22 / September / 2010, 07:26:28 »
Advertisements
Hello.
Did anyone tried to reverse logic of camera image lists?
I found on the wiki that camera uses 24 bytes of memory for every image found on the card.
Does someone know format of this list entries or where to find them in the firmware?
I am asking because when shooting JPEG+CRW - CRW files becomes visible as files of unknown format attached to JPEG only after rebooting camera.
It would be great to add them to object lists during shooting, because in this case deleteing JPEG will delete paired CRW.
Does anyone have some information about this?
Or any thoughts how to find related code.
So far I have no success finding it...

It looks like related code can be found using CobjMngr.c string.
But I can not figure out the details.
« Last Edit: 22 / September / 2010, 07:49:54 by cppasm »

*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: Camera object lists (image list)
« Reply #1 on: 22 / September / 2010, 13:52:51 »
Not really...but I am interested in it mainly to find a way to limit file table RAM usage in long timelapses. They should probably only be accessed using Canon's functions to prevent corruption.

Slightly related to this, I did look into the PTP file table recently (when you hook up the camera to USB, the camera creates a new table in RAM that includes files that it wishes to show to the PC -- PTP doesn't expose files as found in the FAT, instead they are given handler numbers that seem to match offsets of files in this table). The table has 48 bytes for each file (or directory name). I decoded it a little bit very quickly, without trying more than a couple of RAM dumps with a few photos on the card before losing interest. So much of what's below is unconfirmed guesswork:

Table contents (byte offset from file name start, little endian):
0--11 (12 bytes) file name 8.3

12 (1 byte) unknown, always zero

13 (1 byte) seems to contain
0x20 for JPEG and FIR files,
0x10 for ???CANON directories and
0x00 for A,V,MISC.

14--15 (2 bytes) unknown. always zero

16--19 (4 bytes) file size, uint32

... (4 unknown bytes)

24 (1 byte) uncertain: encoded year of capture, uint8,
format: range 000--255, encoded as YXX, where Y is century (0 for 19XX, 1 for 20XX, 2 for early 21XX) and XX is year offset.  Example: 110 is year 2010.

25 (1 byte) uncertain: month of capture in range 0--11, uint8

26 (1 byte) uncertain: day of month of capture in range 1--31, uint8

27 (1 byte) hour of capture in local time zone, uint8

28 (1 byte) minute of capture, uint8

29 (1 byte) second of capture, uint8

... (6 unknown bytes)

36--39 (4 bytes), horizontal resolution (pixels), uint32

40--43 (4 bytes), vertical resolution (pixels), uint32

... (a bunch of unknown bytes)


It is possible that file name doesn't start the entry of each file, but this assumption is not far from correct (stuff described above, dates and such, that comes soon after the file name does belong to that file and not the next one).

The date fields seem to apply to photos only, directories or PS.FIR or video files do not have valid dates (on PC clients, those seem to get the date of the latest photo). PS.FIR date is all zeroes, but for directories there's some data.

Re: Camera object lists (image list)
« Reply #2 on: 22 / September / 2010, 14:55:19 »
Well, this is something...
But as I can see - table entry should contain filename in plain text.
I have tried to create memory dump and search for file names, but could not find any.
I will try once more in the near future.
But maybe camera uses different table format for play mode and for ptp connection (I think this is unlikely, but posible).

I have found the table of pointers to some structures used by CobjMngr - will check this out later.

*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: Camera object lists (image list)
« Reply #3 on: 22 / September / 2010, 15:51:16 »
Well, this is something...
But as I can see - table entry should contain filename in plain text.
I have tried to create memory dump and search for file names, but could not find any.
I will try once more in the near future.
But maybe camera uses different table format for play mode and for ptp connection (I think this is unlikely, but posible).

I have found the table of pointers to some structures used by CobjMngr - will check this out later.

If our old assumption of 24 bytes per file is correct, it's most definitely a different format since this PTP file list uses double that. Actually I was surprised to find plain text file names, I would have expected file names to be packed to save RAM.

For example, for a single 2000-entry table per each xxxCANON directory whose contents have been indexed to RAM, it would make sense to encode file names in some simple fashion such as to a single 32-bit word containing a 16-bit number (e.g. 1234 from IMG_1234.JPG and a single byte or less defining file type JPG, CRW, AVI etc -- there's plenty of bits for miscellaneous flags left in that word even after the file name). After all, these cameras have very strict file naming conventions if you want your files to appear in PLAY mode.

In my experience the camera indexes all file tables that it can fit in RAM from the SD card when you enter PLAY mode for the first time (i.e. you can prevent this by booting to REC mode). But I believe the camera does index the current directory (and only it) when booting directly to REC mode since AFAIK the index gets updated each time you shoot a photo so there must be a table for that available. Or maybe it doesn't do it until you take a photo, in order to minimize startup time.

To search for file tables, you could boot camera in rec mode and take a photo, take a ram dump, then take a bunch of more photos and take another dump and inspect their binary diff on a PC. Or alternatively take a bunch of photos, shut down camera, boot to play mode, take ram dump, power off, remove some photos using a PC and a card reader, put card back in, boot in play mode, take ram dump and then binary diff to the previous dump (I'd do all this booting stuff this way to minimize unrelated changes in RAM not related to file tables).



Re: Camera object lists (image list)
« Reply #4 on: 18 / February / 2011, 15:10:05 »
Can somebody help me with reversing this stuff?
I've a little bit stuck right now.
Trying to dump memory in record and playback mode, but can't find anything...
There are image file names in the dump, but they all are FAT directory entries (FS cache).
This is 100% true - I've analyzed them and it is obviously fat directory entries.
But I still can't find image lists neither in the memory dumps nor in the firmware disassembly (code for working with them).
Any thoughts and help are welcome and highly appreciated :)

One more fact - initial lists are created either in the InitFileModules task or in the tasks created from it (maybe CtgTotalTask).

PS: all this stuff is needed to enable syncronous RAW deletion with corresponding JPEGs without the need to restart camera (they are not linked together until camera is restarted).
« Last Edit: 18 / February / 2011, 15:11:43 by cppasm »

*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: Camera object lists (image list)
« Reply #5 on: 19 / February / 2011, 07:29:15 »
Couldn't the camera be using the FAT cache as a PLAY mode index? One thing you could try is to compare RAM dumps for two different methods of JPG file deletion: PLAY mode delete and OS file delete from Lua (whose effect as we know does not propagate to play mode image list).

Re: Camera object lists (image list)
« Reply #6 on: 19 / February / 2011, 08:38:54 »
No luck so far...
Both actions marks directory entry as deleted (by setting first char of name to 0xE5), no visible differences.
At the moment I've totally changed my research direction - I'm going to hook 'delete' routine of SD card device driver and if JPG file is deleted I'm looking for RAW and deleting if it exists.
It's appears to be much easier, almost done on VxWorks camera, researching DryOS (looks like it's possible too).
The only one thing that I'm afraid of - is that's I'm going too deep.

Re: Camera object lists (image list)
« Reply #7 on: 25 / February / 2011, 08:24:57 »
Not really...but I am interested in it mainly to find a way to limit file table RAM usage in long timelapses. They should probably only be accessed using Canon's functions to prevent corruption.

Slightly related to this, I did look into the PTP file table recently (when you hook up the camera to USB, the camera creates a new table in RAM that includes files that it wishes to show to the PC -- PTP doesn't expose files as found in the FAT, instead they are given handler numbers that seem to match offsets of files in this table). The table has 48 bytes for each file (or directory name). I decoded it a little bit very quickly, without trying more than a couple of RAM dumps with a few photos on the card before losing interest. So much of what's below is unconfirmed guesswork:

Table contents (byte offset from file name start, little endian):
0--11 (12 bytes) file name 8.3

12 (1 byte) unknown, always zero

13 (1 byte) seems to contain
0x20 for JPEG and FIR files,
0x10 for ???CANON directories and
0x00 for A,V,MISC.

14--15 (2 bytes) unknown. always zero

16--19 (4 bytes) file size, uint32

... (4 unknown bytes)

24 (1 byte) uncertain: encoded year of capture, uint8,
format: range 000--255, encoded as YXX, where Y is century (0 for 19XX, 1 for 20XX, 2 for early 21XX) and XX is year offset.  Example: 110 is year 2010.

25 (1 byte) uncertain: month of capture in range 0--11, uint8

26 (1 byte) uncertain: day of month of capture in range 1--31, uint8

27 (1 byte) hour of capture in local time zone, uint8

28 (1 byte) minute of capture, uint8

29 (1 byte) second of capture, uint8

... (6 unknown bytes)

36--39 (4 bytes), horizontal resolution (pixels), uint32

40--43 (4 bytes), vertical resolution (pixels), uint32

... (a bunch of unknown bytes)


It is possible that file name doesn't start the entry of each file, but this assumption is not far from correct (stuff described above, dates and such, that comes soon after the file name does belong to that file and not the next one).

The date fields seem to apply to photos only, directories or PS.FIR or video files do not have valid dates (on PC clients, those seem to get the date of the latest photo). PS.FIR date is all zeroes, but for directories there's some data.


is this the structure that is return from a readfastdir call ?

If so, then in chdk source this size should increase.see here

http://chdk.setepontos.com/index.php?topic=5980.msg60700#msg60700
Ixus 1000 HS


 

Related Topics