Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ??? - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: [1]   Go Down

Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???

  • 5 Replies
  • 9414 Views
*

Offline Lebeau

  • ***
  • 187
Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« on: 21 / December / 2010, 21:35:06 »
Advertisements
May I suggest to create a DNG file using Big-Endian "MM" (Motorola) instead of "II" (Intel) ?

Quote
JHOVE TIFF-hul Module
Header with 0x4D4D002A (if big-endian) or 0x49492A00 (if little-endian) at offset 0
Then in dng.c,
Code: [Select]
// TIFF file header
var = 0x4D4D; // big-endian endian "MM" instead of 0x4949 "II"

In raw.c,
 - generating reverse-ordered thumbnail data (???) [really not sure!]
Code: [Select]
* buf++ = b;
* buf++ = g;
* buf++ = r;
 - reversing byte-order of the DNG header
 - and saving straight Canon Raw data
Code: [Select]
reverse_bytes_order ( get_dng_header ( ), get_dng_header_size ( ) );  //including var = 0x2A !
write ( fd, get_dng_header ( ), get_dng_header_size ( ) );
write ( fd, thumbnail_buf, DNG_TH_WIDTH * DNG_TH_HEIGHT * 3 );
write ( fd, get_raw_image_addr ( ), hook_raw_size ( ) );
Ending with ONE dng_header_size (1536 bytes) instead of TWO hook_raw_size byte-reversing (31 272 480 bytes) !!! 20 360 faster !!!

Am I wrong ? Did I forget something?
I am ready to try and recompile !

P.S.: We could possibly save thumbnail data in YUV !!! PhotometricInterpretation = 6 !
« Last Edit: 21 / December / 2010, 21:38:10 by Lebeau »
Logged
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« Reply #1 on: 21 / December / 2010, 22:04:06 »
Unfortunately you can't do this.

The DNG format supports either big or little endian for the file container.

However the raw image data must be stored in big endian format if the bits per pixel is not 8, 16 or 32. (See the BitsPerSample restriction in section 3 of the DNG spec document.)

Canon P&S cameras have either 10 or 12 bits per pixel so you're stuck, you have to save the raw image data in big endian format.

Regards,
Phil.
« Last Edit: 22 / December / 2010, 19:18:51 by philmoz »
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« Reply #2 on: 21 / December / 2010, 22:10:42 »
Quote from: Lebeau on 21 / December / 2010, 21:35:06
Ending with ONE dng_header_size (1536 bytes) instead of TWO hook_raw_size byte-reversing (31 272 480 bytes) !!! 20 360 faster !!!

With some optimisations I have reduced the DNG saving overhead to 1/2 second on the SX30 camera (14MP = 21Mbyte file size).
RAW save time is ~1 sec on a class 10 SD card.
DNG save time is ~1.5 sec.

I've posted some other topics that discuss the things I have done.

Regards,
Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline Lebeau

  • ***
  • 187
Re: Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« Reply #3 on: 25 / December / 2010, 09:40:57 »
Merci beaucoup Philmoz !

Concerning Chapter 3,
Code: [Select]
BitsPerSample
Supported values are from 8 to 32 bits/sample. The depth must be the same for each sample if SamplesPerPixel is not equal to 1. If BitsPerSample is not equal to 8 or 16 or 32, then [b]the bits must be packed into bytes [/b] using the TIFF default FillOrder of 1 (big-endian), even if the TIFF file itself uses little-endian byte order.
This make restriction on bits within byte, not byte ordering, IMHO.

Code: [Select]
TIFF Tag FillOrder
Default  1 (lower column values in higher order bits)
The [b]logical order of bits within a byte[/b].
The specification defines these values:
1 = pixels with lower column values are stored in the higher-order bits of the byte.
In practice, the use of FillOrder=2 is very uncommon, and is not recommended.

And this correspond to LSB/MSB where B means bit, nor byte, IMHO.

But for now, I will concentrate on YUV video capture and may be come back on DNG generation later.

I am still convinced that a byte swap related to dir_entry's tag, type and/or count would permit reversing DNG header content instead of RAW data.

More to come some day ...  ;)
Logged
*

Offline reyalp

  • ******
  • 14140
Re: Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« Reply #4 on: 25 / December / 2010, 13:33:00 »
Quote from: Lebeau on 25 / December / 2010, 09:40:57
Concerning Chapter 3,
Code: [Select]
BitsPerSample
Supported values are from 8 to 32 bits/sample. The depth must be the same for each sample if SamplesPerPixel is not equal to 1. If BitsPerSample is not equal to 8 or 16 or 32, then [b]the bits must be packed into bytes [/b] using the TIFF default FillOrder of 1 (big-endian), even if the TIFF file itself uses little-endian byte order.
This make restriction on bits within byte, not byte ordering, IMHO.

I'm pretty sure your interpretation is wrong, but you can always try constructing a DNG with your suggested settings and see if any tool understands it correctly.
Logged
Don't forget what the H stands for.
*

Offline Lebeau

  • ***
  • 187
Re: Is DNG saving in "Big-endian" 20 360 times faster than "Little-endian" ???
« Reply #5 on: 31 / January / 2011, 11:20:58 »
Well reyalp, you were right.

I generate a perfectly big-endian formatted dng file but with 10 bits per samples, Photoshop was unable to collaborate :)
Logged
Pages: [1]   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal