Combining header and raw files. - page 2 - General Discussion and Assistance - CHDK Forum supplierdeeply

Combining header and raw files.

  • 14 Replies
  • 5494 Views
Re: Combining header and raw files.
« Reply #10 on: 30 / December / 2009, 05:27:59 »
Advertisements
Hmm, then why save the header separately  ?
For the application that I am working on, there will typically be 200 raw images.
These are just a small section of the full image so are quickly saved (and reversed).
Because they are a small section, no bad-pixel removal is done.
To minimise the amount of code, no EXIF information is saved.
Two header files are created.
One is combined with each of the 200 images to create 200 DNG files.
The other header is attached to 200 joined raw files to create a single DNG image.
CS4 will display both, including thumbnails.

For greater speed and to avoid extra code in the camera, it would be better if a separate programme could run on the PC that reversed the bytes and created the headers (and did bad-pixel repair).
I don't think I know enough about programming to do that, certainly not with a GUI.
The camera could just write required information to a text file that the PC programme would use.
Memory is a severe problem with the IXUS100 if edge-outline and certain SDM features are to be supported.
That is why the number of features and amount of code has to be minimised.


I have one really irritating problem with the modified CHDK code  ..
I am using a previously commented-out entry (number 34) in the conf_info[] array of ConfInfo structures.
This entry is NEVER remembered after turning the camera off.

A new entry added to the end of the list is remembered.
« Last Edit: 30 / December / 2009, 05:37:45 by Microfunguy »

*

Offline ewavr

  • ****
  • 1057
  • A710IS
Re: Combining header and raw files.
« Reply #11 on: 30 / December / 2009, 06:04:51 »
How much memory can you allocate in the other memory areas before problems start ?
On A710 approx 17 Mbytes without camera crash, but all camera operation at such low memory are very-very slow  :).
Quote
The other thought (or possibly combined with this) is reversing one block, start writing, and reverse the next block while the first is writing. ISTR you tried this before and ran into problems, but the hardware must be capable of it ?
If I understand you correcly (I'm not sure about this even with Google translate  :) ), I tried something like this long time ago, but total time remains the same.
With "delayed raw" user can review image or prepare next shoot while RAW is written, for serial shooting this option is useless.

Re: Combining header and raw files.
« Reply #12 on: 30 / December / 2009, 09:52:43 »
How about a command-line tool that finds all files in a folder with 'raw' extension and reverses the bytes ?

Personally, I would then just save raw and the headers.

A batch files would run the reversal command and then join the files to the headers.
« Last Edit: 30 / December / 2009, 09:57:06 by Microfunguy »

*

Offline reyalp

  • ******
  • 13505
Re: Combining header and raw files.
« Reply #13 on: 30 / December / 2009, 16:17:50 »
On A710 approx 17 Mbytes without camera crash, but all camera operation at such low memory are very-very slow  :).
Wow. This could be very useful for other things too.
Quote
If I understand you correcly (I'm not sure about this even with Google translate  :) ), I tried something like this long time ago, but total time remains the same.
write() waits until write is complete, but I think the actual write is done by SD controller with DMA, so you should be able to use the CPU while it is writing. Since reversebytes takes less time than writing raw, we should be able to reverse some bytes, start those writing, and reverse more while the write is going on. Total time should be almost the same as normal raw.

vxworks has asynchronous IO functions, but I'm not sure if canon implements them or does their own thing.
Quote
With "delayed raw" user can review image or prepare next shoot while RAW is written, for serial shooting this option is useless.
Agree, this is a good improvement for general shooting.

For greater speed and to avoid extra code in the camera, it would be better if a separate programme could run on the PC that reversed the bytes and created the headers (and did bad-pixel repair).

I don't think I know enough about programming to do that, certainly not with a GUI.
This should be quite simple compared to most CHDK code. PC is fast enough you don't need special ASM to reverse the bytes, so it's just read CRW, reverse bytes, fix zero pixels (you can re-use the CHDK readpixel/writepixel code) read header info, write everything.  I can post code for this if it would be helpful.

For GUI, there are some simple frameworks that do most of this for you.
Quote
I have one really irritating problem with the modified CHDK code  ..
I am using a previously commented-out entry (number 34) in the conf_info[] array of ConfInfo structures.
This entry is NEVER remembered after turning the camera off.
In trunk CHDK, 34 was renamed, the value is still used:
Code: [Select]
//    CONF_INFO( 34, conf.ns_enable_memdump,      CONF_DEF_VALUE, i:0, NULL),
    CONF_INFO( 34, conf.debug_shortcut_action,  CONF_DEF_VALUE, i:0, NULL), // backwards compatible
Basically 34 was a boolean for +/- dumping memory. I added another option (+/- pages through props/params), but because 0-1 are still valid values that have the same meaning (0 = do nothing, 1 = dump), it didn't need a new conf item. I just renamed the variable to make things clearer.

Quote
A batch files would run the reversal command and then join the files to the headers
Yes, you can do this. The reasons I don't like separate headers:
1) 2x the number of files. If you have a lot of files (very long timelapse for example) this affect the camera performance and available memory a lot. Maybe not if you store them outside of the DCIM tree.
2) transfer over PTP becomes more difficult, because you need 2 extension/prefix combinations that work.
3) More risk of losing one part, renaming or deleting one but not the other etc.

BTW, I'm not saying you shouldn't use this, just explaining why I don't like it for CHDK.
Don't forget what the H stands for.


Re: Combining header and raw files.
« Reply #14 on: 30 / December / 2009, 16:40:29 »
I can post code for this if it would be helpful.

Thanks, I was hoping you would say that  :)

I have no idea how to write stand-alone programmes, I have never done it.
Quote
For GUI, there are some simple frameworks that do most of this for you.

Yes, another thing to learn.
What would be the quickest and easiest ?

Quote
BTW, I'm not saying you shouldn't use this, just explaining why I don't like it for CHDK.

Sure. For my application there are just two different header files for typically 200 partial images.

I have created a new conf entry, that works fine.

 

Related Topics