Faster RAW Processing?

genosmm

« on: 24 / July / 2012, 12:31:42 »

Advertisements

My older CHDK camera is significantly slower when enable RAW.

Can this RAW processing time be significantly decreased if I switch to a faster SD card?

Did a search and while found the Benchmark Read and Write Speeds of various SD cards
http://chdk.wikia.com/wiki/Benchmarks did not find any mention concerning RAW processing speed for various SD cards.

Has anyone made a script that measures the RAW Processing Time?

Thanks

Gene

Logged

reyalp

13392

Re: Faster RAW Processing?

« Reply #1 on: 24 / July / 2012, 12:42:09 »

You can turn on "Show raw saving time" under miscellaneous->debug.

A fast SD card should help some, for CHDK raw, there really isn't any "processing", all the time is used saving the raw to disk. The raw write speed in the benchmarks should theoretically be representative of how fast it will save.

For DNG, there is some processing time too. Enabling "raw buffer cached" may speed this up.

Logged

Don't forget what the H stands for.

genosmm

Re: Faster RAW Processing?

« Reply #2 on: 27 / July / 2012, 11:06:35 »

Reyalp,

Thanks for the "Show raw saving time" under miscellaneous->debug" tip.

Using those key words did a forum search and found "To display the time needed to save RAW files, just enable Show RAW saving time from the Miscellaneous Stuff - Debug Parameters menu.
Since the console output may be overwritten by the Canon OSD quickly, just run a simple script (e.g. "interval.lua" from the full CHDK package)...DNG creation (byte swapping) needs some processing time; to save some 100ms, you can save CHDK RAW files and convert them later on a PC, e.g. with DNG4PS-2..."
http://chdk.setepontos.com/index.php?topic=8204.0

Is there any more detailed instructions concerning how to use "Show RAW saving time"? From the above posting seems that it is Not Permanent (...console output may be overwritten by the Canon OSD quickly)

Also I know nothing about scripts but did check "Debugging" http://chdk.wikia.com/wiki/Debugging but found nothing concerning using "Show RAW saving time".

Thanks again for your help.

Gene

Logged

waterwingz

12088

Re: Faster RAW Processing?

« Reply #3 on: 27 / July / 2012, 11:56:59 »

Quote from: genosmm on 27 / July / 2012, 11:06:35

b
Since the console output may be overwritten by the Canon OSD quickly, j...

When the Canon OSD updates the screen, it doesn't really overwrite the CHDK console - it just hides it. The text on the console window is still there - you just can't see it. The suggestion is to use a small script to periodically write to the display, which will cause it to re-appear. However, if you write too often to the display it will scroll your RAW save information off the top of the display. You can make the display window bigger with a script command should this be a problem.

Quote

Is there any more detailed instructions concerning how to use "Show RAW saving time"?

Just enable the option and the time will be printed to the system console screen - a small text area that will ar on your LCD.

Logged

Ported : A1200 SD940 G10 Powershot N G16

SkyWalker9

301
SX20 IS (fw 1.02b)

Re: Faster RAW Processing?

« Reply #4 on: 27 / July / 2012, 16:01:01 »

Quote from: genosmm on 24 / July / 2012, 12:31:42

...Has anyone made a script that measures the RAW Processing Time?...

I'm including a simple script that might give you enough insite to determine which class/type/brand of sd card gives you the best 'bang for the buck'. The script has no parameters, so you just save it with a BAS extension and add to the CHDK/Scripts folder on the sd card. For the test, you should point the camera at an object that has a relatively controlled light source. Each of the SD cards should be freshly formatted to insure the best results.

Basically the script locks the AutoFocus, sets the camera to non-RAW (JPGs), shoots five shots, then re-sets the camera to RAW and shoots another five shots. The script calculates the average milliseconds for each and displays the results on the OSD. I recommend running the test 3 times to check the results just in case something odd happens during a set.

A typical display would look something like this:
avg time in ms
JPG: 5232
RAW: 7106

Update 7/28/12:
I ran the script with different SD cards on my Powershot SX20 IS. As expected, higher classes of cards tend to out perform when it comes to saving the RAW files. What was interesting was that the much higher cost class 10 SD card was actually slower than the class 2 & 4 SD cards when saving JPGs. There was no improvement in speed for RAW using a class 10 SD card, so in this case the class 4 SD card with the SX20 IS gave the best results for the cost.

Brand Class JPG RAW
--------- ----- ------ ------
SanDisk 2 4374 9940
PNY 4 4374 6898
SanDisk 10 5042 6868
Extreme

Save the script as SdSpeedTest.BAS

Code: [Select]

rem Author SkyWalker9
@title SdSpeed Test
J=0
R=0
rem lock AutoFocus
  set_aflock(1)
rem test for JPG
  set_raw 0
  a=get_tick_count
  J=a
  for i=1 to 5
    shoot
  next i
  a=get_tick_count
  J=(a-J)/5
rem test for RAW
  set_raw 1
  a=get_tick_count
  R=a
  for i=1 to 5
    shoot
  next i
  a=get_tick_count
  R=(a-R)/5
rem Display results in milliseconds
  print "avg time in ms"
  print "JPG: "J
  print "RAW: "R
rem unlock AutoFocus
set_aflock(0)
end

« Last Edit: 28 / July / 2012, 21:24:14 by SkyWalker9 »

Logged

reyalp

13392

Re: Faster RAW Processing?

« Reply #5 on: 28 / July / 2012, 22:22:35 »

Should be noted that shoot() includes a lot more than the saving time, and some of that might vary depending on the scene. aflock should help keep it more constant, but there's still a lot going on.

1/3 of a second worse for class 10 is odd. It's highly unlikely this is directly due to the card being class 10, suggesting there's some other significant variable that isn't being accounted for.

Some ideas
are the cards all freshly formatted ?
are they fat16 or fat32, and what cluster size ?

Logged

Don't forget what the H stands for.

Pier

Re: Faster RAW Processing?

« Reply #6 on: 29 / July / 2012, 01:56:50 »

@SkyWalker9,
I've tested with your Script my SD Card on my SX130IS - hama SDHC 8 GB class 10.
I've made two tests:
1st one in Indoor dark conditions (no flash):
avg time in ms
JPG: 2488
RAW: 4102
2nd in Outdoor very good sunny conditions:
avg time in ms
JPG: 2272
RAW: 3890
hama is not super brand name, but doing fine job !!
Tnx for the Script.

« Last Edit: 29 / July / 2012, 02:11:25 by Pier »

Logged

waterwingz

12088

Re: Faster RAW Processing?

« Reply #7 on: 29 / July / 2012, 09:12:42 »

Quote from: Pier on 29 / July / 2012, 01:56:50

hama is not super brand name, but doing fine job !!

I would be curious to know what speeds you observed when you use the Benchmark option in the CHDK Debug menu ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

Pier

Re: Faster RAW Processing?

« Reply #8 on: 29 / July / 2012, 10:31:37 »

@waterwingz,
The test was made.The results:
Screen
write:8181 kb/s 48 fps
Memory
Write:61826 kb/s
Flash card
Write (RAW): 13105 kb/s
Write (Mem): 11170 kb/s
Write (64K): 4654 kb/s
Read (64K): 6913 kb/s
Was made and second test, the results are similar - very small difference exist.

« Last Edit: 29 / July / 2012, 10:36:14 by Pier »

Logged

SkyWalker9

301
SX20 IS (fw 1.02b)

Re: Faster RAW Processing?

« Reply #9 on: 29 / July / 2012, 11:01:17 »

Quote from: reyalp on 28 / July / 2012, 22:22:35

Should be noted that shoot() includes a lot more than the saving time, and some of that might vary depending on the scene. aflock should help keep it more constant, but there's still a lot going on.

1/3 of a second worse for class 10 is odd. It's highly unlikely this is directly due to the card being class 10, suggesting there's some other significant variable that isn't being accounted for.

Some ideas
are the cards all freshly formatted ?
are they fat16 or fat32, and what cluster size ?

Definitely agree there are probably multiple things affecting the times, both in the script with UBasic & with how the camera may be setup and used.

I tried to eliminate as many variables as I could think of:
- controlled & constant light source, single shot mode
- all SD cards were same size & formatted (full) just for the test; all FAT16, cluster size 64 & bootable (used sdminste.exe)
- the CHDK files & folders were all copied from the same source
- used fully charged batteries; after test, batteries still indicated above 90%
- All tests run back-to-back and within 15 minutes

edits:
- attached benchmarks from a previous test
- JPGs were shot Large (12M 4000x3000); Image quality (compression ratio) Fine
- avg JPG size 2,750 Kb and DNGs 18,255 Kb

Class benchmarks.pdf (13.62 kB - downloaded 69 times.)

« Last Edit: 29 / July / 2012, 14:01:30 by SkyWalker9 »

Logged