CHDKPTP - PC Remote Control Performance Analysis - page 20 - RAW Shooting and Processing - CHDK Forum

CHDKPTP - PC Remote Control Performance Analysis

  • 465 Replies
  • 139804 Views
*

Offline reyalp

  • ******
  • 14128
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #190 on: 14 / September / 2012, 01:38:57 »
Advertisements
Quote "What would "make a mess of" the downloads ? What "function return time description""

Here is your answer and what I was referring to >>> Quote "Shoot is a special case, because that's the camera firmware opening and writing files in ways that we aren't in direct control of."

Hence hypothesis .... I am myshooting simultaneous 2.5M JPGs and 10M CanonCR2s, which I measured to take ~4 seconds to write after image acquisition, much *less* than the 2s dcimdl and now the 1s dcimdl delays.  If shoot did not wait for Canon write complete, the result would be a disastrous mess.  That's my drift ..... because 2s < 4s and 1s << 4s (no FAIL under 100 myshoots), shoot is in fact waiting for completion to a large extent (welcome news).
Oh ffs. I never said that shoot doesn't wait at all. It waits for some portion of the shooting process but, as proven by the fsionotify error not all of it. Exactly what it waits for is ill defined. The final wait in the shoot() code is this:
Code: [Select]
        // XXX FIXME find out how to wait to jpeg save finished
        action_push_delay(conf.script_shoot_delay*100);
FWIW, this http://chdk.setepontos.com/index.php?topic=5690.0 might provide a way of actually detecting when the file was finished, via PT_CompleteFileWrite
Quote
Therefore the delay can be determined and for the S90 might be small (likely less than 1 second), and, I speculate it can be fixed as one value for all of the latest (Digic 3+) cameras.
Given the fact that my D10 (digic IV) hit the FsIoNotify occasionally at 1 sec, I'm gonna go with no.
Don't forget what the H stands for.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #191 on: 14 / September / 2012, 09:52:33 »
S90 JPG+CanonCR2 OVERNIGHT TEST RESULTS

Quote "Given the fact that my D10 (digic IV) hit the FsIoNotify occasionally at 1 sec, I'm gonna go with no."

The S90 is Digic IV and SX110 is Digic III.  I am not sure if you understood what I am attempting.  Based on the hypothesis that there exists a PASS/FAIL dcimdl delay point, the idea is to determine if a *fixed width narrow* time dcimdl marker (delay width window) exists on the just-left-side of which you get a FAST FAIL, and on the just-right-side of which you get LONG PASS (800+ myshoots). 

For example, last night I set delay=700 ms with the more rigorous loop code below I got a FAST FAIL very quickly (7th myshoot I think).  So for the overnight test (again with code below), I iterated setting the delay back to 1s (known to have already PASSED 100 myshoots at 1/8s).  In the morning I can say I have the LONG PASS (it is at present 810+ myshoots, about 7GB data and still running).  Thus the conclusion is the S90 has *at most* a 300 ms window whose PASS/FAIL point is somewhere between 700 ms and 1000ms.   So 300 ms window at this dcimdl scale is not unreasonable and that news to me is *very* encouraging.   In other words, if I continue iterating (could take a couple days) I am certain I can narrow down the 300 ms width even further.  So it does seem in fact there is indeed a FAIL/PASS delay time trigger point where you get "useless" operation, and "perfect" operation.  That is the idea I was after and I am clearly happy with the results so far.  A speculative example of this method for your D10 might turn out to be 1.1s, instead of the overkill 2s you arbitrarily set it to.

This more rigorous myshoot Loop code increases the probability of FAIL.

Code: [Select]
function loop_myshoots(n)
for i=1,n do
myshoot(1/8,8,200,"C:\\CANON_S90\\")
sys.sleep(1000)
myshoot(8,8,200,"C:\\CANON_S90\\")
sys.sleep(1400)
myshoot(1/4,8,200,"C:\\CANON_S90\\")
sys.sleep(1600)
myshoot(4,8,200,"C:\\CANON_S90\\")
sys.sleep(1200)
end
end

Discussion

Quote " action_push_delay(conf.script_shoot_delay*100);"

Hmmm ... that's what I was gonna suggest but you already did it.  Where is the script_shoot_delay set?  Do I have access to it?

Quote "Exactly what it waits for is ill defined."

By doing the iteration, I am in a way attempting to reduce the "ill defined" nature of the shoot call w/o internal knowledge. 

Quote "might provide a way of actually detecting when the file was finished, via PT_CompleteFileWrite"

Wow! -- so there might be a way??  Having read your link I concluded it is not something I can readily do.  Can you please type it in and send me the test code to try?  I am *very* willing to put it through my wringer.  If successful, I will move the S90 from the test bench to the instrumentation bench // a major milestone.  The nice thing is that I can test Canon RAW on the S90 which one could consider worst-case time for shoot ... would help many more cameras and CHDK users, not just S90. 

What do you say?

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #192 on: 14 / September / 2012, 23:52:32 »
UPDATE -- For the S90 I determined the dcimdl delay window down to a range of 850msFAIL and 925msPASS, ie 75 ms wide.  That is definitely good enough to conclude a small fixed window is possible to achieve and it seems to be fairly rigid, and can be optimized to minimize shoot response delay for any camera. 

Of course, because this delay is still camera (and probably image file size) dependent, a generic solution would the the most data transfer-efficient overall and would remove the stigma from the rather hacky fixed dcimdl delay solution.

Therefore I am still very interested in testing out the possibility you offered with PT_CompleteFileWrite.  I would feel much more comfortable with the generic solution.

Could you please review my previous post?  Could we try this?  Please let me know if it is doable to decide if I should wait (my preference) for the next S90 testing phase: instrumentation compatibility.
« Last Edit: 14 / September / 2012, 23:56:41 by SticK »

*

Offline reyalp

  • ******
  • 14128
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #193 on: 15 / September / 2012, 00:01:03 »
That is the idea I was after and I am clearly happy with the results so far.  A speculative example of this method for your D10 might turn out to be 1.1s, instead of the overkill 2s you arbitrarily set it to.
Or far more likely, it's not some exactly fixed time period, and all your careful tuning will become meaningless when some external factor changes.

As I mentioned, I usually test with an ancient slow 1g SD card (no reason to put cycles on a good card). I doubt the  D10 hardware is much slower than the S90, so my best guess is that card speed is a factor. "Card speed" doesn't just mean the the class of the card, it can vary depending on fragmentation and the cards lower level block management.
Quote
By doing the iteration, I am in a way attempting to reduce the "ill defined" nature of the shoot call w/o internal knowledge. 
I'm sure you can narrow it down some for your particular current combination of camera, card etc but this strikes me as a pointless exercise. Does shaving a few hundred ms off the cycle time actually make any real difference ? (while running on USB 1.1, no less!?)

Quote
Hmmm ... that's what I was gonna suggest but you already did it.  Where is the script_shoot_delay set?  Do I have access to it?
Yes, it's in the second item CHDK script menu. Although this menu is for loading scripts, it should affect scrpts started by PTP as well. You could also set it with http://chdk.wikia.com/wiki/Script_commands#set_config_value, the ConfigID is 3. The value is in 1/100ths of a second, although the achieved precisions is likely to be lower.

Quote
Quote "might provide a way of actually detecting when the file was finished, via PT_CompleteFileWrite"

Wow! -- so there might be a way??  Having read your link I concluded it is not something I can readily do.  Can you please type it in and send me the test code to try?  I am *very* willing to put it through my wringer.  If successful, I will move the S90 from the test bench to the instrumentation bench // a major milestone.  The nice thing is that I can test Canon RAW on the S90 which one could consider worst-case time for shoot ... would help many more cameras and CHDK users, not just S90. 

What do you say?
1) This is just an idea based on the function name and some very basic testing I did a long time ago, it might be completely wrong.
2) It needs more than just typing something in, some code has to be written and the concept verified.
3) For most users, the exact time the jpeg finishes writing does not seem very important. That TODO has been in the code for at least 4 years and you're the first I've heard who cared about it.
4) Even if it's simple to implement and works, doing it across the ~200 camera/firmware variants will not be trivial.
So I'll look into it if I get a chance, but I wouldn't suggest holding your breath ;)
Don't forget what the H stands for.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #194 on: 15 / September / 2012, 01:16:15 »
Quote "Or far more likely, it's not some exactly fixed time period, and all your careful tuning will become meaningless when some external factor changes."

The hope is ... once the solution is established for the instrumentation, all external factors will not be (intentionally) changed (if it ain't broke don't fix it or change it -- why I drive a 25-yo antique car and my phone is 8 yo).  I would say the crux of my tuning attempt was to determine the rigidity and width of the delay value under the various files sizes my camera generates, and if the elegant generic solution never comes to pass, this solution could be a reasonable runner-up.  It's not too bad I think.  I'd say the delay is strongly fixed for a specific camera, and as you point out, SD card write time.  In other words, finding that value (eg 925 ms S90 w Duarcell SD 4GB C4) is very reliable, so long as one don't touch the camera except for the shooting parameters.

Quote "I usually test with an ancient slow 1g SD card"

I understand you ... I test with a slow 10 yo USB 1.0 bench computer for like reasons.

Quote "no reason to put cycles on a good card"

While I did whine about SD wear somewhere near the beginning of my thread, modern SD cards use 32-bit ARMs and have very effective error correction & wear-leveling code.  Get yourself a SanDisk, Kingston, or a 4GB Duracell (mine are made in Japan) -- about 10-15 bucks a shot.  Some of these come with wear equations where you can predict the life of a card (in years) with a typical usage ... repetitive writes per day of a file size and number/day etc.  For this kind of thing you're talking dozens of years easy, at one shot per minute all day, 10 MB, for a good quality high-capacity card.

Quote "this strikes me as a pointless exercise."

I strongly disagree.  I think you still haven't quite understood what this was.  I determined the window to be narrow and more, very stable, and that suggests to me that finalizing on my 925ms + 20% headroom (1.1s say) will be *extremely* reliable.  It's really the determination a high confidence level at fastest transfer performance for lack of a generic rigorous solution // that's the gist.

Quote "Does shaving a few hundred ms off the cycle time actually make any real difference ? (while running on USB 1.1, no less!?)"

Absolutely yes.  Apart for the sensitive electrooptics, one of the major reasons for choosing the USB 2.0 SX110 or S90 is for the transfer rate.  The instrumentation existing S50 is USB 1.0.  It prevents me from performing certain types of timed experiments.  Furthermore, my bench is USB 1.0, but the instrumentation computer where this is all headed is USB 2.0.  In fact, I went out today for a C10 SD and will be comparing its performance with the C4s I have now, just in the camera alone -- if I can get an extra few hundred ms in the camera on the 10MB file that would be very useful.

Quote "2) It needs more than just typing something in, some code has to be written and the concept verified."

I am indeed very mindful of that even when I come across as not.

Quote "That TODO has been in the code for at least 4 years and you're the first I've heard who cared about it."

Ahhh .. that's-so-sweet.  But if you think about it ... it's your spectacular remote control performance that led me to care about it in the first place and encourage you for a complete clean solution.

4) Even if it's simple to implement and works, doing it across the ~200 camera/firmware variants will not be trivial.

I don't doubt it ... may I propose we start with D10, S90 & SX110 for the time being and see how it works?  ... those generally cover most of the recent PowerShots that are out there I expect ... not to lose sight of course of this fantastic new performance level just achieved with CHDKPTP! 

This is the loop I set to run overnight ...

Code: [Select]
function loop_myshoots(n)
for i=1,n do
myshoot(1/8,8,200,"C:\\CANON_S90\\")
sys.sleep(1000)
myshoot(8,8,200,"C:\\CANON_S90\\")
sys.sleep(1400)
myshoot(1/4,8,200,"C:\\CANON_S90\\")
sys.sleep(1600)
myshoot(1/4,8,200,"C:\\CANON_S90\\")
sys.sleep(1600)
myshoot(4,8,200,"C:\\CANON_S90\\")
sys.sleep(1200)
myshoot(1/2,3.5,1500,"C:\\CANON_S90\\")
sys.sleep(1100)
myshoot(2,5.6,200,"C:\\CANON_S90\\")
sys.sleep(1300)
myshoot(15,8,200,"C:\\CANON_S90\\")
sys.sleep(1300)
myshoot(1/4,3.5,200,"C:\\CANON_S90\\")
sys.sleep(1600)
myshoot(8,2.5,200,"C:\\CANON_S90\\")
sys.sleep(1400)
end
end

... it's been running for an hour at 925 ms delay .. let you know tomorrow how it pans out.




*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #195 on: 15 / September / 2012, 09:50:09 »
UPDATE -- FAIL at 422 myshoots.  Refined conclusion: 75 ms window (from 850 ms) is too narrow.  Will redo 1000 myshoots with 1s delay (150 ms window) and if successful raise to delay 1.2s as final value.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #196 on: 15 / September / 2012, 13:48:31 »
Improvement note:  Script shoot delay (.1s) would have a more accurate description if you were to rename it as "Script delay after shoot" -- if it fits.

TEST UPDATE -- After going back to 1s to test the 100 x 10-random most recent myshoot block this morning (1000 myshoots JPG+CanonCR2), I got a FAIL on the first call.  I attributed this particular failure (hopefully) to not having hand-cleaned the camera after the last night's FAIL ... ie ... delete the hanging directories and PDN/PUP.  The test has been running for about 3 hours so far.   If it succeeds I will explore Script shoot delay and its relationship to sys.sleep in myshoot.


*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #197 on: 15 / September / 2012, 21:47:32 »
1.0s dcimdl delay TEST RESULT:  FAIL at 952 myshoots,  after approx 11 hrs with the latest loop code.

This was what I was trying to determine with my "pointless test."  Even if my test finished at 1000 myshoots, this glaring result says it would still be inconclusive.  Thus in a sense we're lucky it failed here to get a deeper understanding of the problem.

This data is saying that perhaps a real "sharp cut-off," a delay point where the failure rate is 0, does not exist (within some reasonable delay that is under several seconds since we can't wait "forever").

Does this mean something like ... 1.2s will have a FAIL probability of 1:2000 myshoots? .. 1.4s a probability of 1:4000, etc??  What would 2s have?  If my test runs for 3 days w/o fail at 2s, does it mean it is ZERO FAILURE RATE.  Answer: no, according to this trend // the nature of probabilities.

The result is somewhat discouraging.   I'd vote for PT_CompleteFileWrite to be given a go.  Have you thought about it?  Does it look possible?
« Last Edit: 15 / September / 2012, 21:58:11 by SticK »

*

Offline reyalp

  • ******
  • 14128
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #198 on: 15 / September / 2012, 22:10:38 »
1.0s dcimdl delay TEST RESULT:  FAIL at 952 myshoots,  after approx 11 hrs with the latest loop code.
What failure was it ?

As you get up to high numbers of shots, you may encounter other problems.
Quote
This was what I was trying to determine with my "pointless test."  Even if my test finished at 1000 myshoots, this glaring result says it would still be inconclusive.  Thus in a sense we're lucky it failed here to get a deeper understanding of the problem.

Does this mean something like ... 1.2s will have a FAIL probability of 1:2000 myshoots? .. 1.4s a probability of 1:4000, etc??  What would 2s have?  If my test runs for 3 days w/o fail at 2s, does it mean it is ZERO FAILURE RATE.  Answer: no // the nature of probabilities.
CHDK is a hack. It operates by poking at the guts of a completely undocumented system, using methods that have been found to "work" under some ill-defined conditions on some small subset of the supported cameras. Regardless of the present issue, it would be quite unreasonable to expect industrial reliability.

We would like to make CHDK better of course, but you should have realistic expectations. If ZERO FAILURE RATE is a requirement for your project, then it's unlikely CHDK is an appropriate solution.
Quote
The result is somewhat discouraging.   I'd vote for PT_CompleteFileWrite to be given a go.  Have you thought about it?  Does it look possible?
I spent my CHDK time so far today catching up on maintainer stuff.
Don't forget what the H stands for.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #199 on: 15 / September / 2012, 22:30:46 »
Quote "What failure was it ?"

In listdir, the usual place where it fails now when it does.  I've set it to run again ... do you want a log anyway?

Quote "As you get up to high numbers of shots, you may encounter other problems."

Like what?  Please give examples.

Quote "CHDK is a hack."

I'm not so sure // but putting in this "delay" *is*a *terrible* hack, and setting to some ridiculously high value is hack+slop  // I think it would really be worth a try at PT_CompleteFileWrite ... it might solve perhaps not all but some related problems you may have encountered with other users .. salesman hat :xmas

Quote " If ZERO FAILURE RATE is a requirement for your project, then it's unlikely CHDK is an appropriate solution."

Is there any other solution?  I don't think so.

Quote "I spent my CHDK time so far today catching up on maintainer stuff."

Can you please investigate it?  I am counting on you // if it has the functionality we expect, it would be a solid solution not just for me and not just for CHDKPTP shoot, but, I expect for shoot-write overall  :xmas

 

Related Topics


SimplePortal © 2008-2014, SimplePortal