CHDKPTP - PC Remote Control Performance Analysis - page 28 - RAW Shooting and Processing - CHDK Forum
« previous next »
Pages: 1 ... 23 24 25 26 27 [28] 29 30 31 32 33 ... 47   Go Down

CHDKPTP - PC Remote Control Performance Analysis

  • 465 Replies
  • 101817 Views
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #270 on: 23 / September / 2012, 16:18:06 »
Advertisements
Quote "You should not assume those are the only two possibilities."

Of course ... I know. That's why I am trying to eliminate as many controllable variables as possible.  For example, on my side: I must not play around with the test computer over the network because that seems to interfere with a loop test (happened once so I don't touch it anymore).  Introducing write holdoff into shoot would eliminate another controllable variable.

Refining my side, there's a bit of good news ... since this morning the last test is still running Ok.   It started off at 880K and is at 685K now (500+200+200+400+100 ~= 1400 shoots+dcimdl JPG+CR2, 2800 images so far).  In the middle of this test I had PC disk full and CHDKPTP responded correctly // I am very happy about that.

Quote "300k should be plenty, down to 150 is probably ok unless you are using something really memory heavy."

Considering today's good results, this memory bodes very well if we can handle the controllable variables.  Because RC transfer begs for write holdoff, I am just wondering ... since srsa is working on RC, could he possibly craft out the write holdoff solution and give it to you so I could try it out?  Please don't be angry with me :)

edit: not only did it respond correctly but I was able to recover w/o reboot and continue the test // very nice.

edit: after pruning the code I noticed I have 3s delay (instead of 1.6) on a 3/4s SD card.   That could be a reason why it's running well, but 3s is a huge overkill.

Code: [Select]
function dcimdl(destdir,del,pretend)
    if pretend then
        pretend = '-pretend '
    else
        pretend = ''
    end
    local dcim,err=con:listdir('A/DCIM')
    if not dcim then
        printf('error listing directory %s\n',tostring(err))
    return
    end
    for i,dname in ipairs(dcim) do
        if dname ~= 'CANONMSC' then
            cli:print_status(cli:execute('mdl '..pretend..' -fmatch=%.[JDC][PNR][G2W]$ DCIM/'..dname..' '..destdir))
        end
    end
if del then
cli:print_status(cli:execute('rm '..pretend..' -skip_topdirs DCIM'))
end
end

function shoot_dcimdl(tv, av, sv, destdir)
cli:execute('shoot -tv='..tv..' -av='..av..' -sv='..sv..'')
--con:execwait('shoot -tv='..tv..' -av='..av..' -sv='..sv..'')
sys.sleep(1600)
local l,r=con:execwait('return get_meminfo().free_size')
print(" free", r)   
printf(" free: %s\n",r)   
dcimdl(destdir, true, false)
end

function loop(n)
for i=1,n do
print('myshoot', i)
printf('myshoot %d ', i)
shoot_dcimdl(1/8,2,200,"C:\\CANON_S90\\")
sys.sleep(1400)
if not con:is_connected() then
return;
end
end
end
« Last Edit: 23 / September / 2012, 16:32:42 by SticK »
Logged
*

Offline reyalp

  • ******
  • 13878
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #271 on: 23 / September / 2012, 18:39:16 »
Quote from: SticK on 23 / September / 2012, 16:18:06
Because RC transfer begs for write holdoff, I am just wondering ... since srsa is working on RC, could he possibly craft out the write holdoff solution and give it to you so I could try it out?  Please don't be angry with me :)
I don't tell anyone what to work on, if you have a request for srsa, ask him. You can use the private message function on the forum.

Looking at your code, I don't see anything that would obviously explain the previous error. Very strange.
Logged
Don't forget what the H stands for.
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #272 on: 23 / September / 2012, 19:41:03 »
Quote "I don't tell anyone what to work on, if you have a request for srsa, ask him. You can use the private message function on the forum."

I did not ask you to *tell* srsa what to do.  Since you're talking with him on the RC thread, I felt you could tacitly agree to present this suggestion on my behalf, that is to perhaps advance it for upfront testing since you did tell me he will have it in anyway, but now that I see you're not interested, I will do so myself.  Thanks for the tip.

Quote "Looking at your code, I don't see anything that would obviously explain the previous error. Very strange."

Because an SD card can occasionally stall a write way beyond the delay setting, the only way to rule out camera hardware contention as a potential source of this error at this point, would be .... (my jaw is tired from yapping about it so much and I'm sure your ears can't take it anymore too -- but please understand it's hard for me not to make a case for it) // I will chat with srsa.
Logged
*

Offline srsa_4c

  • ******
  • 4451
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #273 on: 23 / September / 2012, 21:06:24 »
Don't worry, I do read this thread. I'll try to port the FileWriteTask stuff to the S90 in a few days.

@reyalp
If you have a suggestion how this signal could be implemented, I'm all ears :)
Instead of issuing
Code: [Select]
action_push_delay(conf.script_shoot_delay*100); in AS_SHOOT, what should be done? Is this related to state_shooting_progress? If so, that variable is only set to SHOOTING_PROGRESS_DONE in core/main.c, depending on a propcase which is evaluated by shooting_in_progress() in core/shooting.c ...

The other issue is that on cameras with native RAW and RAW + JPEG setting, FileWriteTask will be launched twice. This condition will need to be detected somehow.

Since I don't want to cause chaos in the test branch, could you merge(?) the necessary changesets there? Should this be done by simply merging all trunk changesets since the creation of ptp-remote-capture-test?
Logged
*

Offline reyalp

  • ******
  • 13878
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #274 on: 23 / September / 2012, 21:54:13 »
Quote from: SticK on 23 / September / 2012, 19:41:03
Because an SD card can occasionally stall a write way beyond the delay setting, the only way to rule out camera hardware contention as a potential source of this error at this point, would be ....
Although I don't have a good explanation for that log, something related to file write timing would be very far down the list of possibilities.  Somehow Lua on the camera returned something very strange, or some bug in the chdkptp code caused it to see the wrong thing as a return value. The file write timing thing will basically end up with the FsIoNotify assert, or maybe something with incomplete files.

Quote from: srsa_4c on 23 / September / 2012, 21:06:24
If you have a suggestion how this signal could be implemented, I'm all ears :)
Instead of issuing
Code: [Select]
action_push_delay(conf.script_shoot_delay*100); in AS_SHOOT, what should be done? Is this related to state_shooting_progress? If so, that variable is only set to SHOOTING_PROGRESS_DONE in core/main.c, depending on a propcase which is evaluated by shooting_in_progress() in core/shooting.c ...
That's part of the reason I haven't been eager to dive into it, just detecting when the file is done is only part of the problem.

A simple-ish way might be to make a function like shooting_is_save_complete() function and changing main.c to do something like
Code: [Select]
if ((state_shooting_progress == SHOOTING_PROGRESS_PROCESSING) && (!shooting_in_progress()) && shooting_is_save_complete())
shooting_is_save_complete() could just return true on cameras that don't have filewritehook, or use a flag set by PT_CompleteFileWrite if that turns out to work.

I'm not sure what potential side effects of doing this would be. You could also make AS_WAIT_SAVE wait for both SHOOTING_PROGRESS_ and shooting_is_save_complete(), or add a new stack item AS_WAIT_SAVE2...

It would be good to have the is_save_complete available from script in some way too, for people who use button presses instead of shoot(). We could put it in shooting_in_progress() but I'd be hesitant since that has always just reflected the propcase value before.
Quote
The other issue is that on cameras with native RAW and RAW + JPEG setting, FileWriteTask will be launched twice. This condition will need to be detected somehow.
File name ? ;)
Quote
Since I don't want to cause chaos in the test branch, could you merge(?) the necessary changesets there?
There should only be a few s90 ones, they should be easy to move over.
Quote
Should this be done by simply merging all trunk changesets since the creation of ptp-remote-capture-test?
That's also OK, and would make could make the final remotecapture merge easier down the road, I'll try it and see what happens.
Logged
Don't forget what the H stands for.
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #275 on: 23 / September / 2012, 22:09:39 »
@srsa_4c
That's great to hear // I will gladly give it a thorough workout (with native RAWs too) using the current incarnation of shoot_dcimdl.  Most of all to me, what you're implying that there is a possibility of a doable solution for write holdoff, light at end of tunnel, and when the only outstanding major caveat (memory) now appears predictable where out-of-memory crashes can be controlled, then that gives me confidence to begin planning Phase 2 of testing.  If what you're doing pans out Ok, CHDK+CHDKPTP just might outperform Canon RemoteCapture in many respects.  I will demonstrate how once the your solution is up and running, but, I am already inclined to jump the gun and say this is an amazing tool.  FYI to complement your efforts, with the long delay on the fast SD card I have had no failures all day today and the loop test continues to run.  Right now the S90 is down to 370k (from 888k) and is still fine.  Best of luck, and, thanks. 

Quote "The other issue is that on cameras with native RAW and RAW + JPEG setting,"

The S90 has this option and I intend to use both flavors in the instrumentation -- one disadvantage of the S50 that it can only shoot one or the other.

@reyalp
I am ready to test anything you need.  The USB 1.0 bench is freed up so if you'd like to give the SX110 a work-over too in parallel at one point, I can help.
Logged
*

Offline reyalp

  • ******
  • 13878
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #276 on: 23 / September / 2012, 22:28:42 »
Quote from: SticK on 23 / September / 2012, 22:09:39
@reyalp
I am ready to test anything you need.  The USB 1.0 bench is freed up so if you'd like to give the SX110 a work-over too in parallel at one point, I can help.
At this point, I don't have any hypothesis or ideas for debugging your last script log.
Logged
Don't forget what the H stands for.
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #277 on: 24 / September / 2012, 00:44:07 »
Here's a screen shot instead of text because the DOS and console outputs are different.  Note that in this case the two files were saved on SD but not transferred to PC, and the test continued on till the end (400).  Could this be related to the last mysterious error?  Any ideas what could be the source?
Logged
*

Offline reyalp

  • ******
  • 13878
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #278 on: 24 / September / 2012, 02:00:22 »
Quote from: SticK on 24 / September / 2012, 00:44:07
Here's a screen shot instead of text because the DOS and console outputs are different.  Note that in this case the two files were saved on SD but not transferred to PC, and the test continued on till the end (400).  Could this be related to the last mysterious error?  Any ideas what could be the source?
That's the same "no match" error discussed here:
http://chdk.setepontos.com/index.php?topic=8613.msg90775#msg90775

Could it be related to the strange return here: http://chdk.setepontos.com/index.php?topic=8613.msg91190#msg91190 ? Could be, both seem to involve camera Lua returning "impossible" values.

My thought at this point is to make a stress test that just runs lots of lua repeatedly. If I can reproduce the issue on one of my cameras, I'll have a much better chance of being able to debug it.

My gut feeling is the "no match" error is related to the mysterious script death error linked earlier. I may be able to adjust the lua / ptp error return code to confirm or disprove this, I'll look into it as soon as I get the chance.
Logged
Don't forget what the H stands for.
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #279 on: 24 / September / 2012, 08:26:26 »
S90_100c first potentially successful memory leak operational cutoff test

History: the test began with 888800 free.  At 747032 there was on PC disk full where the system reacted well and the test recovered without reboot.  At 258480 we got the "no match" error I reported yesterday, also recovered with no reboot.  Camera crashed early this morning at 193136, with (2911 - 1) successful shoots.  See attached ROMLOG, ReadFDir error.

Please refer to my comments in the txt, search for "<<==".

Can you please offer your comments?  You mentioned CHDK should work down to 150K.  Is 193K free within range of crash failure considering that native RAWs are being processed with JPGs?

Next tests will be with JPG only, followed by native CR2 only.  If we have no serious errors that cause a camera crash, each should take a day.  The hope is that then the set of three (JPG+CR2, JPG only, and CR2 only) will be repeated for a consistency check.

Logged
Pages: 1 ... 23 24 25 26 27 [28] 29 30 31 32 33 ... 47   Go Up
« previous next »
 

Related Topics