CHDKPTP - PC Remote Control Performance Analysis - page 25 - RAW Shooting and Processing - CHDK Forum  

CHDKPTP - PC Remote Control Performance Analysis

  • 465 Replies
  • 95710 Views
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #240 on: 19 / September / 2012, 17:54:36 »
Advertisements
Quote "I have some ideas of how to fix this, but it will require some major re-working"

No worries // it's fine the way it is now.  The kill is nothing serious and all I have to do is clean the camera // will live with.

I started the USB 2.0 long-test on S90_100C with mem sniff ON and loop breakout.  Runs fast and looks fine so far, keeping in mind that this one uses the FAST SD card too -- hence big changes in test setup = so if there are no errors this evening I will go back to the S90_101a.


*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #241 on: 19 / September / 2012, 22:31:51 »
PART 1 of 2

Ok we have something ... the count is 560 myshoots good, but, I see now it will fail at one point: memory leak // definite.

The screen shots contain all 4 sniffs in dcimdl.  Look at the "Items" at the bottom of file explorer to know the myshoot count.  myshoots = Items / 2.

The *big* question, Canon or CHDK?

dcimdl + shoot = free_size drops by 290 bytes per myshoot


*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #242 on: 19 / September / 2012, 22:41:12 »
PART 2 of 2

Here is the screen where I comment out the body of dcimdl.  Note that each sniff is *one* entry to dcimdl

Hence from this figure, shoot only = 148 bytes / shoot.

So dcimdl adds 289-148 = 141 bytes alone.

Thus dcimdl chews up 141 B and shoot chews up 148 B.  If not Canon, there might be a common leak mechanism.

I am ready to do my part in whatever possible to slay this.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #243 on: 19 / September / 2012, 23:51:21 »
My mistake in our favor I think, in the previous post .......

This two screens attached are shoot only, Fig 1 top is start, Fig 2 bottom is 200 shoots.

The general allocation patterns over 3 shoots per screen are exactly the same, hence CHDK shoot (and Canon) *do not* leak.  For example .... the bottom sniff in Fig 1 (shoot #4) and the bottom of Fig 2 (shoot #200) are exactly the same.

This to me *very* good news.  Conclusion: the only leak source is call(s) in dcimdl.

You should be able to solve this I hope.  If not, I can help.





*

Offline reyalp

  • ******
  • 13673
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #244 on: 20 / September / 2012, 00:05:03 »
This is a puzzle. On my d10, using the attached code (which I believe is still very similar to what you are using) I don't observe any decrease in free memory doing
!sb(10)

Not even the 24 bytes/file mentioned in http://chdk.wikia.com/wiki/CHDK/Camera_RAM_memory_usage (not too surprising since that was based on a much older camera)

I also tried various steps individually (shooting, listdir, downloading) with the same result.

Quote
This to me *very* good news.  Conclusion: the only leak source is call(s) in dcimdl.
The next step would be to figure out which step of dcimdl causes the leak, the listdir, the mdl or the rm. You should be able turn these manually to find out. You can use
=return get_meminfo().free_size
to check the free size.
Don't forget what the H stands for.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #245 on: 20 / September / 2012, 00:36:26 »
STEP 1 - I get the leak with sb() // attached

Next ... start with disabling CR2 and see what happens.

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #246 on: 20 / September / 2012, 00:56:47 »
Setting to JPG only (like your D10 I think) continues to leak.

*

Offline reyalp

  • ******
  • 13673
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #247 on: 20 / September / 2012, 01:28:47 »
Since you believe you have narrowed this down to dcimdl, I would suggest try thing different parts of that in isolation. For example
=shoot()
=return get_meminfo().free_size
!return con:listdir('A/DCIM')
=return get_meminfo().free_size
mdl -fmatch=%.[JDC][PNR][G2W]$ DCIM/... ...
=return get_meminfo().free_size
etc
Don't forget what the H stands for.


*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #248 on: 20 / September / 2012, 01:51:32 »
I did all that, including reducing to JPG only.  Here are the results.

The major offender is rm.  For the others I could not detect anything serious with 20 myshoots per isolation test.

With JPG only, the rm call leaves about 28 bytes per pass (I am not convinced if this is the same as your 24 bytes).  With JPG&CanonCR2, it's the first number ie 290 bytes per pass.

There's the leak, and it's pretty big.  What can I do next?

*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #249 on: 20 / September / 2012, 16:18:51 »
Note: I found that 20 shoots is not enough to get conclusive leak results // ignore the previous test results.  It seems leak conditions are fairly stable until 20 shoots is reached.

These may yield some insight // I'm sure you can deduce my rationale as you follow each test.  I standardized on 100 shoots.  All tests show a screen shot at 1 shoot and 100 shoots.   Some tests have screen shots at 22, 33 and 51 shoots added, so you can observe leak progression.

What I see here ..... from the JPG ONLY tests it is quite obvious that how the directories are deleted have a large impact on leak behavior.  Adding CR2 to JPG effectively doubles the leak in the non-transfer and transfer conditions respectively.   Worst is TEST 3, best is TEST 4.   TEST 1 somewhat agrees with your "24-byte" link. 

JPG ONLY

TEST 1 #67:  100 shoots, no dcimdl
Leak:  35 B/shoot
cold restart

TEST 2 #68:  100 shoots, full dcimdl, DCIM remains but is empty
Leak: 146 B/shoot
cold restart

TEST 3 #69:  100 shoots, full dcimdl, with DCIM delete (blank SD after each shot)
Leak: 201 B/shoot
cold restart

TEST 4 #70:  100 shoots, full dcimdl, files delete only, leaving CANONMSC and 122___09
Leak:  91 B/shoot
cold restart

TEST 5 #71:  consistency test - repeat TEST 4
Leak:  92 B/shoot
cold restart

JPG & CanonCR2

TEST 6 #72: repeat TEST 5
Leak: 201 B/shoot
cold restart

TEST 7 #73:  repeat TEST1
leak:  86 B/shoot
cold restart

Screen shots:
http://www.sendspace.com/file/6klp5y

Thus I can confirm your assessment that the latent FIALs in LONG TESTs might be leak-related on that malloc() call you demonstrated.  I am interested in your expert analysis and thoughts on these results ... Canon ?  CHDK ?

It's possible for me to do a more rigorous isolation sequence in dcimdl, but I'd like your comments and advice first.

 

Related Topics