Quote "When you say X bytes/shot what are you taking as your initial value ? I would suggest taking at least one shot first since it would be unsurprising if some resources don't get allocated until the first shot."
Yes I checked this carefully. I have 4 sniffs per shoot, at the entry to dcimdl and one after each internal call. The initial variations I see with first few shoots are within a couple hundred bytes of the very first value, more or less as you demonstrate. Even after several shoots the free mem can return to its original value. For example, look at the difference between 70a (initial conditions) and 70b in the ZIP, which is 22 shots later: the free mem pattern, +/- a couple hundred bytes, is essentially the same. So you really need to acquire many more than 20 shots before you can start measuring the drop. After that, free mem seems to decline more smoothly. Hence I can take any free mem value up to 20 shoots as the initial value. Overall, the value at 100 shoots -vs- any value up to 20 shoots will give the same result +/- a few percent.
Quote "it uses 24 bytes per file, but allocates enough for 10 files, uses that, repeat."
Sensible and I agree.
Quote "Losing more when files / directories are deleted is odd. OTOH, it would be unexpected by the Canon firmware"
Yes that is what I wanted show you here. Did you mean "expected?" I did *attempt* a convoluted test to determine that but it was unsuccessful ... in essence, while connected to CHDKPTP, sniff, use camera shoot button, sniff ... but doing it ran a script instead (message flash in LL corner) and did not shoot.
Quote "If it's a CHDK leak, then it's hard to see how it would happen without each time an instance of a the leaking command is called. E.g. if there is a memory leak in something called by 'rm' then you should lose some memory each time you call it."
I've tried to give some thought about these scenarios too, but, I don't know the mechanisms so my contribution is very limited.
Quote "If you use printf instead of print"
I see, I will modify my code so it does both and send you a text file of the console.
Quote "The results are interesting: The first 39 shots all return exactly the same value. The subsequent shots seem to consume between 24 and 72 bytes/shot, without any obvious pattern (?!). I had taken some shots (about 10 ?) manually before starting this sequence."
Yes, that's "good," what I reported and what you see are consistent. The equation I used to obtain the previous results is:
X bytes/shoot = ( freemem{shoot 1} - freemem{shoot 100} ) / 100
I think my results are reliable, standardized on 100 shoots.
It is worthwhile to note that going long, say to 500 shoots, the X bytes/shoot is slightly nonlinear. For example, running TEST 6 JPG&CR2 at 100 starts out at 200 bytes, then goes to 219, 223, and 226, part way through settling on 226 after 500 shoots. One obvious explanation is fragmentation.
Quote "I took another 50 shot series and got the same decline immediately."
Those are the kids of things I see too ... if you don't reboot, then you can preserve the cumulative degradation, vary the test sequence, etc. I've done this several times. That's why in my test sequence the reboots are important for consistent results.
Quote "It's very hard for me to think of a way this pattern would be caused by CHDK."
What about this? ... it's is a guess based on pure imagination ... when Canon writes out the file it allocates memory resources and then when a dcimdl command (which may not necessarily malloc() anything) does its thing, those resources aren't deallocated. In a sense CHDK does not know what Canon allocated hence it does not know what to free ?? I wanted to test|determine if Canon frees resources with with its *own* write/delete file, but it seems I can't get Canon to work while the camera is connected to CHDKPTP so I can't operate the camera and sniff memory "manually." If you know of a way to do it, I'll give it a shot.
==>> NEW. Here's something I've seen happen twice on the USB 2.0 computer. In all my tests I have liveview enabled. The event appears random and I am sure this one is not memory related, because this last one happened at ~18 shoots. The link becomes disconnected with "ERROR download failed." The CR2 file is partially transferred, but camera does not crash, Fig 1. That is, without rebooting I can reconnect CHDKPTP, restart the shoot loop, and the old CR2 will be transferred correctly, Fig 2. What could this be?
