tracking crashes / memory leaks in lua scripts

mattkime

« on: 14 / December / 2008, 20:26:32 »

Advertisements

i have a script which i'm using to take a pic every five minutes, rename them, and delete pics as necessary. i'm getting crashes which seem worse if there are more files on the card. any techniques for tracking memory leaks in lua scripts? i've gone through and made sure that as many variables are local as possible but i'm not sure what else to do. its run okay for 20 minutes this time around but it needs to run overnight at minimum for me to trust it

i've posted the code below. please let me know if you see any bad coding patterns

thanks

Code: [Select]

--[[
@title Interval shooting
--]]

--customDir = "000Time"
customDir = "100CANON"

-- function definitions

function getCurrentDir ()
 local dir = os.listdir("A/DCIM", false)
 if(dir) then
  local count = table.getn(dir)
  --if(dir[count] ~= customDir) then
   print("getCurrentDir: "..dir[count])
   return "A/DCIM/"..dir[count]
  --else
   --print("getCurrentDir:A")
   --return nil
  --end
 else
  print("getCurrentDir:B")
  return nil
 end
end

function getNewFile()
 local dir = os.listdir("A/DCIM/100CANON", false)
 if(dir) then
  local count = table.getn(dir)
  if(dir[count] ~= 0) then
   local itemCount = 0
   repeat
    itemCount = itemCount + 1
    if (string.sub(dir[itemCount],1,3) == "IMG") then
    io.write("getNewFile loop count: "..itemCount.."\n")
     return dir[itemCount]
     --return "A/DCIM/100CANON/"..dir[itemCount]
    end
   until itemCount == count
  end
 end
end

function deleteImgFiles()
 local dir = os.listdir("A/DCIM/100CANON", false)
 if(dir) then
  local count = table.getn(dir)
  if(dir[count] ~= 0) then
   local itemCount = 0
   repeat
    itemCount = itemCount + 1
    
    if (string.sub(dir[itemCount],1,3) == "IMG") then
     io.write("delete: "..dir[itemCount].."\n")
     os.remove("A/DCIM/100CANON/"..dir[itemCount])
    end
   until itemCount == count
  end
 end
end 

log = io.open("A/intlog.txt","a+")
io.output(log)

deleteImgFiles()

--os.remove("A/DCIM/100CANON/IMG_0960.JPG")
--os.remove("A/test.txt")

--best image compression - not working in manual mode
--set_prop(23,0)
--best image resultion - not working in manual mode
--set_prop(24,0)
--flash off
--set_prop(16,2)
--set iso to 80
--set_iso(1)
--set color - AWB
--set aperture to 5.6
--set focus
--set vivd off



repeat

 local minute = os.date("%M")
 io.write(minute.."\n")
 --minuteDigit = string.sub(minute,-1)
 --print(minuteDigit)
 if ((minute  % 5 == 0) or (minute == 0)  ) then
 --if ((minuteDigit == "5") or (minuteDigit == "0")  ) then
  if minute ~= lastMinute then
   if(get_jpg_count() < 10) then
   --delete image
    local dir = os.listdir("A/DCIM/100CANON", false)
    os.remove("A/DCIM/100CANON/"..dir[1])
    os.remove("A/DCIM/100CANON/"..dir[2])
    io.write("delete\n")
   end
   shoot()
   
   local dir = os.listdir("A/DCIM/100CANON", false)
   --perhaps this isn't updating in time
   local count = table.getn(dir)
   io.write("count: "..count.."\n")
   print("count:"..count)
   
   --currentFile = dir[count]
   local currentFile = getNewFile()
   
   --print("currentFile:"..currentFile)
   --fileCreatedEpoch = os.stat(currentFile).ctime
   --fileCreatedEpoch = os.stat(dir[count]).ctime
   --fileCreatedDate = os.date("%H-%M",fileCreatedEpoch)
   --fileCreatedDate = os.date("%Y-%m-%d_%H-%M")
   local fileCreatedDate = os.date("%d_%H-%M")
   os.rename("A/DCIM/100CANON/"..currentFile,"A/DCIM/100CANON/"..fileCreatedDate..".JPG")
   print(currentFile.."|"..fileCreatedDate..".JPG")
   io.write(currentFile.."|"..fileCreatedDate..".JPG\n")
   --count = table.getn(dir)
   
   print(minute)
   --print(dir[count])
   lastMinute = minute
  end
 end


until false

Logged

reyalp

12067

Re: tracking crashes / memory leaks in lua scripts

« Reply #1 on: 14 / December / 2008, 20:57:06 »

I suggest checking out Lua 5.1 Reference Manual - collectgarbage

You can use it to monitor how much memory lua is using or force garbage collection, as well as tuning other GC parameters. The lua garbage collector doesn't know how much memory is available, so it is possible to run out of memory despite most of it being used by collectible objects.

As a general rule, you should pass multiple values to print and write instead of using concatenation. Eg, instead of

Code: [Select]

print(currentFile.."|"..fileCreatedDate..".JPG")

use

Code: [Select]

print(currentFile,"|",fileCreatedDate,".JPG")

See this post Night-time time-lapse

As I advised you earlier in String lib! you should use sleep instead of a busy loop. It is possible (even likely) that your issue is still CPU starvation rather than memory. You can get 10ms resolution from os.time or get_tick_count, which should be plenty for your purposes! If you don't want to calculate the correct sleep (which really isn't that hard) just sleep for 100ms every iteration or something like that. Or at least make sure you sleep after every shoot, since I already verified that not doing so causes crashes. (yes, the fact that you can hit CPU starvation issues is a bug, but for the moment you'll have to work around it.)

If you do insist on using a busy loop, writing to your logfile every single iteration is probably not a great idea either.

Local vs global shouldn't be an issue here, although locals are generally more efficient, so it's still a good idea to use them.

Logged

Don't forget what the H stands for.

mattkime

Re: tracking crashes / memory leaks in lua scripts

« Reply #2 on: 15 / December / 2008, 02:10:08 »

i've implemented all your suggestions. thanks!

Logged

mattkime

Re: tracking crashes / memory leaks in lua scripts

« Reply #3 on: 18 / December / 2008, 10:04:36 »

im using collectgarbage every loop but the thing that seems to have fixed it is sleeping until its time to take the next pic.

the camera is now running indefinitely, or at least until the eye-fi card needs to be reset. (anyone friends with an eye-fi developer? i'd love to see that fixed)

Logged