tracking crashes / memory leaks in lua scripts

mattkime

299

« on: 14 / December / 2008, 20:26:32 »

Advertisements

i have a script which i'm using to take a pic every five minutes, rename them, and delete pics as necessary. i'm getting crashes which seem worse if there are more files on the card. any techniques for tracking memory leaks in lua scripts? i've gone through and made sure that as many variables are local as possible but i'm not sure what else to do. its run okay for 20 minutes this time around but it needs to run overnight at minimum for me to trust it

i've posted the code below. please let me know if you see any bad coding patterns

thanks

Code: [Select]

--[[
@title Interval shooting
--]]

--customDir = "000Time"
customDir = "100CANON"

-- function definitions

function getCurrentDir ()
	local dir = os.listdir("A/DCIM", false)
	if(dir) then
		local count = table.getn(dir)
		--if(dir[count] ~= customDir) then
			print("getCurrentDir: "..dir[count])
			return "A/DCIM/"..dir[count]
		--else
			--print("getCurrentDir:A")
			--return nil
		--end
	else
		print("getCurrentDir:B")
		return nil
	end
end

function getNewFile()
	local dir = os.listdir("A/DCIM/100CANON", false)
	if(dir) then
		local count = table.getn(dir)
		if(dir[count] ~= 0) then
			local itemCount = 0
			repeat
				itemCount = itemCount + 1
				if (string.sub(dir[itemCount],1,3) == "IMG") then
				io.write("getNewFile loop count: "..itemCount.."\n")
					return dir[itemCount]
					--return "A/DCIM/100CANON/"..dir[itemCount]
				end
			until itemCount == count
		end
	end
end

function deleteImgFiles()
	local dir = os.listdir("A/DCIM/100CANON", false)
	if(dir) then
		local count = table.getn(dir)
		if(dir[count] ~= 0) then
			local itemCount = 0
			repeat
				itemCount = itemCount + 1
				
				if (string.sub(dir[itemCount],1,3) == "IMG") then
					io.write("delete: "..dir[itemCount].."\n")
					os.remove("A/DCIM/100CANON/"..dir[itemCount])
				end
			until itemCount == count
		end
	end
end 

log = io.open("A/intlog.txt","a+")
io.output(log)

deleteImgFiles()

--os.remove("A/DCIM/100CANON/IMG_0960.JPG")
--os.remove("A/test.txt")

--best image compression - not working in manual mode
--set_prop(23,0)
--best image resultion - not working in manual mode
--set_prop(24,0)
--flash off
--set_prop(16,2)
--set iso to 80
--set_iso(1)
--set color - AWB
--set aperture to 5.6
--set focus
--set vivd off



repeat

	local minute = os.date("%M")
	io.write(minute.."\n")
	--minuteDigit = string.sub(minute,-1)
	--print(minuteDigit)
	if ((minute  % 5 == 0) or (minute == 0)  ) then
	--if ((minuteDigit == "5") or (minuteDigit == "0")  ) then
		if minute ~= lastMinute then
			if(get_jpg_count() < 10) then
			--delete image
				local dir = os.listdir("A/DCIM/100CANON", false)
				os.remove("A/DCIM/100CANON/"..dir[1])
				os.remove("A/DCIM/100CANON/"..dir[2])
				io.write("delete\n")
			end
			shoot()
			
			local dir = os.listdir("A/DCIM/100CANON", false)
			--perhaps this isn't updating in time
			local count = table.getn(dir)
			io.write("count: "..count.."\n")
			print("count:"..count)
			
			--currentFile = dir[count]
			local currentFile = getNewFile()
			
			--print("currentFile:"..currentFile)
			--fileCreatedEpoch = os.stat(currentFile).ctime
			--fileCreatedEpoch = os.stat(dir[count]).ctime
			--fileCreatedDate = os.date("%H-%M",fileCreatedEpoch)
			--fileCreatedDate = os.date("%Y-%m-%d_%H-%M")
			local fileCreatedDate = os.date("%d_%H-%M")
			os.rename("A/DCIM/100CANON/"..currentFile,"A/DCIM/100CANON/"..fileCreatedDate..".JPG")
			print(currentFile.."|"..fileCreatedDate..".JPG")
			io.write(currentFile.."|"..fileCreatedDate..".JPG\n")
			--count = table.getn(dir)
			
			print(minute)
			--print(dir[count])
			lastMinute = minute
		end
	end


until false

Logged

reyalp

14079

Re: tracking crashes / memory leaks in lua scripts

« Reply #1 on: 14 / December / 2008, 20:57:06 »

I suggest checking out Lua 5.1 Reference Manual - collectgarbage

You can use it to monitor how much memory lua is using or force garbage collection, as well as tuning other GC parameters. The lua garbage collector doesn't know how much memory is available, so it is possible to run out of memory despite most of it being used by collectible objects.

As a general rule, you should pass multiple values to print and write instead of using concatenation. Eg, instead of

Code: [Select]

print(currentFile.."|"..fileCreatedDate..".JPG")

use

Code: [Select]

print(currentFile,"|",fileCreatedDate,".JPG")

See this post Night-time time-lapse

As I advised you earlier in String lib! you should use sleep instead of a busy loop. It is possible (even likely) that your issue is still CPU starvation rather than memory. You can get 10ms resolution from os.time or get_tick_count, which should be plenty for your purposes! If you don't want to calculate the correct sleep (which really isn't that hard) just sleep for 100ms every iteration or something like that. Or at least make sure you sleep after every shoot, since I already verified that not doing so causes crashes. (yes, the fact that you can hit CPU starvation issues is a bug, but for the moment you'll have to work around it.)

If you do insist on using a busy loop, writing to your logfile every single iteration is probably not a great idea either.

Local vs global shouldn't be an issue here, although locals are generally more efficient, so it's still a good idea to use them.

Logged

Don't forget what the H stands for.

mattkime

299

Re: tracking crashes / memory leaks in lua scripts

« Reply #2 on: 15 / December / 2008, 02:10:08 »

i've implemented all your suggestions. thanks!

Logged

mattkime

299

Re: tracking crashes / memory leaks in lua scripts

« Reply #3 on: 18 / December / 2008, 10:04:36 »

im using collectgarbage every loop but the thing that seems to have fixed it is sleeping until its time to take the next pic.

the camera is now running indefinitely, or at least until the eye-fi card needs to be reset. (anyone friends with an eye-fi developer? i'd love to see that fixed)

Logged