memory leak

[Joe Van Dyk]

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

Thanks,
Joe

 

[Kirk Haines]

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

That's a large leak. My approach is to first step back and walk through the
code on paper, finding all of the areas where I _might_ be creating
references that don't get cleaned up. Anything that I think _might_ be a
problem I then try to isolate and test individually. I'll put diagnostic
logging in to check object counts or specific contents of hashes or LRU
caches or whatever else I might suspect to be a problem.

In bad cases, you can use ObjectSpace to dump every object known to the the
Ruby interpreter at some specific point in your code, and then skim through
that output to see if you can spot references that are accumulating which
should not be.

File.open('/tmp/objects.out','a+') do |fh|
fh.puts '________'
ObjectSpace.each_object do |obj|
fh.puts "#{obj.class.name} ::: #{obj.inspect}"
end
end

If you are dealing with a C extension, walk through the code. Look for
places where memory is allocated but never freed. When one is used to
programming in a language with garbage collection, it is very easy to
allocate memory in C without making sure that it'll be freed when no longer
needed.


Kirk Haines

 

[Tom Willis]

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

Thanks,
Joe

I'd probably start with the profiler.
http://www.rubygarden.org/ruby?Ruby_Profiler

This will at least identify the hot spots in your code. And you have
to start somewhere right?

If you suspect it's in a lib you depend on, I would write small
scripts that use the only the lib you suspect, and see if similair
behavior appears.

Basically what I try to do is eliminate the knowns, and make the
unknowns become knowns.

 

[Nikolai Weibull]

* Joe Van Dyk (Mar 23, 2005 02:20):

Are there any tools or strategies that I can use to track down the
leak?

http://valgrind.org/,
nikolai

 

[Joel VanderWerf]

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

Thanks,
Joe

It's out of date now, but I used to patch ruby to tell me why a certain
object was not collectable. It's down near the end of:

http://www.rubygarden.org/ruby?GCAndMemoryManagement

This might be worth updating if you know what objects are not being
collected, but can't tell why.

 

[Joel VanderWerf]

* Joe Van Dyk (Mar 23, 2005 02:20):



http://valgrind.org/,
nikolai

But that won't help if the objects are reachable from the root
references in the interpreter. In other words, it's not a true memory
leak, but you've forgotten to clear out a reference somewhere.

 

[Joe Van Dyk]

But that won't help if the objects are reachable from the root
references in the interpreter. In other words, it's not a true memory
leak, but you've forgotten to clear out a reference somewhere.

How can i clear a reference?

I ran into trouble before when I had a bunch of objects that I was
observing. When I overwrote those objects, the old objects stayed
around (because the other objects were still observing them). I think
a call to delete_observers fixed my problem. And I think something
like that might be the cause of my problem.

 

[Bill Kelly]

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

This might be kind of whacky, but maybe:

set_trace_func proc {|event, file, line, id, binding, classname|
printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, classname
print `ps -u #$$`
}

This (on Linux) should print a trace of every line of
your program being executed, followed by some memory usage
information for the process. ... And slow down your app by
3 or 4 orders of magnitude...

Well, just a thought... Like, you could log the output to
a text file, then go back and see where the memory was increasing,
and see if it correlates to any particular part of the code?


Regards,

Bill

 

[Kirk Haines]

This might be kind of whacky, but maybe:

set_trace_func proc {|event, file, line, id, binding, classname|
printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, classname
print `ps -u #$$`
}

For hard to find leaks, that's helpful. Also (and it'd slow the !#$%@!!!
out of your program if you do it a lot):

count = 0; ObjectSpace.each_object {count += 1}

Gives you a count of the current number of objects. If you do this in key
places (probably don't want to do it out of set_trace_func), it might help.


Kirk Haines

 

[Lothar Scholz]

Hello Nikolai,

NW> * Joe Van Dyk (Mar 23, 2005 02:20):
NW> http://valgrind.org/,
NW> nikolai

This will not help for ruby, unless he is using a C extension that
leaks memory. And even for C it's difficult with valgrind. Or do they
now keep a log where the allocating source position of each memory block is
tracked ?

 

[Csaba Henk]

For hard to find leaks, that's helpful. Also (and it'd slow the !#$%@!!!
out of your program if you do it a lot):

count = 0; ObjectSpace.each_object {count += 1}

Gives you a count of the current number of objects. If you do this in key
places (probably don't want to do it out of set_trace_func), it might help.

Maybe rather just

count = ObjectSpace.each_object {}

?

Csaba

 

[ES]

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

250KB is pretty large... are you even *creating*
that many objects per second? If yes, review the
lifetime of those objects.

If not, is this a new problem? If so, did you just
start using a new library?

Using trace is probably helpful, too.

Thanks,
Joe

E

 

[Robert Klemme]

help.

Maybe rather just

count = ObjectSpace.each_object {}

Or a bit more complex

def ostats(last_stat = nil)
stats = Hash.new(0)
ObjectSpace.each_object {|o| stats[o.class] += 1}

stats.sort {|(k1,v1),(k2,v2)| v2 <=> v1}.each do |k,v|
printf "%-30s %10d", k, v
printf " delta %10d", (v - last_stat[k]) if last_stat
puts
end

stats
end

Then you can place lines like this in your program

stats = nil
....
stats = ostats stats
....
stats = ostats stats

To look at changes between lines. Of course this method does create some
instances, too - but no instances of your application classes. You can as
well add filters (by class or by count) etc.

Kind regards

robert

 

[Mark Probert]

Hi ..

Hi,

My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.

Are there any tools or strategies that I can use to track down the leak?

Can you give a bit more of an idea of the platform and the type of
application? I have had issues in the past with threaded net::telnet based
applications if I didn't do the creation / cleanup just right.

Valgrind didn't help. The only thing that did was checking the inner classes
one at a time and rewriting the driver code a number of different ways. I am
still not sure why the leak "went away" ... :-(

Regards,