Memory Usage

[Horacio Sanson]

I have a ruby script that reads data from a database, processes the data (long
texts) and saves the results back to the database. I works great and at good
speed.

The problem is that I ran the script on friday and left it until monday when I
saw some memory related errors in the console (Insufficient Memory
something...). I ran the script again and fired up the windows task manager
and then I saw that for every 10~15 records in the DB the script processes
the memory goes up 1MB and there are no signs of it going down.

I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

regards,
Horacio

 

[Chad Perrin]

I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

Are you making heavy use of symbols? They don't relinquish RAM
unless/until the script exits, as far as I'm aware.

 

[Peter Hickman]

I've had a problem like this using the flickr api and net/http. All it
did was download images from Flickr, it ran pretty much 24/7 but some
days it would just eat massive amounts of virtual ram, 900Mb plus.

I've wrapped the whole thing in a shell script now to get around the
problem. Any idea how to track where the memory is going on a live system?

 

[Gene Tani]

I have a ruby script that reads data from a database, processes the data (long
texts) and saves the results back to the database. I works great and at good
speed.

The problem is that I ran the script on friday and left it until monday when I
saw some memory related errors in the console (Insufficient Memory
something...). I ran the script again and fired up the windows task manager
and then I saw that for every 10~15 records in the DB the script processes
the memory goes up 1MB and there are no signs of it going down.

I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

regards,
Horacio

a couple recent threads on digging into ruby'c GC:
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/4e77925c75949460

and John Carter's thing to dump out your objects and (linux) processes
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/139987

 

[Horacio Sanson]

Thanks,

I already ran the Robert Klemme's code to see the object count of each clas=
s.=20
I ran the code every time my script finished processing each record of the=
=20
database and the numbers are like this:

String: 42812
Array: 16651
WIN32OLE: 3184
Hash: 2065


These are the biggest entries but they do not increase all the time, they=20
fluctuate up and down. Strings goes up to 60,000+ objects and then drops=20
down back to 20,000+ objects. Then starts growing again. This happens with=
=20
all objects. Other classes are below 100 or are even 0.

I do not use symbols apart from those needed by ActiveRecord.

going to test now the MemoryProfiler of John Carter.

BTW, are these numbers normal??


thanks for the tips.
Horacio

Tuesday 13 December 2005 01:17=E3=80=81Gene Tani =E3=81=95=E3=82=93=E3=81=
=AF=E6=9B=B8=E3=81=8D=E3=81=BE=E3=81=97=E3=81=9F:

 

[Gene Tani]

Thanks,

I already ran the Robert Klemme's code to see the object count of each class.
I ran the code every time my script finished processing each record of the
database and the numbers are like this:

String: 42812
Array: 16651
WIN32OLE: 3184
Hash: 2065


These are the biggest entries but they do not increase all the time, they
fluctuate up and down. Strings goes up to 60,000+ objects and then drops
down back to 20,000+ objects. Then starts growing again. This happens with
all objects. Other classes are below 100 or are even 0.

I do not use symbols apart from those needed by ActiveRecord.

going to test now the MemoryProfiler of John Carter.

BTW, are these numbers normal??


thanks for the tips.
Horacio

Tuesday 13 December 2005 01:17ã€Gene Tani ã•ã‚“ã¯æ›¸ãã¾ã—ãŸ:

i just found this, tagged by 1 del.icio.us user (thx whoeve you are!),
but it really talks about the same things.

http://theexciter.com/articles/finding-leaking-ruby-objects?commented=1#c000092