Memory leak or expected behavior?

[syost]

I've always seen my Apache/mod_perl httpds grow surprisingly large in
memory, and I want to fix that. But I'm seeing standalone Perl behavior
that could account for it. It seems that a lexical variable's allocated
memory isn't getting reused at all when it goes out of scope and
returns. I wrote the following code to demonstrate:

#!/usr/bin/perl
use strict;
use BSD::Resource;

foreach my $i (1..5) {
print_rusage("Before scope_it sub ");
scope_it();
print_rusage("After scope_it sub ");
print "\n";
}

sub scope_it {
print_rusage("Before instantiation");
my $tm = TestMem->new();
$tm->alloc();
print_rusage("After instantiation");
}

sub print_rusage {
my $message = shift;
my $pid = $$;
my ($usertime, $systemtime,
$maxrss, $ixrss, $idrss, $isrss, $minflt, $majflt, $nswap,
$inblock, $oublock, $msgsnd, $msgrcv,
$nsignals, $nvcsw, $nivcsw) = BSD::Resource::getrusage();
print "$message: ixrss:$ixrss; idrss:$idrss; isrss:$isrss\n"
}

package TestMem;

sub new {
return bless {};
}

sub alloc {
my $me = shift;
my @bigbuf = (1..1000000); # In real life the array is read from a
file
$me->{mem} = \@bigbuf;
}

sub DESTROY {
my $me = shift;
print STDERR "Destroying " . ref($me) . "\n";
}



Running this prints the following, where the idrss (integral or current
unshared data) number grows each time. Any idea what's happening and
how I can change my code to prevent it?

Thanks,
Steve Yost


Before scope_it sub : ixrss:48544; idrss:2058712; isrss:9472
Before instantiation: ixrss:48544; idrss:2058712; isrss:9472
After instantiation: ixrss:64288; idrss:3318740; isrss:12544
Destroying TestMem
After scope_it sub : ixrss:68880; idrss:3687388; isrss:13440

Before scope_it sub : ixrss:68880; idrss:3687388; isrss:13440
Before instantiation: ixrss:68880; idrss:3687388; isrss:13440
After instantiation: ixrss:84624; idrss:4948952; isrss:16512
Destroying TestMem
After scope_it sub : ixrss:89216; idrss:5318048; isrss:17408

Before scope_it sub : ixrss:89216; idrss:5318048; isrss:17408
Before instantiation: ixrss:89216; idrss:5318048; isrss:17408
After instantiation: ixrss:104960; idrss:6579612; isrss:20480
Destroying TestMem
After scope_it sub : ixrss:110208; idrss:7001436; isrss:21504

Before scope_it sub : ixrss:110208; idrss:7001436; isrss:21504
Before instantiation: ixrss:110208; idrss:7001436; isrss:21504
After instantiation: ixrss:125952; idrss:8263000; isrss:24576
Destroying TestMem
After scope_it sub : ixrss:130544; idrss:8632096; isrss:25472

Before scope_it sub : ixrss:130544; idrss:8632096; isrss:25472
Before instantiation: ixrss:130544; idrss:8632096; isrss:25472
After instantiation: ixrss:146288; idrss:9893660; isrss:28544
Destroying TestMem
After scope_it sub : ixrss:150880; idrss:10262756; isrss:29440

 

[syost]

Thanks for your help, Zentara. I think we're seeing a difference
between what 'ps' reports and what BSD::Resource::getrusage() is
reporting (the latter being broken, or I'm misunderstanding it). I
inserted your print "Check memory and hit enter\n"; in my loop while
keeping getrusage() in place and saw the discrepency between the two.
'ps' shows that about 50 bytes per array entry are allocated the first
time through the loop, and memory allocation doesn't grow after that.

OTOH I do see memory usage as reported by 'ps' grow incrementally in my
Apache/mod_perl httpd processes (usu. from about 10M to 50M over their
lifetimes) , so I guess I've got to look for circular references. Or
maybe there is no leak, but instead the variations in client usage of
each web page bump the memory out, and as we know it's not given back
to the OS until the process dies.
Thanks again. Any other comments welcome.

Steve

 

[syost]

(the latter being broken, or I'm misunderstanding it)

I was misunderstanding it. The BSD::Resource doc says idrss is
"integral or current unshared data" ("integral" or "current" depending
on the OS, I'm now understanding). 'man getrusage' on my FreeBSD system
show that in this case it's integral, described as follows:
"an integral value of the amount of unshared memory residing in the
data segment of a process (expressed in units of kilobytes *
ticks-of-execution)." <forehead-slap>

Steve