Demo of a shrinking perl script

[Peter J. Holzer]

Some time ago we had a discussion about the FAQ "How can I free an array
or hash so my program shrinks?". I claimed that on Linux and other OSs
with modern malloc implementations (probably BSDs including MacOS and
Windows) perl can and does return memory to the OS, but couldn't come up
with an example. Now I found one:


#!/usr/bin/perl
use warnings;
use strict;

sub a {
print "a\tcall\t", vm(), "\n";
my $h;
for ('a' .. 'z') {
$h->{$_} = $_ x 1_000_000;
}
print "a\treturn\t", vm(), "\n";
return $h;
}

sub c {
print "c\tcall\t", vm(), "\n";
a();
print "c\treturn\t", vm(), "\n";
}

c();

sub vm {
open(my $fh, '<', "/proc/$$/status");
while (<$fh>) {
chomp;
return $_ if (/^VmSize:/);
}
}
__END__


On my Linux box, this prints:

c call VmSize: 5056 kB
a call VmSize: 5056 kB
a return VmSize: 31516 kB
c return VmSize: 6036 kB

demonstrating that the process grows by roughly 26 MB as the anonymous
hash is created and then shrinks again when the reference to it goes out
of scope. This also works with a simple hash or array. For simple
(large) scalars it is more complicated: Memory does get freed sometimes,
but it isn't easy to determine when.

hp

 

[DJ Stunks]

Some time ago we had a discussion about the FAQ "How can I free an array
or hash so my program shrinks?". I claimed that on Linux and other OSs
with modern malloc implementations (probably BSDs including MacOS and
Windows) perl can and does return memory to the OS, but couldn't come up
with an example. Now I found one:

#!/usr/bin/perl
use warnings;
use strict;

sub a {
print "a\tcall\t", vm(), "\n";
my $h;
for ('a' .. 'z') {
$h->{$_} = $_ x 1_000_000;
}
print "a\treturn\t", vm(), "\n";
return $h;

}

sub c {
print "c\tcall\t", vm(), "\n";
a();
print "c\treturn\t", vm(), "\n";

}

c();

sub vm {
open(my $fh, '<', "/proc/$$/status");
while (<$fh>) {
chomp;
return $_ if (/^VmSize:/);
}}

__END__

On my Linux box, this prints:

c call VmSize: 5056 kB
a call VmSize: 5056 kB
a return VmSize: 31516 kB
c return VmSize: 6036 kB

demonstrating that the process grows by roughly 26 MB as the anonymous
hash is created and then shrinks again when the reference to it goes out
of scope. This also works with a simple hash or array.

So it shrinks, but leaks memory?

I'm also not clear why you return the hash ref $h from a(), but call
a() from void context. Perhaps this is the source of the memory
leak. I'm not on linux at the moment or I'd try it myself.

-jp

 

[Peter J. Holzer]

Some time ago we had a discussion about the FAQ "How can I free an array
or hash so my program shrinks?". I claimed that on Linux and other OSs
with modern malloc implementations (probably BSDs including MacOS and
Windows) perl can and does return memory to the OS, but couldn't come up
with an example. Now I found one:

#!/usr/bin/perl
use warnings;
use strict;

sub a {
print "a\tcall\t", vm(), "\n";
my $h;
for ('a' .. 'z') {
$h->{$_} = $_ x 1_000_000;
}
print "a\treturn\t", vm(), "\n";
return $h;

}

sub c {
print "c\tcall\t", vm(), "\n";
a();
print "c\treturn\t", vm(), "\n";

}

c(); [...]
On my Linux box, this prints:

c call VmSize: 5056 kB
a call VmSize: 5056 kB
a return VmSize: 31516 kB
c return VmSize: 6036 kB

demonstrating that the process grows by roughly 26 MB as the anonymous
hash is created and then shrinks again when the reference to it goes out
of scope. This also works with a simple hash or array.

So it shrinks, but leaks memory?

I'm also not clear why you return the hash ref $h from a(), but call
a() from void context.

Leftover from an earlier version of the test program. There used to be
an intermediate sub b, but that turned out not to be necessary.

Perhaps this is the source of the memory leak.

I think the "leak" is an internal buffer used to construct
"$_ x 1_000_000" which is kept around just in case a() is called again
(it's 1_000_000 rounded up to the next page size, and if you do an strace
of the program you can see that it allocates *27* memory regions of that
size, although the hash contains only 26 entries (and when the hash is
destroyed only 26 are freed).

hp

 

[xhoster]

So it shrinks, but leaks memory?

While a single instance of shrinking is enough to say that it can indeed
shrink, a single increase in memory does not make a leak. That extra
memory can perhaps be reused later and so is not lost. And if you call c()
in a loop, you will see that this is in fact true--no leak.

c call VmSize: 9980 kB
a call VmSize: 9980 kB
a return VmSize: 36440 kB
c return VmSize: 10960 kB
c call VmSize: 10960 kB
a call VmSize: 10960 kB
a return VmSize: 36440 kB
c return VmSize: 10960 kB
c call VmSize: 10960 kB
a call VmSize: 10960 kB
a return VmSize: 36440 kB
c return VmSize: 10960 kB

Xho