Memory leak with Inline::C

[Anno Siegel]

I'm seeing a memory leak with the Inline code below. The sub by_inline()
returns a varying number of scalars, which is probably the crux of the
matter.

The code works as expected (returning the exponents of 2 used in the
binary representation of the argument), but the loop leaks. It leaks
just the same when by_inline() is called in void context.

I don't see anything obviously wrong with the code, in particular the
newly created SVs are carefully mortalized, but I'd like a second opinion.

I'd also be grateful if one or the other could find the time to run
the code and report back. For me, the process grows to 500+ MB before
the loop runs out (I kill it when swapping becomes audible).

Conditions here:
Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3

Update:
It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.

Anno

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;
use Vi::QuickFix;

my @res = by_inline( int rand 2 ** 32) for 1 .. 100_000;

use Inline C => <<EOC;
void by_inline( int x) {
int i;
Inline_Stack_Vars;
Inline_Stack_Reset;
i = 0;
while ( x ) {
if ( x & 1 ) {
Inline_Stack_Push( sv_2mortal( newSViv( i)));
}
x >>= 1;
i ++;
}
Inline_Stack_Done;
}
EOC
__END__

 

[A. Sinan Unur]

On 11 Jul 2004, you wrote in comp.lang.perl.misc:

Anno:

I haven't had a chance to take a serious look (I am taking baby steps
into finding out about Perl internals), but

I'd also be grateful if one or the other could find the time to run
the code and report back. For me, the process grows to 500+ MB before
the loop runs out (I kill it when swapping becomes audible).

Conditions here:
Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3

Update:
It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.

FBSD 5.2, Perl 5.8.4, Inline 0.44, gcc 3.3

script gets killed by OS rather quickly. Didn't try it on my Win XP.

 

[ctcgag]

I'm seeing a memory leak with the Inline code below. The sub by_inline()
returns a varying number of scalars, which is probably the crux of the
matter.

....

The problem is not with Inline::C, it is with plain C. Certain arguments
to the function throw your C code into an infinite loop. I suspect it is
due to the fact that not all numbers up to 2**32 fit into a signed int.

I changed the signature line to:
void by_inline (unsigned x) {

And the problem went away.

Xho

 

[ctcgag]

I'm seeing a memory leak with the Inline code below. The sub by_inline()
returns a varying number of scalars, which is probably the crux of the
matter.

....

The problem is not with Inline::C, it is with plain C. Certain arguments
to the function throw your C code into an infinite loop. I suspect it is
due to the fact that not all numbers up to 2**32 fit into a signed int.

I changed the signature line to:
void by_inline( unsigned int x) {

And the problem went away.

Xho

 

[Tassilo v. Parseval]

Also sprach Anno Siegel:

I'm seeing a memory leak with the Inline code below. The sub by_inline()
returns a varying number of scalars, which is probably the crux of the
matter.

Actually, the problem is that your program may produce infinite loops.

The code works as expected (returning the exponents of 2 used in the
binary representation of the argument), but the loop leaks. It leaks
just the same when by_inline() is called in void context.

I don't see anything obviously wrong with the code, in particular the
newly created SVs are carefully mortalized, but I'd like a second opinion.

I'd also be grateful if one or the other could find the time to run
the code and report back. For me, the process grows to 500+ MB before
the loop runs out (I kill it when swapping becomes audible).

Conditions here:
Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3

Update:
It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.

Anno

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;
use Vi::QuickFix;

my @res = by_inline( int rand 2 ** 32) for 1 .. 100_000;

A value in the above range might not fit into a signed 32bit integer. In
this case it gets wrapped around and 'x' in the below C function ends up
being negative. So 'x' will eventually become -1 and never turn zero.
And there you have your infinite while-loop.

use Inline C => <<EOC;
void by_inline( int x) {
int i;
Inline_Stack_Vars;
Inline_Stack_Reset;
i = 0;
while ( x ) {
if ( x & 1 ) {
Inline_Stack_Push( sv_2mortal( newSViv( i)));
}
x >>= 1;
i ++;
}
Inline_Stack_Done;
}
EOC
__END__

One fix would be to increase the range of your numbers:

void by_inline(double X) {
long long x = X;
...
}

Tassilo

 

[Tassilo v. Parseval]

Also sprach Purl Gurl:

Range of values for signed versus unsigned are also
system dependent which may skew results for readers.

Default syntax is signed int for int declarations.

Semantic actually. The range of data-types has nothing to do with
syntax.

These are typical limits for ANSI C found in limits.h
but may vary on specific machine types,

CHAR_BIT 8 /* number of bits in a 'char' */
SCHAR_MIN -127 /* minimum value for 'signed char' */
SCHAR_MAX 127 /* maximum value for 'signed char' */
UCHAR_MAX 255 /* maximum value for 'unsigned char' */
SHRT_MIN -32767 /* minimum value for '(signed) short (int)' */
SHRT_MAX 32767 /* maximum value for '(signed) short (int)' */
USHRT_MAX 65535 /* maximum value for 'unsigned short' */
INT_MIN -32767 /* minimum value for '(signed) int' */
INT_MAX 32767 /* maximum value for '(signed) int' */
UINT_MAX 65535 /* maximum value for 'unsigned int' */
LONG_MIN -2147483647 /* minimum value for '(signed) long (int)' */
LONG_MAX 2147483647 /* maximum value for '(signed) long (int)' */
ULONG_MAX 4294967295 /* maximum value for 'unsigned long (int)' */

Nowadays (per C99), there should also be a LLONG_MAX and siblings
available which represents a 64bit integer. This should do for most
purposes.

Tassilo

 

[Anno Siegel]

...

The problem is not with Inline::C, it is with plain C. Certain arguments
to the function throw your C code into an infinite loop. I suspect it is
due to the fact that not all numbers up to 2**32 fit into a signed int.

Yes. C's >> is an arithmetic shift, and pulls in one-bits with negative
integers. (Perl's >> appears to be a logical shift.) So a single
call got stuck and kept allocating. That would "leak" some, yes.

I changed the signature line to:
void by_inline( unsigned int x) {

And the problem went away.

Thank you, thank you!

Boy did I misread my results. Apologies to Inline, Ingy, and anyone
else concerned. They are absolutely innocent of leaking.

Anno

 

[Anno Siegel]

Also sprach Anno Siegel:


Actually, the problem is that your program may produce infinite loops.

Indeed. I replied to Xho, who made the same observation. Thanks
again.

Anno

 

[Tassilo v. Parseval]

Also sprach Purl Gurl:

Yes it does. If you declare int default syntax behavior
is of signed int type syntax.

The syntax you use to declare an int directly affects
its value range.

This confinement of the value does not happen at the stage of syntactic
analysis. Simple example:

x >>= 1;

Syntactically the above is correct. It is still valid syntax if you do

double x;

/* ... */

x >>= 1;

Now however it contradicts the defined semantics of the right-shift
operator which cannot be applied to floating point values.

If your intent is to nitpick over choice of words,
syntax versus semantic, I am not interested.

If you had ever indulged with compiler construction and tools such as
yacc, you'd know that this is hardly just a nitpick over choice of words.

This is how machine dependency comes into play. 16 bit, 32 bit
and 64 bit representations. Various effects can be observed,
additionally, by the type of compiler you use and the command
syntax you use.

Not if the compiler conforms to Ansi99. If it does, 'long long' will
always be 64bit, regardless of the machine. It's one of the rare cases
about C where one doesn't have to worry about the machine, only about
the compiler.

ANSI C, however, will always be fashionable as will
Perl, save perhaps for when Perl 6 comes into large
scale usage. Perl 6 is like jumping from ANSI C into
Microsoft .net programming; bloatware.

Remains the question, which ANSI. As far as I know, Microsoft's C
compiler is still C89 (or not even that). So very often it's also the
question how quickly vendors will react to new standards.

Tassilo

 

[Lukas Mai]

Tassilo v. Parseval schrob:
[...]

Not if the compiler conforms to Ansi99. If it does, 'long long' will
always be 64bit, regardless of the machine. It's one of the rare cases
about C where one doesn't have to worry about the machine, only about
the compiler.

'long long' will be _at least_ 64 bits wide. It could be more.

HTH, Lukas