recursive closures?

[Michele Dondi]

I was wondering if it could be possible for a sub to return a
recursive closure. Of course this turns out to be possible, although,
to the best of my efforts, not in "one step", see e.g. the following
code:

#!/usr/bin/perl -l
# Oversimplified example

use strict;
use warnings;

sub mkrecsub {
my $k=shift;
my @memoize = ( $k => [0,1] );
my $tmp = sub {
my ($n, $func) = @_;
return $memoize[$k][$n] if defined $memoize[$k][$n];
$memoize[$k][$n] =
$func->($n-1, $func) + $func->($n-2, $func);
};
sub { $tmp->(shift, $tmp) };
}

my $Fib = mkrecsub(1);
print $Fib->($_) for 1..73;
__END__


Now I wonder if there is/could be (could not find any!) some sort of
identifier that refers to the current (anomynous) sub; say, something
like __SUB__ or __THIS__, to be used like this:

sub {
...
__SUB__->(...);
}


Michele

 

[Uri Guttman]

I was wondering if it could be possible for a sub to return a
recursive closure. Of course this turns out to be possible, although,
to the best of my efforts, not in "one step", see e.g. the following
code:

sub mkrecsub {
my $k=shift;
my @memoize = ( $k => [0,1] );
my $tmp = sub {

make my $sub be a separate line. then you can refer to it INSIDE the
closure and also assign to it. a recent post had someone else with the
same problem. you can't refer to a my variable in the same statement
that declares it.

so something like:

my $sub ;

$sub = sub{ blah; $sub->() }


should work. i have seen damian do it for sure and i think that is the
general syntax idea. search google groups for this and you may find a
hit with his code doing that.

uri

 

[Tassilo v. Parseval]

Also sprach Michele Dondi:

"MD" == Michele Dondi <[email protected]> writes:

I was wondering if it could be possible for a sub to return a
recursive closure. Of course this turns out to be possible, although,
to the best of my efforts, not in "one step", see e.g. the following

[snip]
so something like:

my $sub ;

$sub = sub{ blah; $sub->() }

should work. i have seen damian do it for sure and i think that is the

To be fair, I tried both solutions. For some reason I included in my
post the most clumsy one... however I think that it would be
*conceptually* cool to be allowed to do something like (pseudo-code):

print sub {
my $n=shift;
return $n if $n<=1;
__THIS__->($n-1) + __THIS__->($n-2);
}->(10);

This can be done by sprinkling in an always global variable to hold the
reference, such as $_:

print +(local $_ = sub {
my $n = shift;
return $n if $n<=1;
$_->($n-1) + $_->($n-2);
})->(12);
__END__
144

You need the braces, though.

Tassilo

 

[Michele Dondi]

I was wondering if it could be possible for a sub to return a
recursive closure. Of course this turns out to be possible, although,
to the best of my efforts, not in "one step", see e.g. the following

[snip]
so something like:

my $sub ;

$sub = sub{ blah; $sub->() }

should work. i have seen damian do it for sure and i think that is the

To be fair, I tried both solutions. For some reason I included in my
post the most clumsy one... however I think that it would be
*conceptually* cool to be allowed to do something like (pseudo-code):

print sub {
my $n=shift;
return $n if $n<=1;
__THIS__->($n-1) + __THIS__->($n-2);
}->(10);


Michele

 

[Brian McCauley]

my $sub ;
$sub = sub{ blah; $sub->() }

That leaks.

use Scalar::Util qw( weaken );
my $weak_sub;
my $sub = sub{ blah; $weak_sub->() }
weaken($weak_sub = $sub);

--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\

 

[Uri Guttman]

That leaks.

it isn't real code anyhow. i don't recall if damian's leaked or he
weakened it.

use Scalar::Util qw( weaken );
my $weak_sub;
my $sub = sub{ blah; $weak_sub->() }
weaken($weak_sub = $sub);

it would only leak if you let it fall out of scope without destroying it
(but how?).

and i can't seem to find damian's stuff on google. i will email him
about it.

uri

 

[Brian McCauley]

it isn't real code anyhow. i don't recall if damian's leaked or he
weakened it.


it would only leak if you let it fall out of scope without destroying it
(but how?).

"You only need weak references if you don't want use some other way to
take care of tracking and breaking the circular dependancy on every
possible exit path" is true of anything that uses weak references.

Of course if you want $sub to be a variable that is forcably undefined
implicitly at the end of a scope scope then Perl has just such a thing
- the local() variable!

Now, of course, it is an annoying miss-feature that local() can't be
applied to lexical variables so if you want to use local() you are for
forced to use package variables. What one really wants to do is...

# Won't work
my $sub;
local $sub = sub{ blah; $sub->() }

....but one is forced to do...

our $sub;
local $sub = sub{ blah; $sub->() }

--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\

 

[Uri Guttman]

"You only need weak references if you don't want use some other way to
take care of tracking and breaking the circular dependancy on every
possible exit path" is true of anything that uses weak references.

but someone pointed out that it would leak without weakening one of
its refs (it is circular of course).

well, damian posted this back to me. he refutes the need for the weak
ref or an explicit breaking of the circular refs. i haven't tried this
but i trust him to have.

------------------------

They're not correct. At least, not under 5.8.0 or later (and probably
earlier too). And you can verify that by blessing the closure and
seeing it die:

my $fib;
$fib = bless sub {
my ($n) = @_;
return 1 if $n < 2;
return $fib->($n-1) + $fib->($n-2);
}, 'Bang';

# Test...

print $fib->(7), "\n";

$fib = 7; # Should "Bang!" here

print $fib, "\n";

package Bang;

sub DESTROY { print "Bang!\n" }

------------------------

$fib inside the code is the same $fib outside of it, so the code ref
only has one ref which is in $fib. when $fib goes out of scope, the
whole pad is freed and the ref count goes to 0 like it should and it
gets destroyed.

the key to this idea in general is that you have to declare the var
(with my or our) before you refer to it inside the closure code. that
was what i was telling the OP and damian just showed a running example
with normal destruction and non-leaking semantics.

uri

 

[Anno Siegel]

[...]

Of course if you want $sub to be a variable that is forcably undefined
implicitly at the end of a scope scope then Perl has just such a thing
- the local() variable!

Now, of course, it is an annoying miss-feature that local() can't be
applied to lexical variables so if you want to use local() you are for
forced to use package variables. ...

(Just a side note)

That's not entirely true. Vexingly, you can use anonymous parts of
lexical aggregates just fine with local:

my @a = ( 123);
{
local $a[ 0] = 456;
print $a[ 0]; # 456
}
print $a[ 0]; # 123

I agree that the inability to use local() with lexicals is sometimes
annoying. It wouldn't be good style to use it all the time, but
occasionally the dynamic restoration behavior of local() is just what
the doctor ordered.

Anno

 

[Tassilo v. Parseval]

Also sprach Anno Siegel:

[...]

Of course if you want $sub to be a variable that is forcably undefined
implicitly at the end of a scope scope then Perl has just such a thing
- the local() variable!

Now, of course, it is an annoying miss-feature that local() can't be
applied to lexical variables so if you want to use local() you are for
forced to use package variables. ...

(Just a side note)

That's not entirely true. Vexingly, you can use anonymous parts of
lexical aggregates just fine with local:

my @a = ( 123);
{
local $a[ 0] = 456;
print $a[ 0]; # 456
}
print $a[ 0]; # 123

I agree that the inability to use local() with lexicals is sometimes
annoying. It wouldn't be good style to use it all the time, but
occasionally the dynamic restoration behavior of local() is just what
the doctor ordered.

Actually there is a nice solution available for the given problem. our()
has very similar scoping rules to my(). As long as the self-referential
closure isn't meant to span over several packages, it should work as
desired because now we can in fact localize.

I am quite amazed that I did eventually find a situation where our()
isn't totally useless.

Tassilo

 

[Brian McCauley]

well, damian posted this back to me. he refutes the need for the weak
ref or an explicit breaking of the circular refs. i haven't tried this
but i trust him to have.

Who is "they" here? Damian or "someone".

At least, not under 5.8.0 or later (and probably earlier too). And
you can verify that by blessing the closure and seeing it die:

The code you cite demonstrates that destruction will take place if you
_do_ explicitly break the cirular ref.

package Bang;

sub DESTROY { print "Bang!\n" }

package main;

# Test...

my $fib;
$fib = bless sub {
my ($n) = @_;
return 1 if $n < 2;
return $fib->($n-1) + $fib->($n-2);
}, 'Bang';
#############
print $fib->(7), "\n";

$fib = 7; # Should "Bang!" here

Yes indeed because here you are explicity breaking the circular
reference. If you just let $fib fall out of scope without explicitly
breaking the circular reference first then the closure would not be
destroyed (until the global destrustion phase).

Using assignment to break the circular reference like this is very
much a second-class approach. Firstly on aesthetic/maintainability
grounds: you really want whatever you have do to prevent the leak to
be lexically close to the declaration of $fib. Secondly on reliablity
grounds: in the general case you have to ensure that all exit paths
(last, next, die, return) from the code marked ############# above
pass though the statement where the circular reference is explicitly
broken.

In practice this mean that if you want to avoid weak references or
local() then you have to put the assignment in an AtExit object (or
something with equivalent semantics).

the key to this idea in general is that you have to declare the var
(with my or our) before you refer to it inside the closure code.

I think you are mixing issues here. That is necessary for a
completely different reason. The scope of Perl variable declarations
starts at the next statement.

that was what i was telling the OP and damian just showed a running
example with normal destruction and non-leaking semantics.

Where can I see this example?

--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\

 

[Uri Guttman]

Who is "they" here? Damian or "someone".

they being those (you?) who claim that recursive closures will leak.

The code you cite demonstrates that destruction will take place if you
_do_ explicitly break the cirular ref.

that is no difference from $fib going out of scope.

i just changed the code to do just that:

{
my $fib;
$fib = bless sub {
my ($n) = @_;
return 1 if $n < 2;
return $fib->($n-1) + $fib->($n-2);
}, 'Bang';

# Test...

print $fib->(7), "\n";
}

package Bang;

sub DESTROY { print "Bang!\n" }

it prints:

21
Bang!

so it sure looks like it got destroyed upon exiting scope. what more do
you want? i see no leak there

uri

 

[Lukas Mai]

i just changed the code to do just that:

{
my $fib;
$fib = bless sub {
my ($n) = @_;
return 1 if $n < 2;
return $fib->($n-1) + $fib->($n-2);
}, 'Bang';

# Test...

print $fib->(7), "\n";
}

# add this:
print "still running...\n";
END {print "global destruction:\n";}

package Bang;

sub DESTROY { print "Bang!\n" }

it prints:

21

still running...
global destruction:

Bang!

so it sure looks like it got destroyed upon exiting scope. what more do
you want? i see no leak there

It's destroyed during global destruction, not upon exiting scope.

HTH, Lukas

 

[Uri Guttman]

# add this:
print "still running...\n";
END {print "global destruction:\n";}
still running...
global destruction:

It's destroyed during global destruction, not upon exiting scope.

interesting. and if you do assign something else to $fib DESTROY is
called immediately. anyhow, in stem i have tons of circular refs
(objects inside callbacks to those object) and i use explicit loop
breaking in all the right places. i just don't like the feeling of using
weak refs.

uri

 

[Ben Morrow]

Also sprach Anno Siegel:



Actually there is a nice solution available for the given problem. our()
has very similar scoping rules to my(). As long as the self-referential
closure isn't meant to span over several packages, it should work as
desired because now we can in fact localize.

No, because this variable is now accessible as $Package::var from
anywhere in the dynamic scope of the local statement, which opens you
up to strange-action-at-a-distance effects again.

I believe that in Perl6 you can temp() (local()'s new name) lexicals.

Ben

 

[Tassilo v. Parseval]

Also sprach Ben Morrow:

No, because this variable is now accessible as $Package::var from
anywhere in the dynamic scope of the local statement, which opens you
up to strange-action-at-a-distance effects again.

Only in theory. I've never mistakenly written $Package::var when I
didn't mean it. And that's why strictures will grant you access to
global variables as long as they are package-qualified.

Tassilo

 

[Lukas Mai]

interesting. and if you do assign something else to $fib DESTROY is
called immediately.

Yes, because:

# (N) means a reference count of N
{
my $fib; # fib --> (1) undef
sub{... $fib ...} # fib --> (2) undef <-- (0) SUB
$fib = <---^ # fib --> (2) REF --> (1) SUB
# ^ |
# \-----------/
Now when exiting the scope, what's left is
(1) REF --> (1) SUB
^ |
\-----------/
i.e. a reference cycle.
But when assigning "foo" to $fib, we get
fib --> (2) "foo" (0) SUB
^ |
\-----------/
and then
fib --> (1) "foo".

At least I think that's what's happening here.

Lukas

 

[Brian McCauley]

In a distant thread long ago I wrote...

[ Of memory leaks in recursive closures ]

Of course if you want $sub to be a variable that is forcably undefined
implicitly at the end of a scope scope then Perl has just such a thing
- the local() variable!

our $sub;
local $sub = sub{ blah; $sub->() }

For the sake of the archives I think I should correct this...

The SV pointed to be *sub{SCALAR} that holds the CODE ref will _not_
get undefined (restored to its previous value) on exit from the scope.

It remains untouched. What happens is that the scalar reference held
in *sub{SCALAR} is restored to its previous value. In this way the
reference count of SV containing the CODE ref falls to zero and it is
garbage collected (and hense so it the closure).

--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\