Function decorator that caches function results

[Steven D'Aprano]

given that the wikipedia page says

"a closure is an abstraction representing a function, plus the
lexical environment (see static scoping) in which the function
was created."

and you're still focussing on type(function), it sure looks as if you missed
certain parts of that explanation.

let's take it again, with emphasis on some important words:

"a closure is an ABSTRACTION representing a function, PLUS the
lexical ENVIRONMENT (see static scoping) in which the function
was created."


that's because "closure" is an abstract concept.

So are functions, modules, classes, instances, ints and strings.

there is no "closure" object
in Python, just as there is no "program" object.

Clearly there is no DISTINCT closure object. If there were, I wouldn't
need to ask how one can tell them apart, because type() would just report
that one was a function and one was a closure. I don't have a problem with
that. But read on...

function objects always con-
tain all the information they need about their context, and the sum of that is
what forms the closure.

If what you say is true, then all functions are closures, and closure is
just a synonym for function, and there is no difference between a function
and a closure. Then why bother to create the term? Clearly, whoever
invented the term did so to distinguish the two.

At a practical, Python level, there is a difference between a function
before and after it gets made into a closure using, e.g. the original
poster's memoization technique. In Python at least, it is not true that a
function and a function-turned-into-closure is the same thing.

See, for example, this:-
.... def do_nothing(*args, **kwargs):
.... return fn(*args, **kwargs)
.... return do_nothing
........ return 0
....'0xf6e0a144'

In one case, the "raw" function has None stored in its func_closure
attribute. In the other, it has a tuple containing at least one object of
type cell. That cell object appears to contain a reference back to the
original "raw" function.

Now, I'll tell you what I think is going on: spam1() obviously has lexical
scope, but that scope doesn't need to be saved with the function because
it is implicitly understood by Python.

The output of closefy(), on the other hand, has scope different from
the "normal" Python scope. So its output has to package up enough
information to re-create it's lexical scope in a cell object, and
that gets stored in the output's func_closure attribute.

So no, there is no "abstract" difference between a closure and a raw
function, but there is a practical difference. Now perhaps in some other
languages there is no practical difference either, but (as far as I can
see) Python is not that language.

Am I close?

Here are another two code snippets that show something of what Python is
doing:-
.... def shrubbery():
.... return 0
.... return shrubbery
....

True

.... n = 0
.... def shrubbery():
.... return n
.... return shrubbery
....(<cell at 0xf70a2b24: int object at 0x901a158>,)

 

[Fredrik Lundh]

In effect, 'cache' and 'fn' are replaced by the objects they reference
before the cached_result function is returned.

not really; accesses to "free variables" always go via special cell objects
(rather than direct object references), and the compiler uses special byte
codes for all accesses to such objects.

this snippet illustrates the differences:

def func1():
return a

def func2(a):
return a

def wrapper(a):
def func3():
return a
return func3

def decode(func):
import dis
print func.__name__ + ":"
dis.dis(func)
print "co_freevars =", func.func_code.co_freevars
print "func_closure =", func.func_closure
print

decode(func1)
decode(func2)
decode(wrapper(10))

if you run this on a recent version of Python, you get something like:

func1:
2 0 LOAD_GLOBAL 0 (a)
3 RETURN_VALUE
co_freevars = ()
func_closure = None

func2:
5 0 LOAD_FAST 0 (a)
3 RETURN_VALUE
co_freevars = ()
func_closure = None

func3:
9 0 LOAD_DEREF 0 (a)
3 RETURN_VALUE
co_freevars = ('a',)
func_closure = (<cell at 0x0092C970: int object at 0x008A537C>,)

note the use of LOAD_DEREF for the variable access.

Other opcodes include STORE_DEREF (that updates a local variable
through a cell, used inside the "defining" scope), and MAKE_CLOSURE
(which sets up the function object for functions that actually uses
nested scopes; this is where the func_closure attribute is initialized).

(now, if I weren't limited to plain text, I could have drawn a nice little
diagram for you, but that's an entirely different thread...)

</F>

 

[Fredrik Lundh]

So are functions, modules, classes, instances, ints and strings.

they're hardly abstract; they all have distinct object implementations in the
Python runtime. there's an include file and a main implementation file for each
one of them, and you can access the type objects for each one of them from
Python code. any implementation of Python has to implement them. there's
no such thing for "closures"; they're the sum of a lot of different parts, all of
which can be implemented in many different ways.

If what you say is true, then all functions are closures, and closure is
just a synonym for function, and there is no difference between a function
and a closure.

having something isn't the same thing as being something.

if you replace "function" with "function object", you get a bit closer to the
truth. e.g. in this snippet

def wrapper(a):
def func3():
return a
return func

"func" and "wrapper" are functions, but if you call wrapper twice, you get
two different function objects, each with it's own distinct closure.

(to see this in action, take the snippet I posted in an earlier post and
add

f = wrapper(10); decode(f)
f = wrapper(10); decode(f)

to the end. this prints:

--------------------------------------------------------------------
func3:
9 0 LOAD_DEREF 0 (a)
3 RETURN_VALUE
4 LOAD_CONST 0 (None)
7 RETURN_VALUE
co_freevars = ('a',)
func_closure = (<cell at 0x0083EE50: int object at 0x00798250>,)

--------------------------------------------------------------------
func3:
9 0 LOAD_DEREF 0 (a)
3 RETURN_VALUE
4 LOAD_CONST 0 (None)
7 RETURN_VALUE
co_freevars = ('a',)
func_closure = (<cell at 0x0083EDB0: int object at 0x0079A960>,)

)

same function, same code, different closures.

(note that in CPython, functions only exist on the syntax level; when
you writing something that you think is a function, the CPython com-
piler will turn this into a code object and translate "def" to bytecode;
the actual function object isn't created until you execute those byte-
codes)

At a practical, Python level, there is a difference between a function
before and after it gets made into a closure using, e.g. the original
poster's memoization technique. In Python at least, it is not true that a
function and a function-turned-into-closure is the same thing.

function or function object? all instances of the latter have a global en-
vironment (func_globals), some also have default arguments (func_defaults),
some also have references to nested scopes (func_freevars). all of them
have all these attributes; that they're set to None when empty is just an
memory optimization.

'0xf6e0a144'

In one case, the "raw" function has None stored in its func_closure
attribute. In the other, it has a tuple containing at least one object of
type cell. That cell object appears to contain a reference back to the
original "raw" function.

func_closure is the name of an attribute. it contains information about
a certain part of a function object's closure (free variables in an existing
outer scope), but, despite its name, it's not *the* closure.

So no, there is no "abstract" difference between a closure and a raw
function, but there is a practical difference.

only if you're obsessed with CPython implementation details.

</F>

 

[Diez B. Roggisch]

Clearly there is no DISTINCT closure object. If there were, I wouldn't

need to ask how one can tell them apart, because type() would just report
that one was a function and one was a closure. I don't have a problem with
that. But read on...




If what you say is true, then all functions are closures, and closure is
just a synonym for function, and there is no difference between a function
and a closure. Then why bother to create the term? Clearly, whoever
invented the term did so to distinguish the two.

You seem to still not getting to it: they aren't supposed to be
distinguished - a function is a function. But it _can_ have a closure,
which (shortly spoken) enriches it's local variables lookup. Speaking in
terms of "is a" could be seen as some inheritance relation. Which
functions and functions with a closure _could_ have if one wanted to.
But that is true for other OO-concepts two. See below.

At a practical, Python level, there is a difference between a function
before and after it gets made into a closure using, e.g. the original
poster's memoization technique. In Python at least, it is not true that a
function and a function-turned-into-closure is the same thing.

See, for example, this:-

<snip>

Nope. You mix things here. spam1 is a function. spam2 is actually bound
to the function-object created by do_nothing - a function that has a
closure because it is nested in closefy, and accesses variables from its
outer lexical scope - thus i needs a closure. Part of do_nothing's
closure is spam1 (or, better to say, the function reference that has
been originally bound to spam1 - spam1 is just a name). Important is
that the closure gets created at runtime, so each returned instance of
do_nothing has its own. But the original spam1 implementation is
untouched contrary to what you state above. It still is teh same
thing, and has no closure.

Consider this:

class Foo(object):
pass

f = Foo()
g = Foo()
g.bar = "whatever"

f and g are both Foo - but one has a different attribute set. And if Foo
had some semantics that did something different based on bar, would you
also say they're supposed have two different classes? You could in fact
do that - create one class for each incarnation of possible state. But
then the concept of classes and objects gets somewhat blurred, as each
instance would have its own class. The same is true for functions.

So no, there is no "abstract" difference between a closure and a raw
function, but there is a practical difference. Now perhaps in some other
languages there is no practical difference either, but (as far as I can
see) Python is not that language.

Am I close?

Closer

Regards,

Diez

 

[Steven D'Aprano]

nowhere. at least no officially; see "Rebinding names in enclosing
scopes" in the design document for a discussion:

http://www.python.org/peps/pep-0227.html

That's interesting to read, and does clarify a few things that weren't
straight in my head, but I notice that the author, Jeremy Hylton,
distinguishes between Python creating function definitions that are
closures from those that aren't:

"Each def or lambda expression that is executed will create a closure if
the body of the function or any contained function has free variables."

Presumably that means that if there are no free variables, no closure
is created.

no, because Python doesn't prevent you from digging into the
internals:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440515

That's blackest voodoo, and as the warning says, could cause Python to
sigfault if you get the opcodes wrong.

I think that this is close enough to "really private" to satisfy most
people. Maybe even Paul Rubin *wink*

 

[Fredrik Lundh]

"Each def or lambda expression that is executed will create a closure if
the body of the function or any contained function has free variables."

Presumably that means that if there are no free variables, no closure
is created.

you're quoting selectively; the word "closure" isn't used anywhere
in the design document except for a section that talks about "flat
closures", which is an implementation approach used in CPython
(as the sentence before the one you quoted explains).

I don't think it's necessarily a good idea to treat a reference to an
algorithm and an attribute on an object in a specific implementation
as the definition of an computer term...

</F>

 

[Ron Adam]

Ron Adam wrote:




not really; accesses to "free variables" always go via special cell objects
(rather than direct object references), and the compiler uses special byte
codes for all accesses to such objects.

this snippet illustrates the differences:

def func1():
return a

def func2(a):
return a

def wrapper(a):
def func3():
return a
return func3

def decode(func):
import dis
print func.__name__ + ":"
dis.dis(func)
print "co_freevars =", func.func_code.co_freevars
print "func_closure =", func.func_closure
print

decode(func1)
decode(func2)
decode(wrapper(10))

if you run this on a recent version of Python, you get something like:

func1:
2 0 LOAD_GLOBAL 0 (a)
3 RETURN_VALUE
co_freevars = ()
func_closure = None

func2:
5 0 LOAD_FAST 0 (a)
3 RETURN_VALUE
co_freevars = ()
func_closure = None

func3:
9 0 LOAD_DEREF 0 (a)
3 RETURN_VALUE
co_freevars = ('a',)
func_closure = (<cell at 0x0092C970: int object at 0x008A537C>,)

note the use of LOAD_DEREF for the variable access.

Other opcodes include STORE_DEREF (that updates a local variable
through a cell, used inside the "defining" scope), and MAKE_CLOSURE
(which sets up the function object for functions that actually uses
nested scopes; this is where the func_closure attribute is initialized).

(now, if I weren't limited to plain text, I could have drawn a nice little
diagram for you, but that's an entirely different thread...)

</F>

Thanks Fred, That's something useful to know when working with
decorators I think. And it should answer Stevens question too.

Abstract terminology is great once you already know the things and
concepts it refers to. But if you don't, finding where the abstract
connects to reality is sometimes not easy. There's nothing better than
a good example to illistrate the concept. ;-)

Cheers,
Ron

 

[Steven D'Aprano]

you're quoting selectively; the word "closure" isn't used anywhere
in the design document except for a section that talks about "flat
closures", which is an implementation approach used in CPython
(as the sentence before the one you quoted explains).

Given that when I asked about closures, you pointed me at that document
for further information, I thought maybe I could take it as authoritative.

I don't think it's necessarily a good idea to treat a reference to an
algorithm and an attribute on an object in a specific implementation as
the definition of an computer term...

Are you suggesting that Jeremy Hylton got it wrong? That's fine if he did,
but it would help if you said so rather than beating around the bush.

Is it correct to say that Python creates closures *ever*?

Is it correct to say that Python creates closures when a def or lambda
statement is executed?

Is it correct to say that Python *always* creates a closure whenever a def
or lambda is executed? Or is it only for *certain* defs/lambdas?

Is it correct to talk about functions *having* closures, or that they
*are* closures, or that the function is *part of* a closure?

If all functions are closures, what does it mean to say that some
languages (e.g. Java, C++, C#) don't have closures but merely simulate
them? Are there any languages that don't have closures?


Thanks for your patience, I'm sure I'll get there eventually.

 

[Steven D'Aprano]

You seem to still not getting to it: they aren't supposed to be
distinguished - a function is a function. But it _can_ have a closure,
which (shortly spoken) enriches it's local variables lookup.

[penny drops] Now we're getting somewhere... a closure is something
_added_ to a function. So we should talk about functions-without-closures
and functions-with-closures.

Speaking in
terms of "is a" could be seen as some inheritance relation.

[penny rises again] Dammit, just when I thought I got it. So a closure is
like a subclass of function? Functions are a higher level classification,
like mammals. There are mammals that are primates (functions that _are_
closures) and mammals that aren't (functions that _aren't_ closures).

Which
functions and functions with a closure _could_ have if one wanted to.
But that is true for other OO-concepts two. See below.



<snip>

Nope. You mix things here. spam1 is a function.
Yes.

spam2 is actually bound
to the function-object created by do_nothing -
Yes.

a function that has a closure because it is nested in closefy, and
accesses variables from its outer lexical scope - thus i needs a
closure.

Right! So closures are something which functions _have_ -- which is why
Python functions have an attribute called func_closure, which is empty in
functions without closures, and contain a tuple of cells in those with
them.

Now somebody is going to tell me this is an implementation detail and it
isn't true in the general language independent case...

Part of do_nothing's closure is spam1 (or, better to say, the
function reference that has been originally bound to spam1 - spam1 is
just a name).

Yes, I can see that.

Important is that the closure gets created at runtime, so
each returned instance of do_nothing has its own. But the original spam1
implementation is untouched contrary to what you state above. It still
is teh same thing, and has no closure.

Consider this:

class Foo(object):
pass

f = Foo()
g = Foo()
g.bar = "whatever"

f and g are both Foo

Now you're trying to trick me... they are both *instances* of Foo, not
Foo. Foo is a class, and f and g are not classes, let alone the same class.

- but one has a different attribute set. And if Foo
had some semantics that did something different based on bar, would you
also say they're supposed have two different classes? You could in fact
do that - create one class for each incarnation of possible state. But
then the concept of classes and objects gets somewhat blurred, as each
instance would have its own class. The same is true for functions.

Ah, but in languages like Pascal that can take functions as arguments, but
can't return functions as output, *all* functions have to be created
separately. To take your analogy and run with it, Pascal would be like a
language that didn't allow f and g to have different attributes unless
they belonged to different classes.

Python is not like that, which is why you can write a function to return
functions (a factory function?). If the output function needs access to
the namespace of the factory function, Python adds a closure to that
output function, giving it access to the objects in that namespace.

 

[Steven D'Aprano]

only if you're obsessed with CPython implementation details.

No. I'm obsessed with finding out what closures are, since nobody seems to
have a good definition of them!

However, I have learnt some things: closures are something which functions
HAVE, not ARE. The func_closure attribute is just part of the
implementation of the closure. Some languages have closures and some
don't. And a closure is something that lets a function object access the
lexical scope that existed when the function object was created, e.g. the
namespace of the function which created it.

Am I getting there now?

 

[Piet van Oostrum]

SD> [penny drops] Now we're getting somewhere... a closure is something
SD> _added_ to a function. So we should talk about functions-without-closures
SD> and functions-with-closures.

Well, I have never heard that definition. For more than half of my life I
only have heard the definition that a closure is the combination of a
function (i.e. its code) plus an environment containing the unbound
variables. Depending on the language the variables may be bound to values,
memory locations or some other things.

 

[Paul Rubin]

No. I'm obsessed with finding out what closures are, since nobody seems to
have a good definition of them!

Why don't you read SICP:

http://mitpress.mit.edu/sicp/full-text/book/book.html

You will be a much wiser person for having done so.

However, I have learnt some things: closures are something which
functions HAVE, not ARE.

Nah, that describes the data layout through which CPython implements
closures, not the bigger idea of what closures are.

The func_closure attribute is just part of the implementation of the
closure. Some languages have closures and some don't. And a closure
is something that lets a function object access the lexical scope
that existed when the function object was created, e.g. the
namespace of the function which created it.

I'd say, the closure is the function's executable code, plus the
lexical environment that got recorded at the time the function was
created.

 

[Paul Rubin]

Is it correct to say that Python *always* creates a closure whenever a def
or lambda is executed? Or is it only for *certain* defs/lambdas?

The word closure is being used two different ways. First of all the
computer-science term "closure" which means a combination of a
function and its lexical environment. Second of all the data field
named "closure" which lives inside certain function objects in Python.
The second one is created only in certain Python functions but that's
an implementation detail. It was added because in the old days,
Python didn't support closures at all, so implementing them was done
straightforwardly by adding a field that contains the closure
environment, for Python functions that needed it.

So the answer to your first question above is "always" in the CS sense
and "only for certain defs/lambdas" in the implementation sense.

Is it correct to talk about functions *having* closures, or that they
*are* closures, or that the function is *part of* a closure?

The function is part of the closure. I think you're a little confused
by implementation issues. In Python, functions are first-class
objects, like class instances or strings. If you say
x = 'foo' + 'bar'
a new string is created out of nowhere. If you say
def f(x): return x+3
a new function is created out of nowhere. Lambda notation maybe
shows more clearly that functions are values:
f = (lambda x: x+3)
is the same as the "def f" above.

Let's go back a step. Do you understand what a free variable is?
Let's say:
def f():
print x
f refers to x but x is not defined in f. We say x is "free" in f.

In the old days, this could only mean one thing: x was a global
variable. Python didn't have nested scopes. If a variable wasn't
found in a function's locals, it could only come from the globals
that are shared across all functions.

Python now has nested scopes. That means x doesn't have to come
from the globals, it can be a local from some surrounding function:
def g(n):
x = n # x is local to g
def f():
print x
f()
When you run g(3), Python prints 3. That's not too confusing, right?

Suppose instead of calling f, you return it. You can do that in Python:
def g(n):
x = n # x is local to g
def f():
print x
return f

h3 = g(3) # h3 is the function g creates when you run it

What will happen when you call h3()? It prints 3. How did it know to
do that? Because h3 doesn't just contain executable bytecode, it also
contains f's lexical environment from when g ran. (f's lexical
environment means the values of f's free variables from outer scopes
at the time f was created). Now suppose you say:

h5 = g(5) # make a new function

See, h3 and h5 are two different functions:

h3() # still prints 3
h5() # prints 5

They contain the same executable code, but different lexical environments.

The combination of executable code plus lexical environment is called a
closure (CS term). In Python, every function is a (CS) closure. The
environment lives in a field called func_closure or something like that.

However, when there are no free variables, Python doesn't bother
creating that field. That was ALWAYS the case in old versions of
Python. Python didn't recognize nested scopes; there were only locals
and globals.

If all functions are closures, what does it mean to say that some
languages (e.g. Java, C++, C#) don't have closures but merely
simulate them? Are there any languages that don't have closures?

Those languages listed there don't have closures. They don't even
have first-class functions. Imagine what the h3 and h5 examples from
above would do in those languages.

To be fair, there's an argument that closures are un-Pythonic: you can
use class instances instead. You could say:

class G:
def __init__(self, n): self.x = n
def __call__(self): return self.x
h3 = G(3)
h5 = G(5)

and h3() prints 3 and h5() prints 5, just like the corresponding
closures. But now you've polluted the name space with this extra
class G, and (depending on the situation) maybe you've written more
cumbersome code. It's partly a matter of taste and preferences. To
Lisp and Scheme users, closures are very natural. To OOP programmers,
maybe they're confusing and unnecessary.

 

[Diez B. Roggisch]

[penny drops] Now we're getting somewhere... a closure is something
_added_ to a function. So we should talk about functions-without-closures
and functions-with-closures.
Yes.

Speaking in
terms of "is a" could be seen as some inheritance relation.

[penny rises again] Dammit, just when I thought I got it. So a closure is
like a subclass of function? Functions are a higher level classification,
like mammals. There are mammals that are primates (functions that _are_
closures) and mammals that aren't (functions that _aren't_ closures).

Hrmpf. No, I wanted to say the exact opposite. My bad - just ignore that
paragraph of mine in the earlier post, ok?

Right! So closures are something which functions _have_ -- which is why
Python functions have an attribute called func_closure, which is empty in
functions without closures, and contain a tuple of cells in those with
them.
Yes.


Now somebody is going to tell me this is an implementation detail and it
isn't true in the general language independent case...

Paul Rubin did so, and I guess it is, although I don't know for sure.

Now you're trying to trick me... they are both *instances* of Foo, not
Foo. Foo is a class, and f and g are not classes, let alone the same class.

As two functions are instances of the "class of functions". I just
wanted to point out that two instances of one class can have differnet
state. The analogy isn't perfect, as this special state "closure" isn't
available easily for you to alter, and gets created behind the scenes.
But as each function with a closure is created on the fly - same code,
different closure data - it sort of fitted, I hoped.

Ah, but in languages like Pascal that can take functions as arguments, but
can't return functions as output, *all* functions have to be created
separately. To take your analogy and run with it, Pascal would be like a
language that didn't allow f and g to have different attributes unless
they belonged to different classes.

Python is not like that, which is why you can write a function to return
functions (a factory function?). If the output function needs access to
the namespace of the factory function, Python adds a closure to that
output function, giving it access to the objects in that namespace.

Hm, no. You can return functions in other languages, e.g. C. Don't know
pascal good enough. But they will always be the same without any state
associated to them, whereas in python you have some state. The same way
as bound methods know about their first arguments, functions with
closures know about - well, their closure

A closure is really only an environment, adding values to locals() -
implemented as cells in python - that adds to the local variables when
the function is executed, and is dynamically created when a function
with a closure is defined. This works because of the interpreted nature
of python.


Regards,

Diez

 

[Tom Anderson]

No. I'm obsessed with finding out what closures are, since nobody seems
to have a good definition of them!

On the contrary - the problem is that several people have good but
incompatible definitions of them!

I think you pretty much understand the mechanics of what's going on; i've
spent god knows how long trying to write a clear but accurate definition
of closures, but i can't, so i'm just going to say that (a) closures are
functions, and (b) the things in func_closure are not closures - they're
the variables over which a closure (the function you're inspecting) is
closed; this is just sloppy terminology on the part of python's
implementors.

Okay, a crack at a definition: a closure is a function in which some of
the variable names refer to variables outside the function. And i don't
mean global variables - i mean somebody else's locals; call them 'remote
variables'. The 'somebody else' who owns those locals is a function whose
scope encloses the definition of the closure function. A crucial point is
that these names keep working even after the scope where they started out
dies - when a closure escapes the 'mother scope' that houses its remote
variables, those variables effectively transcend, becoming, well, i don't
know. Nonlocal variables or something.

For some reason, this makes me think of Stephen Baxter novels. And a
little of H. P. Lovecraft. I guess the point is that you shouldn't delve
too deeply into the mysteries of functional programming if you wish to
retain your humanity.

tom

 

[Paul Rubin]

Okay, a crack at a definition: a closure is a function in which some
of the variable names refer to variables outside the function.

That's misleading, I'd say a closure is a combination of a function
(executable code) and a lexical environment (the values of the
function's free variables, as taken from surrounding scopes at the
time the function was created). For example:
def f(n):
def g(): return n
return g
h1 = f(1)
h2 = f(2)

h1 and h2 are two different closures. They have the same executable
code but their environments are different. In h1, n=1, but in h2, n=2.
So, h1() will return 1 but h2() will return 2. Is there really anything
confusing about this? All that's happened is that when you call f, f
allocates a memory slot for n. g makes a reference to the slot and
then f returns. Since the reference to the slot still exists, the slot
doesn't get GC'd. When you call f again, it allocates a new slot.

This is all described in SICP (mitpress.mit.edu/sicp).

 

[Piet van Oostrum]

PR> Tom Anderson <[email protected]> writes:

PR> That's misleading, I'd say a closure is a combination of a function
PR> (executable code) and a lexical environment [snip]
PR> This is all described in SICP (mitpress.mit.edu/sicp).

Where the word closure is used for something completely different (it
mentions that others use the word closure as we do here).
:=(

 

[Fredrik Lundh]

Okay, a crack at a definition: a closure is a function in which some of
the variable names refer to variables outside the function. And i don't
mean global variables - i mean somebody else's locals; call them 'remote
variables'.

in Python, the global variables are someone else's locals.

</F>

 

[Tom Anderson]

That's misleading,

You're right. I don't think it's wrong, but it depends on the reader
knowing what i mean by 'variable'. I didn't make it clear that variables
live in an *invocation* of a function, not a definition. IOW, in this
code:

def foo():
x = 1
foo()
foo()

There are two occurrances of assignment, and they are to two *different*
variables.

Hmm. Maybe i *am* wrong - looking at your example:

For example:
def f(n):
def g(): return n
return g
h1 = f(1)
h2 = f(2)

h1 and h2 are closures, while g is "a function in which some of the
variable names refer to variables outside the function", but is not a
closure.

So a closure is something that is created when a function is taken out of
its lexical environment (through being returned, or passed downwards, or
stored on the heap); when you take it out, it retains a connection to that
environment. What we need now is a good metaphor for this ...

I'd say a closure is a combination of a function (executable code) and a
lexical environment (the values of the function's free variables, as
taken from surrounding scopes at the time the function was created).

I was trying to avoid the word 'lexical', since a lot of people aren't too
clear on what that means.

h1 and h2 are two different closures. They have the same executable
code but their environments are different. In h1, n=1, but in h2, n=2.
So, h1() will return 1 but h2() will return 2. Is there really anything
confusing about this? All that's happened is that when you call f, f
allocates a memory slot for n. g makes a reference to the slot and
then f returns. Since the reference to the slot still exists, the slot
doesn't get GC'd. When you call f again, it allocates a new slot.

That's quite a good way of explaining it. The thing about closures is that
they're really obvious when you actually write them, but rather tricky to
define in words. So, perhaps the best explanation is to walk the reader
through an example.

This is all described in SICP (mitpress.mit.edu/sicp).

A lot of things are described in SICP. ISTM that someone should not have
to read the whole of SICP to understand what closures are.

tom