closures and dynamic binding

[Michele Simionato]

So if anything were to be done to the language to
fix this, it really should be focused on fixing the
semantics of the for-loop. Unfortunately, the
fact that the loop variable leaks out of the scope
of the loop is regarded as a feature, so anything
which changes that seems to be a non-starter.

And Guido stated many times in the past that he is happy with the for
loop as it is,
so I don't think this will never change, even if the question keep
getting asked here
and there.
Notice that even generator expressions, where the loop variable does
not leak outside the loop,
have the same behavior.
The behavior of the list comprehension is a good test of how much
functional a language is;
Common Lisp and Python behaves in the same way (there is a single loop
variable which is
mutated at each iteration) wherea Scheme and Haskell introduce a new
binding at each iteration.

 

[Aaron \Castironpi\ Brady]

Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.
It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

The double lambda is conceptually more sound in some
ways, and can be made to work correctly in all cases.

The root of the problem actually has nothing to do with
lambdas or static vs. non-static scoping. It's the fact
that Python's for-loop doesn't create a new environment
for the loop variable each time around, but re-uses a
slot in the containing environment.

Contrast this with Scheme, where the equivalent of a
for-loop *does* create a new environment for each
value of the loop variable. Effectively it's using a
double lambda, except that one of the lambdas is
folded into the syntax of the loop, so you don't
notice it.

So if anything were to be done to the language to
fix this, it really should be focused on fixing the
semantics of the for-loop. Unfortunately, the
fact that the loop variable leaks out of the scope
of the loop is regarded as a feature, so anything
which changes that seems to be a non-starter.

I agree that the default argument syntax is an abuse, but it
accomplishes exactly what I want: to create a copy of a namespace. I
don't think there's a way to create a closure in Python without
another function, so you might need new syntax if you wanted to.

Otherwise, using function syntax, I want a new namespace on each
iteration that nests inside the old one, except for one variable which
overrides the outer scope. I agree that a new variable isn't the
obviously correct meaning of a for loop, and functions are the same as
a new scope, just you have to call them, so why not use them as is?

(untested)
for i in range( 3 ):
def f( n ):
def g( ):
return n
return g
closures[ i ]= f( i )

Or:

(non-standard)
for i in range( 3 ):
closure f( i ):
def g( ):
return i
return g
closures[ i ]= f

Here the only difference is whether you call 'f' or not. 'closure'
would theoretically "call itself", and make a copy of its scope upon
execution of the definition, overriding the arguments. So, functions
are the same.

 

[Terry Reedy]

Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.

I disagree. It is one way to evaluate an expression when a function is
compiled.

It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

So? Many things do not work 'in general'. If one wants multiple
closures with a variable number of arguments, one should use a def
statement and some other binding method, such as given below

Here are four ways to get the list of closures desired:
All print 0 ... 9 with for f in lst: print(f()) #3.0

lst = []
for i in range(10):
lst.append(eval("lambda: %d" %i))

# use exec instead of eval with def statement instead of lambda expression

lst = []
def f(i): return lambda: i
for i in range(10):
lst.append(f(i))

#I would most likely use this, with a def instead of lambda inside f for
any real, non-trivial example.

def populate(n):
n -= 1
if n >= 0: return populate(n)+[lambda:n]
else: return []
lst = populate(10)

# body recursion

def populate(i,n,lst):
if i < n: return populate(i+1,n,lst+[lambda:i])
else: return lst
lst = populate(0,10,[])

# tail recursion

Terry Jan Reedy

 

[Terry Reedy]

greg <[email protected]> writes:

I don't think it has anything to do with variable leaking out of the
loop. Common Lisp doesn't leak the loop variable, and it behaves the
same. It is more a result of the different views of what iteration
is. Common Lisp and Python define iteration in terms of repeating the
same instructions over and over, not different from what C does, in
which case it makes sense to reuse the same environment for all loop
passes. (Python's language ref defines that a standard "assignment"
is done for each new iteration.) In contrast, Scheme regards
iteration as a special case of recursion,

Whereas I regard iteration as 'within-namespace recursion' ;-)
which amounts to what Python, etc, do.

 

[Aaron \Castironpi\ Brady]

jhermann wrote:

Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.

I disagree.  It is one way to evaluate an expression when a function is
compiled.

It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

So?  Many things do not work 'in general'. If one wants multiple
closures with a variable number of arguments, one should use a def
statement and some other binding method, such as given below

Here are four ways to get the list of closures desired:
All print 0 ... 9 with for f in lst:  print(f()) #3.0

lst = []
for i in range(10):
     lst.append(eval("lambda: %d" %i))

# use exec instead of eval with def statement instead of lambda expression

lst = []
def f(i): return lambda: i
for i in range(10):
     lst.append(f(i))

#I would most likely use this, with a def instead of lambda inside f for
any real, non-trivial example.

def populate(n):
   n -= 1
   if n >= 0: return populate(n)+[lambda:n]
   else: return []
lst = populate(10)

# body recursion

def populate(i,n,lst):
     if i < n: return populate(i+1,n,lst+[lambda:i])
     else: return lst
lst = populate(0,10,[])

# tail recursion

Terry Jan Reedy

Is there a way to get at the 'reduce' / 'setstate' mechanism that
pickle uses without going through the serialization? I.e. doing a
serial round trip?

In either case, 'copy' module or 'loads( dumps( obj ) )' gets you a
copy of the object, but 'def' is the only way to get its own namespace.

 

[greg]

Common Lisp behaves similar to Python in this
regard:

* (loop for i from 0 to 2 collect (lambda () i))

I wouldn't call the CL loop macro the equivalent of
a for-loop. It's more like a while-loop.

The Lisp equivalent of a Python for-loop would be
to use one of the mapping functions. Then, since
the loop body is a lambda, you get a new scope for
each iteration automatically.

I don't think it has anything to do with variable leaking out of the
loop.

That's right, it doesn't. If I were designing a
for-loop from scratch, I wouldn't let it leak, but
that part seems to be necessary for backwards
compatibility.

> In contrast, Scheme regards
iteration as a special case of recursion, and R5RS "do" prescribes
assigning loop variables to "fresh locations" to match what recursion
normally does.

I'd say it prescribes that because it's useful
behaviour, not because it has anything to do with
recursion.