Closures in leu of pointers?

[cts.private.yahoo]

Hi,

I'd like to use closures to set allow a subroutine to set variables in its caller, in leu of pointers. But I can't get it to work. I have the following test pgm, but I can't understand its behaviour:

It uses a function p2() from the module modules.closure1b:

def p2 (proc):
proc ("dolly")

I thought the following worked like I expected it to:


from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
print p2 (p11)

p1('hello')

$ python closures1c.py
p1: entered: hello
p11: entered: dolly
hellodollyworld
None

In other words, p1() is passed "hello" for msg1, "world" goes to the local msg3 and then p11() is invoked out of a remote module and it can access not only its own argument (msg2) but also the variables local to p1(): "hellodollyworld".

But if I try to set the variable local to p1(), all of a sudden python seems to forget everything we agreed on.

If I add this line to the script above:
msg3 = "goodbye"
as follows:

from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
msg3 = "goodbye" # <- new
print p2 (p11)

p1('hello')

then all of a sudden, I get this:

p1: entered: hello
p11: entered: dolly
Traceback (most recent call last):
File "closures1c.py", line 13, in <module>
p1('hello')
File "closures1c.py", line 11, in p1
print p2 (p11)
File "/home/mellman/eg/python/modules/closures1b.py", line 2, in p2
proc ("dolly")
File "closures1c.py", line 9, in p11
print msg1 + msg2 + msg3
UnboundLocalError: local variable 'msg3' referenced before assignment


Huh? msg3 isn't more referenced than it was before!

Can anyone explain this to me?

 

[Fábio Santos]

Hi,

I'd like to use closures to set allow a subroutine to set variables in

its caller, in leu of pointers. But I can't get it to work. I have the
following test pgm, but I can't understand its behaviour:

It uses a function p2() from the module modules.closure1b:

def p2 (proc):
proc ("dolly")

I thought the following worked like I expected it to:


from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
print p2 (p11)

p1('hello')

$ python closures1c.py
p1: entered: hello
p11: entered: dolly
hellodollyworld
None

In other words, p1() is passed "hello" for msg1, "world" goes to the

local msg3 and then p11() is invoked out of a remote module and it can
access not only its own argument (msg2) but also the variables local to
p1(): "hellodollyworld".

But if I try to set the variable local to p1(), all of a sudden python

seems to forget everything we agreed on.

If I add this line to the script above:
msg3 = "goodbye"
as follows:

from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
msg3 = "goodbye" # <- new
print p2 (p11)

p1('hello')

then all of a sudden, I get this:

p1: entered: hello
p11: entered: dolly
Traceback (most recent call last):
File "closures1c.py", line 13, in <module>
p1('hello')
File "closures1c.py", line 11, in p1
print p2 (p11)
File "/home/mellman/eg/python/modules/closures1b.py", line 2, in p2
proc ("dolly")
File "closures1c.py", line 9, in p11
print msg1 + msg2 + msg3
UnboundLocalError: local variable 'msg3' referenced before assignment


Huh? msg3 isn't more referenced than it was before!

Can anyone explain this to me?

The fact that msg3 is assigned to in that scope makes it a local variable.
It doesn't matter if the assignment happens later. Python will treat it as
local, and so won't look for it outside the local scope/closure.

The fix is to declare msg3 as global, I think.

 

[Peter Otten]

I'd like to use closures to set allow a subroutine to set variables in its
caller, in leu of pointers.

"leu"? Must be a Fench word

But I can't get it to work. I have the
following test pgm, but I can't understand its behaviour:

It uses a function p2() from the module modules.closure1b:

def p2 (proc):
proc ("dolly")

I thought the following worked like I expected it to:


from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
print p2 (p11)

p1('hello')

$ python closures1c.py
p1: entered: hello
p11: entered: dolly
hellodollyworld
None

In other words, p1() is passed "hello" for msg1, "world" goes to the local
msg3 and then p11() is invoked out of a remote module and it can access
not only its own argument (msg2) but also the variables local to p1():
"hellodollyworld".

But if I try to set the variable local to p1(), all of a sudden python
seems to forget everything we agreed on.

If I add this line to the script above:
msg3 = "goodbye"
as follows:

from modules.closures1b import p2

def p1(msg1):
msg3 = "world"
print "p1: entered: ", msg1
def p11(msg2):
print "p11: entered: ", msg2
print msg1 + msg2 + msg3
msg3 = "goodbye" # <- new
print p2 (p11)

p1('hello')

then all of a sudden, I get this:

p1: entered: hello
p11: entered: dolly
Traceback (most recent call last):
File "closures1c.py", line 13, in <module>
p1('hello')
File "closures1c.py", line 11, in p1
print p2 (p11)
File "/home/mellman/eg/python/modules/closures1b.py", line 2, in p2
proc ("dolly")
File "closures1c.py", line 9, in p11
print msg1 + msg2 + msg3
UnboundLocalError: local variable 'msg3' referenced before assignment


Huh? msg3 isn't more referenced than it was before!

Can anyone explain this to me?

You picked the most obnoxious variable names I can think of, but the actual
problem is simple:

Python statically determines the scope of a variable. If you rebind a name
it becomes a local variable unless you explicitly declare it as global or --
in Python 3 -- as nonlocal. For example:

Wrong:
.... n = 0
.... def inner():
.... print(n)
.... n += 1
.... return inner
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in inner
UnboundLocalError: local variable 'n' referenced before assignment

With nonlocal declaration (Python 3 only):.... n = 0
.... def inner():
.... nonlocal n
.... print(n)
.... n += 1
.... return inner
....2

With a mutable variable as a pseudo-namespace (workaround for Python 2):
.... n = [0]
.... def inner():
.... print n[0]
.... n[0] += 1
.... return inner
....1

 

[cts.private.yahoo]

Well, it would have been French if I had spelled it right - since you force me overcome my laziness, I see I should have spelled it lieu ...

Thank you. You reminded me of the (weak) workaround of using arrays and confirmed my suspicion that I although I can read the variable, I won't be able to write to it. I still don't understand why not, though...

As for python 3 ... "nonlocal"? I see I'm not alone in picking obnoxious names ...

 

[cts.private.yahoo]

Alas, one reason it's a weak workaround is that it doesn't work - at least, not how I wish it would:


$ cat ptrs

x = 34

def p1 (a1):

a1[0] += 12

p1 ([x])

print (x)

$ python ptrs
34

 

[Peter Otten]

As for python 3 ... "nonlocal"? I see I'm not alone in picking obnoxious
names ...

tous chez...

Alas, one reason it's a weak workaround is that it doesn't work - at
least, not how I wish it would:

$ cat ptrs

x = 34

def p1 (a1):

a1[0] += 12

p1 ([x])

print (x)

$ python ptrs
34

That doesn't work with 'nonlocal' either.

You can compare Python's names with (automatically dereferenced and
garbage-collected) pointers, but there is no way to have a name refer to
another name. The C-like

f(var *value)
{
value = ...
}

is the only way to pass a variable in Python. There is no way to emulate

f(var **value)
{
*value = ...
}

When this came up a few days ago I linked to

http://learntofish.wordpress.com/2012/01/09/call-by-object-reference-call-by-sharing/

but I was actually looking for the classic, so there:

http://python.net/~mwh/hacks/objectthink.html

PS: If you're reading this and love the French language -- I am deeply sorry
for the pain I'm causing you...

 

[cts.private.yahoo]

"PS: If you're reading this and love the French language -- I am deeply sorry
for the pain I'm causing you..."

It's obviously a team effort...

My French ain't so hot, either. I had to google your "tout chez" until I ran into the explanation:

hallo also ich gucke super gerne two and a half men und da wird öfters tout chez (keine ahnung ob es so geschrieben wird) gesagt. ich hab gegooglet aber nichts gefunden. ich habs auch überstzen lassen aber da kommt nur raus "alles bei"...das wirds ja wohl nicht sein^^ könnte mir also jemand sagen was es genau bedeutet wenn man das sagt?

The answer for us TV-challenged non-views:

Es heißt: "touché"

 

[Michael Torrie]

Alas, one reason it's a weak workaround is that it doesn't work - at least, not how I wish it would:


$ cat ptrs

x = 34

def p1 (a1):

a1[0] += 12

p1 ([x])

print (x)

$ python ptrs
34

you'll have to use it more like this (and also changing your names to be
a bit more sane):

x = [ 34, ]

def test_func( out ):
out[0] += 12

test_func(x)

print (x)

 

[Michael Torrie]

Thank you. You reminded me of the (weak) workaround of using arrays
and confirmed my suspicion that I although I can read the variable, I
won't be able to write to it. I still don't understand why not,
though...

The real problem here is that you don't understand how python variables
work. And in fact, python does not have variables. It has names that
bind to objects. If you assign a value to a name, you're not
overwriting a variable, you're rebinding a name to a new object in the
default scope (unless it's declared global, or nonlocal in python 3, the
latter I assume uses the same scope in which it was first declared, but
I could be wrong). Since the name is in the local scope, the original
name in the caller's scope is still bound to its original object.

Furthermore, most primitive objects in python (strings, ints, etc) are
immutable, which means they can *never change*. "Writing" to a variable
simply dereferences the old object (which may still be referenced in the
caller's scope) and creates a new object and binds it to the name.

A list is mutable, which is why it's one solution to emulate a variable.

As for python 3 ... "nonlocal"? I see I'm not alone in picking
obnoxious names ...

nonlocal at least has meaning.

 

[Mark Lawrence]

"PS: If you're reading this and love the French language -- I am deeply sorry
for the pain I'm causing you..."

It's obviously a team effort...

My French ain't so hot, either. I had to google your "tout chez" until I ran into the explanation:

hallo also ich gucke super gerne two and a half men und da wird öfters tout chez (keine ahnung ob es so geschrieben wird) gesagt. ich hab gegooglet aber nichts gefunden. ich habs auch überstzen lassen aber da kommt nur raus "alles bei"...das wirds ja wohl nicht sein^^ könnte mir also jemand sagen was es genau bedeutet wenn man das sagt?

The answer for us TV-challenged non-views:

Es heißt: "touché"

Try reading Stephen Clarke's "1000 Years of Annoying the French".
Perfect when summing up how good they are are at raising white flags
wherever and whenever it suits them.

--
"Steve is going for the pink ball - and for those of you who are
watching in black and white, the pink is next to the green." Snooker
commentator 'Whispering' Ted Lowe.

Mark Lawrence

 

[cts.private.yahoo]

I love the title. Reminds me of Ivanhoe ... great time travel.

 

[Antoon Pardon]

Op 29-06-13 16:02, Michael Torrie schreef:

The real problem here is that you don't understand how python variables
work. And in fact, python does not have variables. It has names that
bind to objects.

I don't understand why members of this list keep saying this. Sure the
variables in python behave differently than those in C and algol But
they behave similarly as those in smalltalk and lisp and I haven't seen
anyone claim that smalltalk and lisp don't have variables.

We might as well say that C doesn't have variables, it has names
pointing to memory locations or value containers or something
like that.

AFAICS there is no reason why "variable" wouldn't be appropiate
for python names as opposed to C names.

 

[rusi]

Op 29-06-13 16:02, Michael Torrie schreef:

I don't understand why members of this list keep saying this. Sure the
variables in python behave differently than those in C and algol But
they behave similarly as those in smalltalk and lisp and I haven't seen
anyone claim that smalltalk and lisp don't have variables.

We might as well say that C doesn't have variables, it has names
pointing to memory locations or value containers or something
like that.

AFAICS there is no reason why "variable" wouldn't be appropiate
for python names as opposed to C names.

Well mathematicians (or to be more precise functional programmers pretending to be mathematicians) claim that any imperative language does not have variables.

And recently on this list I saw the exact opposite claim -- functional languages dont have variables.

I also remember my statistics teacher dinning it into us -- a random variable is not a variable.

So each one varies according to his own notions I guess

 

[Michael Torrie]

Op 29-06-13 16:02, Michael Torrie schreef:

I don't understand why members of this list keep saying this. Sure the
variables in python behave differently than those in C and algol But
they behave similarly as those in smalltalk and lisp and I haven't seen
anyone claim that smalltalk and lisp don't have variables.

We might as well say that C doesn't have variables, it has names
pointing to memory locations or value containers or something
like that.

Sure but a memory location that contains say an int in C *is* a
variable, with or without a name. You can change the int stored in
that memory address at will, as part of your normal course. Python's
basic data types are immutable. At best we could say they are read-only
variables.

So no, saying Python doesn't have variables is not the same as saying C
doesn't have variables but only memory locations. They are different
concepts entirely, though on the surface they look similar.

AFAICS there is no reason why "variable" wouldn't be appropiate
for python names as opposed to C names.

Sure I see your point, but then again, calling them variables is what
led to the OP's issue in the first place. So yes they look like
variables, and for the most part act like them, except when they don't.
Hence the confusion and why I bring up the difference between python's
name binding mechanism and how a variable works. It's exactly the
concept that was tripping up the OP.

 

[Michael Torrie]

x = [ 34, ]

def test_func( out ):
out[0] += 12

test_func(x)

print (x)

Well, actually
print (x[0])

 

[cts.private.yahoo]

Thank you guys for saying what I was biting my tongue about (thanks everybody for the help, BTW!).

This "python-think" stuff was starting to get on my nerves - but then it occurred to me that - although having many powerful features - it has so many weird restrictions that it requires a special way of thinking and problem solving.

I have to work with perl's object-orientation stuff again for awhile, in order to see if either has an advantage.

 

[Steven D'Aprano]

Thank you. You reminded me of the (weak) workaround of using arrays

I think you mean lists, rather than arrays. Python does have an array
type, but it is much more restricted.

If you want an indirect reference to a value, the simplest ways are via a
object such as a list, dict or mutable object with named attributes.
That's not really a "weak workaround". In C you would dereference a
pointer, in Python you "dereference" a list:

ptr = &obj;
value = *ptr;

becomes:

ptr[0] = obj
value = ptr[0]


although don't push the analogy too far, Python lists aren't pointers in
the C sense.

What Python doesn't have -- and it doesn't seem to me that it could have,
without support from the interpreter, is a simple way to indirectly refer
to another *name*, rather than another object.

and
confirmed my suspicion that I although I can read the variable, I won't
be able to write to it. I still don't understand why not, though...

In Python 2, you simply can't because the interpreter doesn't support it.

As for python 3 ... "nonlocal"? I see I'm not alone in picking
obnoxious names ...

Huh?

You have "global" for global names. Python require declarations for local
names, but if it did it would probably use "local". What name would you
pick to declare names nonlocal other than "nonlocal"?

 

[Steven D'Aprano]

Op 29-06-13 16:02, Michael Torrie schreef:

I don't understand why members of this list keep saying this. Sure the
variables in python behave differently than those in C and algol But
they behave similarly as those in smalltalk and lisp and I haven't seen
anyone claim that smalltalk and lisp don't have variables.

We might as well say that C doesn't have variables, it has names
pointing to memory locations or value containers or something like that.

AFAICS there is no reason why "variable" wouldn't be appropiate for
python names as opposed to C names.

You are absolutely correct in principle. But in practice, there are ten
bazillion C, Pascal, COBOL, and BASIC programmers who understand the word
"variable" to mean a named memory location, for every Smalltalk or Lisp
programmer who understands a "variable" as a name binding. So it's pure
weight of numbers thing.

The average Lisp programmer will be completely aware that "variable" can
mean various things, and take care to determine what the word means in
Python. She will immediately grok what we mean, even if she thinks that
the "no variables" part is just an affectation ("Heh, those wacky Python
dudes think they don't have variables!") but at least she'll understand
the name binding part.

On the other hand, the average C programmer is barely aware that there
are other languages at all, let alone that some of them differ from C in
semantics as well as syntax. So by emphasising the differences ("Python
has no variables? It has name bindings?") we increase the likelihood that
he'll learn the differences in semantics as well as syntax.

So, in a very practical sense, "Python has no variables, it has name
bindings" is completely wrong except in the sense that really matters:
Python's variables don't behave identically to C variables.

 

[Michael Torrie]

Thank you guys for saying what I was biting my tongue about
(thanks everybody for the help, BTW!).

Sometimes it's best to state the actual problem you're trying to solve
and see if there's a pythonic solution that fits it rather than to hack
a solution transliterated from C. I'm curious as to if you did get
something working and what you ended up with.

This "python-think" stuff was starting to get on my nerves - but then
it occurred to me that - although having many powerful features - it
has so many weird restrictions that it requires a special way of
thinking and problem solving.

Interesting point of view. "Pythonic" ways of programming is in my mind
the number one appeal of Python. It's quite clean yet practical. Has
enough of the intellectual purity of LISP, Smalltalk, and functional
languages to be appealing, yet the practicality of a traditional
procedural language.

In any language, though you have to grasp the data model. Usually the
criticisms of Python come from not a failure to do this, either because
it's hard to learn at first, or because people dislike learning
something different than what they are used to. A while back we had a
fairly pleasant gentleman come on the list from a C# background. His
frustrations with Python stemmed from wanting it to be like C#, which of
course it isn't. He did not have much success and I'm afraid was left
with a sour taste of Python, which of course had nothing to do with the
language itself. Python certainly has inconsistencies and there are
newbie behavioral gotchas.

I have to work with perl's object-orientation stuff again for awhile,
in order to see if either has an advantage.

Your original post mentioned nothing about object-orientation, so I have
no idea how you intend to use OO design, but I think you'll find
Python's model fairly workable and consistent.

 

[Steven D'Aprano]

Thank you guys for saying what I was biting my tongue about (thanks
everybody for the help, BTW!).

This "python-think" stuff was starting to get on my nerves - but then it
occurred to me that - although having many powerful features - it has so
many weird restrictions that it requires a special way of thinking and
problem solving.

Er, no, you're confused. Python's restrictions aren't "weird", they make
absolutely perfect sense once you understand the programming model.
Python has a nice, clean semantic model, far cleaner than most other
languages I've looked at, which *cannot decide* on what model they wish
to present so they end up with a hodge-podge of bits of different models
all lumped together and every piece of random syntax you could ever hope
(or perhaps fear) to see.

If you insist in thinking of Python as "Perl with different syntax", then
you won't understand why you can't do certain things -- and you'll
equally not realise that you CAN do other things that other languages
don't support.