scope of function parameters

[Henry Olders]

I just spent a considerable amount of time and effort debugging a program. The made-up code snippet below illustrates the problem I encountered:

def main():
a = ['a list','with','three elements']
print a
print fnc1(a)
print a

def fnc1(b):
return fnc2(b)

def fnc2(c):
c[1] = 'having'
return c

This is the output:
['a list', 'with', 'three elements']
['a list', 'having', 'three elements']
['a list', 'having', 'three elements']

I had expected the third print statement to give the same output as the first, but variable a had been changed by changing variable c in fnc2.

It seems that in Python, a variable inside a function is global unless it's assigned. This rule has apparently been adopted in order to reduce clutter by not having to have global declarations all over the place.

I would have thought that a function parameter would automatically be considered local to the function. It doesn't make sense to me to pass a global to a function as a parameter.

One workaround is to call a function with a copy of the list, eg in fnc1 I would have the statement "return fnc2(b[:]". But this seems ugly.

Are there others who feel as I do that a function parameter should always be local to the function? Or am I missing something here?

Henry

 

[Mel]

I just spent a considerable amount of time and effort debugging a program.
The made-up code snippet below illustrates the problem I encountered:

def main():
a = ['a list','with','three elements']
print a
print fnc1(a)
print a

def fnc1(b):
return fnc2(b)

def fnc2(c):
c[1] = 'having'
return c

This is the output:
['a list', 'with', 'three elements']
['a list', 'having', 'three elements']
['a list', 'having', 'three elements']

I had expected the third print statement to give the same output as the
first, but variable a had been changed by changing variable c in fnc2.

It seems that in Python, a variable inside a function is global unless
it's assigned. This rule has apparently been adopted in order to reduce
clutter by not having to have global declarations all over the place.

I would have thought that a function parameter would automatically be
considered local to the function. It doesn't make sense to me to pass a
global to a function as a parameter.

It doesn't look like a question of local or global. fnc2 is passed a
container object and replaces item 1 in that container. You see the results
when fnc2 prints the object it knows as `c`, and you see again when main
prints the object it knows as `a`. Python doesn't pass parameters by
handing around copies that can be thought of as local or global. Python
passes parameters by binding objects to names in the callee's namespace. In
your program the list known as `a` in main is identically the same list as
the one known as `c` in fnc2, and what happens happens.

Mel.

 

[Terry Reedy]

I just spent a considerable amount of time and effort debugging a program.
The made-up code snippet below illustrates the problem I encountered:

def main():
a = ['a list','with','three elements']
print a
print fnc1(a)
print a

def fnc1(b):
return fnc2(b)

def fnc2(c):
c[1] = 'having'
return c

This is the output:
['a list', 'with', 'three elements']
['a list', 'having', 'three elements']
['a list', 'having', 'three elements']

I had expected the third print statement to give the same output as the
first, but variable a had been changed by changing variable c in fnc2.

It seems that in Python, a variable inside a function is global unless
it's assigned. This rule has apparently been adopted in order to reduce
clutter by not having to have global declarations all over the place.

I would have thought that a function parameter would automatically be
considered local to the function.

Function *parameters* are names, the first *local names* of the function.

You are right, in a way;-). Global *names* are just names. When you call
a function, you pass *objects* as *arguments*. Of course, you may refer
to the object by a global name to pass it, or you can pass a string
object that contains a global name.

It doesn't look like a question of local or global. fnc2 is passed a
container object and replaces item 1 in that container. You see the results
when fnc2 prints the object it knows as `c`, and you see again when main
prints the object it knows as `a`. Python doesn't pass parameters by
handing around copies that can be thought of as local or global. Python
passes parameters by binding objects to names in the callee's namespace. In
your program the list known as `a` in main is identically the same list as
the one known as `c` in fnc2, and what happens happens.

Right. Python has one unnamed 'objectspace'. It has many, many
namespaces: builtins, globals for each module, locals for each function
and class, and attributes for some instances. Each name and each
collection slot is associated with one object. Each object can have
multiple associations, as in the example above.

 

[Steven D'Aprano]

I just spent a considerable amount of time and effort debugging a
program. The made-up code snippet below illustrates the problem I
encountered: [...]
Are there others who feel as I do that a function parameter should
always be local to the function? Or am I missing something here?

The nature of Henry's misunderstanding is a disguised version of the very
common "is Python call by reference or call by value?" question that
periodically crops up. I wrote a long, but I hope useful, explanation for
the (e-mail address removed) mailing list, which I'd like to share here:

http://mail.python.org/pipermail/tutor/2010-December/080505.html


Constructive criticism welcome.

 

[Chris Angelico]

 <URL:http://www.computerworld.com/s/article/print/93903/I_m_OK_The_Bull_Is_Dead>

I agree with the gist of that. My take on this is: When I'm talking to
my boss, I always assume that the phone will ring ten seconds into my
explanation. Ten seconds is enough for "Dad, I'm OK; the bull is
dead", it's enough for "I've solved Problem X, we can move on now";
it's enough for "Foo is crashing, can't ship till I debug it". If
fortune is smiling on me and the phone isn't ringing, I can explain
that Problem X was the reason Joe was unable to demo his new module,
or that the crash in Foo is something that I know I'll be able to pin
down in a one-day debugging session, but even if I don't, my boss
knows enough to keep going with.

Of course, there's a significant difference between a mailing list
post and a detailed and well copyedited article. Quite frequently I'll
ramble on list, in a way quite inappropriate to a publication that
would be linked to as a "hey guys, here's how it is" page. Different
media, different standards.

Chris Angelico
"Forty thousand million billion THEGS quotes? That must be worth a fortune!"
-- definitely a fan of THEGS --

 

[Chris Angelico]

Right. But Steven specifically asked for constructive criticism, which I
took as permission to treat the referenced post as an article in need of
copy editing

Indeed. Was just saying that there are times when you need to get the
slug out first, and times when it's okay to be a little less impactual
(if that's a word). Although it's still important to deliver your
message promptly.

Of course, there are other contexts where you specifically do NOT want
to give everything away at the beginning. Certain styles of rhetoric
demand that you set the scene, build your situation, and only at the
climax reveal what it is you are trying to say.

Ahh! wordsmithing, how we love thee.

Chris Angelico

 

[Steven D'Aprano]

Informative, but it “buries the lead†as our friends in the press corps
would say.

Thank you, that's a good point.


[...]

More on this style:

<URL:http://www.computerworld.com/s/article/print/93903/

I_m_OK_The_Bull_Is_Dead>


Or as they say in the fiction-writing trade, "shoot the sheriff on the
first page".

 

[Laurent Claessens]

Le 29/05/2011 23:42, Ben Finney a écrit :

Or no names. So it's less accurate (though better than talking of
“variablesâ€) to speak of Python objects “having namesâ€.

Could you give an example of an object that has no name ? I've missed
something ...

Laurent

 

[Chris Rebert]

Le 29/05/2011 23:42, Ben Finney a écrit :

Could you give an example of an object that has no name ? I've missed
something ...

def foo():
return 5

print(foo())

The int object 5 has no name here.

Cheers,
Chris

 

[Laurent]

def foo():
return 5

print(foo())

The int object 5 has no name here.

Cool. I was thinking that "5" was the name, butFile "<stdin>", line 1
5.__add__(6)
^
SyntaxError: invalid syntax

while
11


Very well. I learned something today.
Thanks
Laurent

 

[Daniel Kluev]

Could you give an example of an object that has no name ? I've missed
something ...

<object object at 0xb73d04d8>

 

[Terry Reedy]

Cool. I was thinking that "5" was the name, but
File "<stdin>", line 1
5.__add__(6)

Try 5 .__add__(6)

Modules, classes, and functions have a .__name__ attribute (I call it
their 'definition name') used to print a representation. As best I can
remember, other builtin objects do not.

 

[Laurent Claessens]

Le 30/05/2011 11:02, Terry Reedy a écrit :

Try 5 .__add__(6)

What is the rationale behind the fact to add a space between "5" and
".__add__" ?
Why does it work ?

Laurent

 

[Steven D'Aprano]

Le 30/05/2011 11:02, Terry Reedy a écrit :

What is the rationale behind the fact to add a space between "5" and
".__add__" ?
Why does it work ?

Because . is an operator just like + * & etc.
'HELLO WORLD'


In the case of integer literals, you need the space, otherwise Python
will parse 5. as a float:

File "<stdin>", line 1
5.__add__
^
SyntaxError: invalid syntax<method-wrapper '__add__' of int object at 0x8ce3d60>

 

[Steven D'Aprano]

Could you give an example of an object that has no name ? I've missed
something ...

mylist = [None, 42, "something"]

The list object has a name, mylist.

The three objects inside the list have no names.

 

[Peter Otten]

Le 30/05/2011 11:02, Terry Reedy a écrit :

What is the rationale behind the fact to add a space between "5" and
".__add__" ?
Why does it work ?

It's a hint for the tokenizer.

$ cat show_tokens.py
import sys

from tokenize import generate_tokens
from cStringIO import StringIO
from token import tok_name

_name_width = max(len(name) for name in tok_name.itervalues())

def show_tokens(s):
for token in generate_tokens(StringIO(s).readline):
name = tok_name[token[0]]
value = token[1]
print "%-*s %r" % (_name_width, name, value)

if __name__ == "__main__":
show_tokens(" ".join(sys.argv[1:]))

$ python show_tokens.py 5.__add__
NUMBER '5.'
NAME '__add__'
ENDMARKER ''

$ python show_tokens.py 5 .__add__
NUMBER '5'
OP '.'
NAME '__add__'
ENDMARKER ''

 

[Laurent Claessens]

It's a hint for the tokenizer.

I didn't know the tokenizer. Now I understand.
Thanks

Laurent

 

[Laurent Claessens]

It's a hint for the tokenizer.

I didn't know the tokenizer. Now I understand.
Thanks

Laurent

 

[Jussi Piitulainen]

Le 30/05/2011 11:02, Terry Reedy a écrit :

What is the rationale behind the fact to add a space between "5" and
".__add__" ?
Why does it work ?

Oh joy.

[5][0].__add__([6][-1])

11

The parser just needs the help to detect the intended token boundary
instead of another, unintened one. As the others already say.

 

[Terry Reedy]

Le 30/05/2011 11:02, Terry Reedy a écrit :

What is the rationale behind the fact to add a space between "5" and
".__add__" ?
Why does it work ?

Others have given you specific answers, here is the bigger picture.

For decades, text interpreter/compilers have generally run in two phases:
1. a lexer/tokenizer that breaks the stream of characters into tokens;
2. a parser that recognizes higher-level syntax and takes appropriate
action.

Lexers are typically based on regular grammars and implemented as very
simple and fast deterministic finite-state automata. In outline (leaving
out error handling and end-of-stream handling), something like:

def lexer(stream, lookup, initial_state):
state = initial_state
buffer = []
for char in stream:
state,out = lookup[state,char]
if out:
yield output(buffer)
# convert list of chars to token expected by parser, clear buffer
buffer += char

There is no backup and no lookahead (except for the fact that output
excludes the current char). For python, lookup[start,'5'] ==
in_number,False, and lookup[in_number,'.'] == in_float,False.
11.0

works because lookup[in_float,'.'] == start,True, because buffer now
contains a completed float ready to output and '.' signals the start of
a new token.

I believe we read natural language text similarly, breaking it into
words and punctuation. I believe the ability to read programs depends on
being able to adjust the internal lexer a bit. Python is easier to read
than some other algorithm languages because it tends to have at most one
punctuation-like symbol unit between words, as is the case in the code
above.