Scope, type and UnboundLocalError

[Paddy]

Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.

The documentation: http://docs.python.org/ref/naming.html does not give
me enough info to determine why the difference exists as it does not
seem to mention types at all..

The code:

===== scope_and_type.py =======
m = 0
n = [0]

def int_access0():
m = m + 1
return m
def int_access1():
m += 1
return m
def list_access0():
n[0] = n[0] + 1
return n
def list_access1():
n[0] += 1
return n

try:
print "\nint_access0:", int_access0()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nint_access1:", int_access1()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nlist_access0:", list_access0()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nlist_access1:", list_access1()
except UnboundLocalError, inst:
print " ERROR:\n", inst


print "\n (m,n) = ", (m,n)


p = (0,)
def tuple_access():
return p[0]
try:
print "\ntuple_acces:", tuple_access()
except UnboundLocalError, inst:
print " ERROR:\n", inst
print "\n p = ", p

===== END scope_and_type.py =======

The output:int_access0: ERROR:
local variable 'm' referenced before assignment

int_access1: ERROR:
local variable 'm' referenced before assignment

list_access0: [1]

list_access1: [2]

(m,n) = (0, [2])

tuple_acces: 0

p = (0,)

 

[Frank Millman]

Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.

There has just been a long thread about this. I think I understand it
now. Here is my explanation.

Ignoring nested scopes for this exercise, references to objects (i.e.
variable names) can exist in the local namespace or the global
namespace. Python looks in the local namespace first, and if not found
looks in the global namespace.

Any name assigned to within the function is automatically deemed to
exist in the local namespace, unless overridden with the global
statement.

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.

With the statement 'n[0] = n[0] + 1', n is not being assigned to, as it
is mutable. Therefore Python looks in the global namespace, finds n
there, and uses it successfully.

My 2c

Frank Millman

 

[Paddy]

Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.

With the statement 'n[0] = n[0] + 1', n is not being assigned to, as it
is mutable. Therefore Python looks in the global namespace, finds n
there, and uses it successfully.

My 2c

Frank Millman

So, to paraphrase to test my understanding:

in the statement: ' n[0] = n[0] + 1' it is the object referenced by the
name n that is being assigned to rather than n itself, so n is not
'tagged' as a local variable by the LHS of the assignment.

Thanks Frank. all is is now clear

 

[Bruno Desthuilliers]

Frank Millman a écrit :

There has just been a long thread about this. I think I understand it
now. Here is my explanation.

Ignoring nested scopes for this exercise, references to objects (i.e.
variable names) can exist in the local namespace or the global
namespace. Python looks in the local namespace first, and if not found
looks in the global namespace.

Any name assigned to within the function is automatically deemed to
exist in the local namespace, unless overridden with the global
statement.

And this even of the local bindings sequentially comes after another
access to the name, ie:

g = 0

def fun():
x = g # UnboundLocalError here
g += 1
return x

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.
Right

With the statement 'n[0] = n[0] + 1', n is not being assigned to, Right

as it
is mutable.

n is effectively mutable, but this is totally irrelevant. In your
snippet, n is not 'assigned to', it's "mutated" (ie a state-modifying
method is called). The snippet:

n[0] = n[0] + 1

is syntactic sugar for

n.__setitem__(0, n.__getitem__(0) + 1)

IOW, it's just method calls on n.

 

[Bruno Desthuilliers]

Paddy a écrit :

Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.

With the statement 'n[0] = n[0] + 1', n is not being assigned to, as it
is mutable. Therefore Python looks in the global namespace, finds n
there, and uses it successfully.

My 2c

Frank Millman

So, to paraphrase to test my understanding:

in the statement: ' n[0] = n[0] + 1' it is the object referenced by the
name n that is being assigned to rather than n itself, so n is not
'tagged' as a local variable by the LHS of the assignment.

Nope. You got it plain wrong - cf my answer to Frank in this thread.

Thanks Frank. all is is now clear

It is obviously not.

 

[Paddy]

Frank Millman a écrit :

There has just been a long thread about this. I think I understand it
now. Here is my explanation.

Ignoring nested scopes for this exercise, references to objects (i.e.
variable names) can exist in the local namespace or the global
namespace. Python looks in the local namespace first, and if not found
looks in the global namespace.

Any name assigned to within the function is automatically deemed to
exist in the local namespace, unless overridden with the global
statement.

And this even of the local bindings sequentially comes after another
access to the name, ie:

g = 0

def fun():
x = g # UnboundLocalError here
g += 1
return x

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.
Right

With the statement 'n[0] = n[0] + 1', n is not being assigned to, Right

as it
is mutable.

n is effectively mutable, but this is totally irrelevant. In your
snippet, n is not 'assigned to', it's "mutated" (ie a state-modifying
method is called). The snippet:

n[0] = n[0] + 1

is syntactic sugar for

n.__setitem__(0, n.__getitem__(0) + 1)

IOW, it's just method calls on n.

So,
An assignment statement may assign an object to a name, in which case
the name is 'tagged' as being local,
An assignment statement may mutate a mutable object already bound to a
name, in which case the assignment will not 'tag' the name as being
local.

I guess Bruno, you mean irrelevant as in 'although only mutable objects
can have their state modified; if n has a mutable value but the
assignment statement changed the object referred to by n, then the name
would be tagged as local'?

- Peace, Paddy.

 

[Paddy]

So,
An assignment statement may assign an object to a name, in which case
the name is 'tagged' as being local,
An assignment statement may mutate a mutable object already bound to a
name, in which case the assignment will not 'tag' the name as being
local.

I guess Bruno, you mean irrelevant as in 'although only mutable objects
can have their state modified; if n has a mutable value but the
assignment statement changed the object referred to by n, then the name
would be tagged as local'?

- Peace, Paddy.

No, that last paragraph still does not convey what I meant.


.... irrelevant as in 'although only mutable objects can have their
state modified; if n has a mutable value but the assignment statement
changed n to refer to another object, then the name would be tagged as
local'?

Oh bosh, can anyone come at it from a different tack?

Ta.

 

[Fredrik Lundh]

... irrelevant as in 'although only mutable objects can have their
state modified; if n has a mutable value but the assignment statement
changed n to refer to another object, then the name would be tagged as
local'?

Oh bosh, can anyone come at it from a different tack?

look for "Assignment of an object to a single target is recursively
defined as follows" on this page (or at the corresponding page in the
language reference):

http://pyref.infogami.com/assignments

</F>

 

[Bruno Desthuilliers]

Paddy a écrit :

Frank Millman a écrit :

Paddy wrote:


Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.



There has just been a long thread about this. I think I understand it
now. Here is my explanation.

Ignoring nested scopes for this exercise, references to objects (i.e.
variable names) can exist in the local namespace or the global
namespace. Python looks in the local namespace first, and if not found
looks in the global namespace.

Any name assigned to within the function is automatically deemed to
exist in the local namespace, unless overridden with the global
statement.

And this even of the local bindings sequentially comes after another
access to the name, ie:

g = 0

def fun():
x = g # UnboundLocalError here
g += 1
return x

With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.
Right


With the statement 'n[0] = n[0] + 1', n is not being assigned to,
Right


as it
is mutable.

n is effectively mutable, but this is totally irrelevant. In your
snippet, n is not 'assigned to', it's "mutated" (ie a state-modifying
method is called). The snippet:

n[0] = n[0] + 1

is syntactic sugar for

n.__setitem__(0, n.__getitem__(0) + 1)

IOW, it's just method calls on n.

So,
An assignment statement may assign an object to a name,

An assignment statement binds an object to a name.

in which case
the name is 'tagged' as being local,

Unless that names has been declared as global or lives in another
namespace (ie is an element of a mutable collection or an attribute of
another object).

An assignment statement may mutate a mutable object already bound to a
name, in which case the assignment will not 'tag' the name as being
local.

consider this code:

class Foo(object):
def __init__(self, baaz):
self.baaz = baaz

def bar(foo, bak):
foo.baaz = bak

Which name is getting rebound here ? foo, or foo.baaz ?

I guess Bruno, you mean irrelevant as in 'although only mutable objects
can have their state modified; if n has a mutable value but the
assignment statement changed the object referred to by n, then the name
would be tagged as local'?

Unless the name has been declared as global, yes.

 

[Dennis Lee Bieber]

So,
An assignment statement may assign an object to a name, in which case
the name is 'tagged' as being local,

Reverse... Python does not "assign" objects to names... It "assigns"
names to objects. One object can have multiple names.

In the absence of a "global <name>" statement, any unqualified
<name> found on the left side of an "=" is a local name (the name -- not
the object it is bound to -- is held as part of the current stack frame
and is removed on return from the function; the object may or may not be
garbage collected depending upon any other names bound to it).

A qualified name is one with some sort of component specifier:
<name>.<component>, <name>[<component>]. These access items that are
inside the object that <name> is bound on. The component access
basically is a function/method call telling the object itself to change
the <component> binding, not the top-level <name> binding.

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Paddy]

Paddy a écrit :

Frank Millman a écrit :

Paddy wrote:


Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.



There has just been a long thread about this. I think I understand it
now. Here is my explanation.

Ignoring nested scopes for this exercise, references to objects (i.e.
variable names) can exist in the local namespace or the global
namespace. Python looks in the local namespace first, and if not found
looks in the global namespace.

Any name assigned to within the function is automatically deemed to
exist in the local namespace, unless overridden with the global
statement.

And this even of the local bindings sequentially comes after another
access to the name, ie:

g = 0

def fun():
x = g # UnboundLocalError here
g += 1
return x


With the statement 'm = m + 1', as m is assigned to on the LHS, it is
deemed to be local, but as m does not yet have a value on the RHS, you
get Unbound Local Error.

Right


With the statement 'n[0] = n[0] + 1', n is not being assigned to,

Right


as it
is mutable.

n is effectively mutable, but this is totally irrelevant. In your
snippet, n is not 'assigned to', it's "mutated" (ie a state-modifying
method is called). The snippet:

n[0] = n[0] + 1

is syntactic sugar for

n.__setitem__(0, n.__getitem__(0) + 1)

IOW, it's just method calls on n.

So,
An assignment statement may assign an object to a name,

An assignment statement binds an object to a name.

in which case
the name is 'tagged' as being local,

Unless that names has been declared as global or lives in another
namespace (ie is an element of a mutable collection or an attribute of
another object).

An assignment statement may mutate a mutable object already bound to a
name, in which case the assignment will not 'tag' the name as being
local.

consider this code:

class Foo(object):
def __init__(self, baaz):
self.baaz = baaz

def bar(foo, bak):
foo.baaz = bak

Which name is getting rebound here ? foo, or foo.baaz ?

I guess Bruno, you mean irrelevant as in 'although only mutable objects
can have their state modified; if n has a mutable value but the
assignment statement changed the object referred to by n, then the name
would be tagged as local'?

Unless the name has been declared as global, yes.

Thanks Bruno, that clears it up for me (I had purposefully left out the
global statement from my example, but should have included it in the
explanation).

 

[Paddy]

So,
An assignment statement may assign an object to a name, in which case
the name is 'tagged' as being local,

Reverse... Python does not "assign" objects to names... It "assigns"
names to objects. One object can have multiple names.

In the absence of a "global <name>" statement, any unqualified
<name> found on the left side of an "=" is a local name (the name -- not
the object it is bound to -- is held as part of the current stack frame
and is removed on return from the function; the object may or may not be
garbage collected depending upon any other names bound to it).

A qualified name is one with some sort of component specifier:
<name>.<component>, <name>[<component>]. These access items that are
inside the object that <name> is bound on. The component access
basically is a function/method call telling the object itself to change
the <component> binding, not the top-level <name> binding.

Hi Dennis, in the last paragraph you do not state specifically where
the name for the component name is looked for. Do you mean that for
component name accesses,where the 'base' is not declared gobal, the
'base' name nevertheless is always looked for in the global scope?

(Please excuse my pedantry).

 

[Paddy]

look for "Assignment of an object to a single target is recursively
defined as follows" on this page (or at the corresponding page in the
language reference):

http://pyref.infogami.com/assignments

</F>

I had a look Frederick but the page lead to:
http://pyref.infogami.com/subscriptions
Which again failed to mention namespace issues.

- Pad.

 

[Bruno Desthuilliers]

Paddy a écrit :

So,
An assignment statement may assign an object to a name, in which case
the name is 'tagged' as being local,

Reverse... Python does not "assign" objects to names... It "assigns"
names to objects. One object can have multiple names.

In the absence of a "global <name>" statement, any unqualified
<name> found on the left side of an "=" is a local name (the name -- not
the object it is bound to -- is held as part of the current stack frame
and is removed on return from the function; the object may or may not be
garbage collected depending upon any other names bound to it).

A qualified name is one with some sort of component specifier:
<name>.<component>, <name>[<component>]. These access items that are
inside the object that <name> is bound on. The component access
basically is a function/method call telling the object itself to change
the <component> binding, not the top-level <name> binding.

Hi Dennis, in the last paragraph you do not state specifically where
the name for the component name is looked for.

First in the local namespace, then in enclosing namespaces until the
global (read : module level) namespace, and finally in the builtins.

Do you mean that for
component name accesses,where the 'base' is not declared gobal, the
'base' name nevertheless is always looked for in the global scope?

unless it's found in another namespace before - same rules apply as for
a non-qualified name.

 

[Paddy]

Bruno Desthuilliers wrote:
<SNIP>

Ta.

 

[Dennis Lee Bieber]

Hi Dennis, in the last paragraph you do not state specifically where
the name for the component name is looked for. Do you mean that for
component name accesses,where the 'base' is not declared gobal, the
'base' name nevertheless is always looked for in the global scope?

No, that name follows the standard "local to global" search -- that
is, if a binding is made to an object locally, then component access
will be against that local binding.

c = newObject()
d = newObject()

def f():
c = newObject() #local binding
c.<component> = <something> #local look-up
#does not affect "global_c" as
# "c" -> "local_c"
c = d #another local binding, but to a global object
c.<component> = <something> #local name, but changes
#global object because "c" -> "global_d"

def g():
c.<component> = <something> #global look-up

def h():
global c
c = newObject() #new global binding
f() #no effect on global object
g() #effects global object from two lines up



This is why my feelings are that the search scheme is always
local->global for all names. But the assignment operator determines if a
binding is allowed based on local or declared global usage. The name,
not the object, is what is allocated locally when found unqualified on
the left side. This would happen during the "compile" pass; then during
the run pass it might find a right hand usage of the name before
anything has been bound to it.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Fredrik Lundh]

I had a look Frederick but the page lead to:
http://pyref.infogami.com/subscriptions
Which again failed to mention namespace issues.

it says

The primary must evaluate to an object of a sequence or mapping type.

and links to a page that explains what a primary is. the interesting page is the
one about "atoms" (which includes identifiers), which point you to

http://pyref.infogami.com/naming-and-binding

which explains it all.

</F>

 

[Paddy]

Hi,
I am trying to work out why I get UnboundLocalError when accessing an
int from a function where the int is at the global scope, without
explicitly declaring it as global but not when accessing a list in
similar circumstances.

The documentation: http://docs.python.org/ref/naming.html does not give
me enough info to determine why the difference exists as it does not
seem to mention types at all..

The code:

===== scope_and_type.py =======
m = 0
n = [0]

def int_access0():
m = m + 1
return m
def int_access1():
m += 1
return m
def list_access0():
n[0] = n[0] + 1
return n
def list_access1():
n[0] += 1
return n

try:
print "\nint_access0:", int_access0()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nint_access1:", int_access1()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nlist_access0:", list_access0()
except UnboundLocalError, inst:
print " ERROR:\n", inst
try:
print "\nlist_access1:", list_access1()
except UnboundLocalError, inst:
print " ERROR:\n", inst


print "\n (m,n) = ", (m,n)


p = (0,)
def tuple_access():
return p[0]
try:
print "\ntuple_acces:", tuple_access()
except UnboundLocalError, inst:
print " ERROR:\n", inst
print "\n p = ", p

===== END scope_and_type.py =======

The output:int_access0: ERROR:
local variable 'm' referenced before assignment

int_access1: ERROR:
local variable 'm' referenced before assignment

list_access0: [1]

list_access1: [2]

(m,n) = (0, [2])

tuple_acces: 0

p = (0,)

I blogged the following as a summary:
(From:
http://paddy3118.blogspot.com/2006/07/python-functions-assignments-and-scope.html)

Python Functions: Assignments And Scope

Explaining why this works:

n = [0]
def list_access():
n[0] = n[0] + 1
return n

try:
print "\nlist_access:", list_access()
except UnboundLocalError, inst:
print " ERROR:\n", inst

And this throws the exception:

m = 0
def int_access():
m = m + 1
return m

try:
print "\nint_access:", int_access()
except UnboundLocalError, inst:
print " ERROR:\n", inst

To execute a source program, the Python compiler compiles your
original source into 'byte codes' - a form of your program that
is easier for the Python interpreter to later run. In generating this
byte code, the byte code compiler will determine which variable names
in a function are local to that function, (so alowing it to optimise
accesses to such local names).

The rule for determining if a variable is local to a function is:

* If there is a global statement for the name in the function
then the name is accessed from the global scope.
* If there is no global statement for the name, and if there
are assignments to the 'bare' name within the function then the
name
is of local scope.
( A bare name assignment means assignment to a
name, where the name occurs without attribute references,
subscripts, or slicing s, just the bare name).
* Otherwise the name will be looked up in reverse order of all
enclosing scopes, until it is found.

In the second example, function int_access; name m is flagged as
local by the byte code compiler as the bare name is being assigned
to. The interpreter therefore looks for a value of m to increment
only in the local scope, cannot find a value, then raises the
UnboundLocalError exception.
In function list_access, the bare
name n is not assigned to, so n is found when looking back
through enclosing scopes.
References

1.


http://groups.google.com/group/comp.lang.python/browse_frm/thread/db9955da70c4e0ca
2.

http://pyref.infogami.com/assignments
3.

http://pyref.infogami.com/naming-and-binding
4.

http://www.python.org/doc/2.4/ref/global.html

END.