Need help with Python scoping rules

[kj]

The only thing one is entitled to expect when learning a new language is
that the language's implementation follows the language specs.

In fact, the official docs, when they discuss scopes, are off to
a bad start:

Names refer to objects. Names are introduced by name binding
operations. Each occurrence of a name in the program text refers
to the binding of that name established in the innermost function
block containing the use.

The first paragraph implies that binding can only occur within
functions, which is either incorrect, or presupposes a definition
of "function" that is not what most programmers would recognize.

In general, I found the docs very unclear on the subject of scoping.
PEP 227 is much better, but I wouldn't have thought of it as "the
specs".

kynn

 

[Jan Kaliszewski]

The class is a scope, and inside the class scope, you can access local
names. What you can't do is access the class scope from inside nested
functions.

s/from inside nested functions/from inside nested scopes

Besides that detail, I fully agree.

*j

 

[Miles Kaufmann]

I think I understand the answers well enough. What I *really*
don't understand is why this particular "feature" of Python (i.e.
that functions defined within a class statement are forbidden from
"seeing" other identifiers defined within the class statement) is
generally considered to be perfectly OK. IMO it's a bizarre,
inexplicable blindspot (which, among other things, gives rise to
a certain worry about what other similar craziness lurks under
Python's image of rationality). I have never seen even a half-hearted
justification, from a language design point of view, for why this
particular "feature" is worth having.

Guido's design justifications:
http://mail.python.org/pipermail/python-dev/2000-November/010598.html

--

My personal justification:

Python has used the same basic method of class creation since the very
beginning: create a new local namespace, execute the class suite in
that namespace, and then create a class, using the contents of the
namespace as the class attributes. The important thing to note here
is that there are really *two* namespaces--the local namespace that
exists while the class suite is being executed (what I call the "suite
namespace"), and the namespace of the class itself--and the first
ceases to exist when the second is created. The two namespaces
generally contain the same names at the point that the transfer
occurs, but they don't have to; the metaclass (which constructs the
class) is free to mess with the dictionary of attributes before
creating the class.

Suppose for a moment that the suite namespace *were* visible to nested
scopes. The simplest and most consistent implementation would be to
have a closure generated by a class statement be similar to that
generated by a function--i.e., the closure would be over the suite
namespace. This hardly seems desirable, though, because the suite
namespace and the class namespace would get out of sync when different
objects were assigned to the class namespace:

class C:
x = 1
def foo(self):
print x
print self.x

1
1

1
2

Surely such an implementation would be considered an even larger
Python wart then not having the suite namespace visible to nested
scopes at all. But it's better than the alternative of trying to
unify the class suite namespace and the class namespace, which would
be a nightmare of special cases (adding/deleting class attributes?
descriptors? __getattr__?) and require an implementation completely
separate from that of normal nested scopes.

-Miles

P.S. Just for fun:

import types

def make_class(*bases):
"""Decorator to allow you to (ab)use a function as a class definition.

The function must take no arguments and end with 'return locals()';
bases are (optionally) specified as arguments to make_class;
metaclasses other than 'type' are not supported.
.... def C():
.... greeting = 'Hello'
.... target = 'world'
.... def greet(self):
.... print '%s, %s' % (self.greeting, target)
.... return locals()
....Hello, world
"""

def decorator(func):
return type(func.func_name, bases, func())
if len(bases) == 1 and isinstance(bases[0], types.FunctionType):
func = bases[0]
bases = (object,)
return decorator(func)
if not bases:
bases = (object,)
return decorator

 

[Piet van Oostrum]

k> No, the fact() function here represents an internal "helper"
k> function. It is meant to be called only once to help initialize
k> a class variable that would be inconvenient to initialize otherwise;
k> this helper function is not meant to be called from outside the
k> class statement. Granted, in the example I gave, the "helper"
k> function (factorial) is a bit silly, but that was just intended as
k> a simple and familiar example of a recursive function. The actual
k> function that motivated this post would be considerably more
k> difficult to explain and would have obscured the point of the post.

Classes don't have helper functions; they have methods. Instance
methods, static methods or class methods. Your's isn't either of these.
Methods are to be called like `something.method(...)'.

 

[kj]

Guido's design justifications:
http://mail.python.org/pipermail/python-dev/2000-November/010598.html

Ah! Clarity! Thanks! How did you find this? Did you know of
this post already? Or is there some special way to search Guido's
"design justifications"?

...because the suite
namespace and the class namespace would get out of sync when different
objects were assigned to the class namespace:

class C:
x = 1
def foo(self):
print x
print self.x
1
2

But this unfortunate situation is already possible, because one
can already define

class C:
x = 1
def foo(self):
print C.x
print self.x

which would lead to exactly the same thing.

I need to learn more about metaclasses, though, to fully understand
your post.

Many thanks!

kynn

 

[Miles Kaufmann]

Ah! Clarity! Thanks! How did you find this? Did you know of
this post already? Or is there some special way to search Guido's
"design justifications"?

I just checked the python-dev archives around the time that PEP 227
was written.

But this unfortunate situation is already possible, because one
can already define

class C:
x = 1
def foo(self):
print C.x
print self.x

which would lead to exactly the same thing.

You're right, of course. If I had been thinking properly, I would
have posted this:

.... the suite namespace and the class namespace would get out of sync
when different objects were assigned to the class namespace:

# In a hypothetical Python with nested class suite scoping:
class C:
x = 1
@classmethod
def foo(cls):
print x
print cls.x
1
1 1
2

With your example, the result is at least easily explainable: self.x
is originally 1 because the object namespace inherits from the class
namespace, but running 'o.x = 2' rebinds 'x' in the object namespace
(without affecting the class namespace). It's a distinction that
sometimes trips up newbies (and me, apparently ), but it's
straightforward to comprehend once explained. But the distinction
between the class suite namespace and the class namespace is far more
subtle; extending the lifetime of the first so that it still exists
after the second is created is, IMO, asking for trouble (and trying to
unify the two double so).

-Miles

 

[Bruno Desthuilliers]

kj a écrit :

In fact, the official docs, when they discuss scopes, are off to
a bad start:

Names refer to objects. Names are introduced by name binding
operations. Each occurrence of a name in the program text refers
to the binding of that name established in the innermost function
block containing the use.

The first paragraph implies that binding can only occur within
functions, which is either incorrect, or presupposes a definition
of "function" that is not what most programmers would recognize.

Indeed, and you're right to point it. I strongly suggest you submit a
ticket to the doc maintainers.

In general, I found the docs very unclear on the subject of scoping.
PEP 227 is much better, but I wouldn't have thought of it as "the
specs".

Well, here again, it sure would help to make clear that peps (and
release specific "what's new") *are* part of the doc. But for sure, what
used to be a simple and clear reference is now a bit of a mess, despite
the hard work of the doc team. Side-effect of the rapid growth of the
language I guess...

 

[Ethan Furman]

But this unfortunate situation is already possible, because one
can already define

class C:
x = 1
def foo(self):
print C.x
print self.x

which would lead to exactly the same thing.

This is not the same thing, and definitely not exactly the same thing.
In your example you are explicitly stating whether you want the original
class variable, or the current, and possibly different, instance
variable. Further, the instance variable will not be different from the
class variable, even after C.x = whatever, unless the instance has had
that variable set specifically for it.

In [1]: class C(object):
...: x = 9
...: def doit(self):
...: print C.x
...: print self.x
...:

In [2]: test = C()

In [3]: test.doit()
9
9

In [4]: C.x = 10

In [5]: test.doit()
10
10

In [6]: test.x = 7

In [7]: test.doit()
10
7

~Ethan~

 

[kj]

This is not the same thing, and definitely not exactly the same thing.

Thanks for the clarification!

kynn