Class Variable Access and Assignment

[Steven D'Aprano]

One other way, to implement the += and likewise operators would be
something like the following.

Assume a getnsattr, which would work like getattr, but would also
return the namespace where the name was found. The implementation
of b.a += 2 could then be something like:

ns, t = getnsattr(b, 'a')
t = t + 2
setattr(ns, 'a')


I'm not arguing that this is how it should be implemented. Just
showing the implication doesn't follow.

Follow the logical implications of this proposed behaviour.

class Game:
current_level = 1
# by default, games start at level one

def advance(self):
self.current_level += 1


py> antoon_game = Game()
py> steve_game = Game()
py> steve_game.advance()
py> steve_game.advance()
py> print steve_game.level
3
py> print antoon_game.level

What will it print?

Hint: your scheme means that class attributes mask instance attributes.

 

[Steven D'Aprano]

Please explain why this is illegal.

x = 1
def f():
x += 1

Because names in function namespaces don't have inheritance.

 

[Steven D'Aprano]

That is an implemantation detail. The only answer that you are given
means nothing more than: because it is implemented that way.

You keep saying "that's an implementation detail" and dismissing the
question, but that's the heart of the issue. What does b.a += 2 *mean*? It
doesn't mean "sort the list referenced by b.a" -- we agree on that much.
You seem to think that it means "increment the object currently named
b.a by two". But that's not what it means.

b.a += 2 has a precise meaning, and for ints and many other objects that
meaning is the same as b.a = b.a + 2. Yes, it is an implementation detail.
So what? It is an implementation detail that "b.a += 2" doesn't mean "sort
the list referenced by b.a" too.

In some other language, that's precisely what it could mean -- but Python
is not that language.

b.a has a precise meaning too, and again you have got it wrong. It doesn't
mean "search b's namespace for attribute a". It means "search b's
namespace for attribute a, if not found search b's class' namespace, and
if still not found, search b's class' superclasses". It is analogous to
nested scopes. In fact, it is a type of nested scope.

In some other language, b.a could mean what you think it means, but Python
is not that language. That's a deliberate design decision. Nested
attribute search gives the most useful results in the most common cases,
while still being easy to work around in the rare cases where it is not
what is wanted.

It has nothing to do with a model for inheritance, but with a model of
name resolution.

Which is designed to act the way it does in order to produce the
inheritance model. You can't have that inheritance model without that name
resolution.

The hierarchie of searching an instance first in an object and then in
a class isn't that different from searching first in a local namespace
and then in a more global namespace.

When we search names in a function we don't resolve the same name in
different name spacese each occurence of the same name in the same
function occurs in the same namespace.

That's because it isn't needed for function namespaces. Names in a
function don't inherit state or behaviour from names in a higher-level
scope. Attribute names in classes do.

But with class variables we can have that one and the same name
on a line refers to two different namespaces at the same time.
That is IMO madness. You may argue that the madness is of little
importance, you can argue that because of the current implementation
little can be done about it. But I don't see how one can defend
it as sane behaviour.

Because inheritance is useful, sensible, rational behaviour for OO
programming.

 

[Steven D'Aprano]

[snip]

Look at that: the object which is referred to depends on how many times
you've already been through the loop. How nuts is that?

It is each time the 'x' from the same name space. In the code above the
'a' is not each time from the same namespace.

I also think you new very well what I meant.

I'm supposed to be a mindreader now? After you've spent multiple posts
ranting that, quote, "I don't think it is sane that which object is
refered to depends on how many times you already went through the loop",
I'm supposed to magically read your mind and know that you don't actually
object to what you say you object to, but to something completely
different?

 

[Steven D'Aprano]

Can you name one? Any code that relies on it seems extremely dangerous to me.

Dangerous? In what way?


A basic usage case:

class Paper:
size = A4
def __init__(self, contents):
# it makes no sense to have class contents,
# so contents go straight into the instance
self.contents = contents


To internationalise it for the US market:

Paper.size = USLetter

Now create a document using the default paper size:

mydocument = Paper("Four score and seven years ago")
print mydocument.size == USLetter
=> True

Now create a document using another paper size:

page = Paper("Eleventy MILLION dollars")
page.size = Foolscap

Because that's an instance attribute, our default doesn't change:

assert Paper().size == mydocument.size == USLetter
assert page.size != mydocument.size

In case it wasn't obvious, this is the same inheritance behaviour Python
objects exhibit for methods, except that it isn't normal practice to add
methods to instances dynamically. (It is more common to create a
subclass.) But you can do it if you wish, at least for classes you create
yourself.

Objects in Python inherit behaviour from their class.
Objects in Python inherit state from their class, unless their state is
specifically stored in a per-instance basis.


Here's another usage case:

class PrintableWidget(Widget):
prefix = "START "
suffix = " STOP"

def __str__(self):
return self.prefix + Widget.__str__(self) + self.suffix

PrintableWidgets now print with a default prefix and suffix, which can be
easily changed on a per-instance basis without having to create
sub-classes for every conceivable modification:

english_gadget = PrintableWidget("data")
print english_gadget
=> prints "START data STOP"

dutch_gadget = PrintableWidget("data")
dutch_gadget.prefix = "BEGIN "
dutch_gadget.suffix = " EINDE"
print dutch_gadget
=> prints "BEGIN data EINDE"




I have to ask... did OO programming suddenly fall out of favour when my
back was turned? People still use C++, C#, Objective-C and Java, right?
Why are so many folks on this list having trouble with inheritance? Have I
missed something?

 

[Bengt Richter]

Then couldn't we expect that the namespace resolution is also done
only once?

I say that if the introduction on += like operators implied that the
same mentioning of a name would in some circumstances be resolved to
two different namespaces, then such an introduction would better have
not occured.

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

I think I would rather seek consistency in terms of
order of evaluation and action. IOW, the right hand side
of an assignment is always evaluated before the left hand side,
and operator precedence and syntax defines order of access to names
in their expression context on either side.

The compilation of function bodies violates the above, even allowing
future (execution-wise) statements to influence the interpretation
of prior statements. This simplifies defining the local variable set,
and allows e.g. yield to change the whole function semantics, but
the practicality/purity ratio makes me uncomfortable ;-)

If there were bare-name properties, one could control the meaning
of x = x + 1 and x += 1, though of course one would need some way
to bind/unbind the property objects themselves to make them visible
as x or whatever names.

It might be interesting to have a means to push and pop objects
onto/off-of a name-space-shadowing stack (__nsstack__), such that the first place
to look up a bare name would be as an attribute of the top stack object, i.e.,

name = name + 1

if preceded by

__nsstack__.append(my_namespace_object)

would effectively mean

my_namespace_object.name = my_namespace_object.name + 1

by way of logic like

if __nsstack__:
setattr(__nsstack__[-1], getattr(__nstack__[-1], name) + 1))
else:
x = x + 1


Of course, my_namespace_object could be an instance of a class
that defined whatever properties or descriptors you wanted.
When you were done with that namespace, you'd just __nsstack__.pop()

If __nsstack__ is empty, then of course bare names would be looked
up as now.

BTW, __nsstack__ is not a literal proposal, just a way to illustrate the concept ;-)
OTOH, is suppose a function could have a reseved slot for a name space object stack
that wouldn't cost much run time to bypass with a machine language check for NULL.

BTW2, this kind of stack might play well with a future "with," to guarantee name
space popping. Perhaps "with" syntax could even be extended to make typical usage
slick ;-)

Regards,
Bengt Richter

 

[Christopher Subich]

It might be interesting to have a means to push and pop objects
onto/off-of a name-space-shadowing stack (__nsstack__), such that the first place
to look up a bare name would be as an attribute of the top stack object, i.e.,

name = name + 1

Don't be that specific; just unify Attributes and Names.

Instead of the 'name' X referring to locals()['X'] or globals()['X'],
have a hidden "namespace" object/"class", with lookups functioning akin
to class inheritence.

This would allow, in theory, more uniform namespace behaviour with outer
scoping:

x = 1
def f():
x += 1 # would work, as it becomes
setattr(namespace,'x',getattr(namespace,'x')+1), just like attribute loookup

Also, with a new keyword "outer", more rational closures would work:

def makeincr(start=0):
i = start
def inc():
outer i
j = i
i += 1
return j
return inc

From a "namespace object" point of view, 'outer i' would declare i to
be a descriptor on the namespace object, such that setting actions would
set the variable in the inherited scope (getting actions wouldn't
actually need modification, since it already falls-through). At the
first level, 'outer' would be exactly the same as 'global' -- indeed, it
would be reasonable for the outer keyword to entirely replace global
(which is actually module-scope).

As it stands, the different behaviours of names and attributes is only a
minor quirk, and the fix would definitely break backwards compatibility
in the language -- it'd have to be punted to Py3k.

 

[Paul Rubin]

Follow the logical implications of this proposed behaviour.

class Game:
current_level = 1
# by default, games start at level one

That's bogus. Initialize the current level in the __init__ method
where it belongs.

 

[Mike Meyer]

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

Yes. That's to much bondage for programmers who've become accustomed
to freedom. Explain why this should be illegal:

class C:

... def __getattr__(self, name):
... x = 1
... return locals()[name]
... def __setattr__(self, name, value):
... globals()[name] = value
...

o = C()
o.x = o.x + 1
x

2

I'll answer with a contra question.

Please explain why this is illegal.

x = 1
def f():
x += 1

f()

It isn't illegal, it just requires a different syntax.

<mike

 

[Mike Meyer]

The most obvious solution is to permit (or even require) the
programmer to list the instance variables as part of the class
definition. Anything not in the list is not an instance variable,
i.e. they don't get created dynamically. That's what most other
languages I can think of do. Some Python users incorrectly think this
is what __slots__ does, and try to use __slots__ that way. That they
try to do that suggests that the approach makes some sense.

That breaks the ability to add attributes dynamically, which is
usefull. If you need an extra piece of data with some existing class,
it's much easier to just add an attribute to hold it than to create a
subclass for the sole purpose of adding that attribute.

<mike

 

[Mike Meyer]

Why do you think so? I see nothing here that couldn't work with
a reference resolved during compile time.

a - in the global name space - doedn't exist when f is compiled, and
hence can't be dereferenced at compile time. Of course, sufficiently
advanced analysis can figure out that a would exist before f is run,
but that's true no matter how a is added. That isn't the way python
works.

No the second is not a direct result of the first. Since there is
only one reference, I see nothing wrong with the environment
remebering the reference and reusing it if it needs the reference
a second time.

Please stay on topic: we're talking about "a = a + 1", not "a += 1".
The former has two references, not one. I've already agreed that the
semantics of += are a wart.

<mike

 

[Mike Meyer]

equal? Some things are a matter of objective fact: should CPython use a
byte-code compiler and virtual machine, or a 1970s style interpreter that
interprets the source code directly?

For the record, I've only seen one interpreter that actually
interpreted the source directly. Pretty much all of the rest of them
do a lexical analysis, turning keywords into magic tokens (dare I say
"byte codes") and removing as much white space as possible. Or maybe
that's what you meant?

<mike

 

[Magnus Lycka]

That's bogus. Initialize the current level in the __init__ method
where it belongs.

But there is a relevant use case for this:

If you have a class hierarchy, where the difference between the
classes is mainly/completely a matter of data, i.e. default
values. Then it's very convenient to use such defaults in the
class scope.

Of course, you *could* have an __init__ in the base class that
copies this data from class scope to instance scope on instance
creation, but why make it more complicated?

You could also imagine cases where you have many instances and
a big immutable variable which typically stays as default, but
must sometimes vary between instances.

As I explained in another post, member lookups in the instance
must look in the class to find methods, so why not get used to
the fact that it works like this, and use it when it's convenient.
It's not as if anyone puts a gun to your head and force you to
use this feature.

 

[Paul Rubin]

A basic usage case:

class Paper:
size = A4
def __init__(self, contents):
# it makes no sense to have class contents,
# so contents go straight into the instance
self.contents = contents

So add:

self.size = Paper.size

and you've removed the weirdness. What do you gain here by inheriting?

 

[Bengt Richter]

Because the class variable doesn't define a self-mutating __iadd__
(which is because it's an immutable int, of course). If you want
b.__dict__['a'] += 2 or b.__class__.__dict__['a'] += 2 you can
always write it that way ;-)

(Of course, you can use a descriptor to define pretty much whatever semantics
you want, when it comes to attributes).

Because b.a += 2 expands to b.a = b.a + 2. Why would you want b.a =

No, it doesn't expand like that. (Although, BTW, a custom import could
make it so by transforming the AST before compiling it ;-)

Note BINARY_ADD is not INPLACE_ADD:
... b.a += 2
... b.a = b.a + 2
... 2 0 LOAD_GLOBAL 0 (b)
3 DUP_TOP
4 LOAD_ATTR 1 (a)
7 LOAD_CONST 1 (2)
10 INPLACE_ADD
11 ROT_TWO
12 STORE_ATTR 1 (a)

3 15 LOAD_GLOBAL 0 (b)
18 LOAD_ATTR 1 (a)
21 LOAD_CONST 1 (2)
24 BINARY_ADD
25 LOAD_GLOBAL 0 (b)
28 STORE_ATTR 1 (a)
31 LOAD_CONST 0 (None)
34 RETURN_VALUE

And BINARY_ADD calls __add__ and INPLACE_ADD calls __iadd__ preferentially.

About __ixxx__:
"""
These methods are called to implement the augmented arithmetic operations
(+=, -=, *=, /=, %=, **=, <<=, >>=, &=, ^=, |=).
These methods should attempt to do the operation in-place (modifying self)
and return the result (which could be, but does not have to be, self).
If a specific method is not defined, the augmented operation falls back
to the normal methods. For instance, to evaluate the expression x+=y,
where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not define
a __iadd() method, x.__add__(y) and y.__radd__(x) are considered, as with
the evaluation of x+y.
"""

<something> to correspond to b.__class__.a = <something>?

I'm not saying that it couldn't, if that was the model for inheritance you
decided to use. I'm asking why would you want it? What is your usage case
that demonstrates that your preferred inheritance model is useful?

It can be useful to find-and-rebind (in the namespace where found) rather
than use separate rules for finding (or not) and binding. The tricks for
boxing variables in closures show there is useful functionality that
is still not as convenient to "spell" as could be imagined.
It is also useful to find and bind separately. In fact, IMO it's not
separate enough in some cases ;-)

I've wanted something like
x := expr
to spell "find x and rebind it to expr" (or raise NameError if not found).
Extending that to attributes and augassign,
b.a +:= 2
could mean find the "a" attribute, and in whatever attribute dict it's found,
rebind it there. Or raise an Exception for whatever failure is encountered.
This would be nice for rebinding closure variables as well. But it's been discussed,
like most of these things ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

That's a wart in +=, nothing less. The fix to that is to remove +=
from the language, but it's a bit late for that.

Hm, "the" fix? Why wouldn't e.g. treating augassign as shorthand for a source transformation
(i.e., asstgt <op>= expr becomes by simple text substitution asstgt = asstgt <op> expr)
be as good a fix? Then we could discuss what

b.a = b.a + 2

should mean ;-)

OTOH, we could discuss how you can confuse yourself with the results of b.a += 2
after defining a class variable "a" as an instance of a class defining __iadd__ ;-)

Or point out that you can define descriptors (or use property to make it easy)
to control what happens, pretty much in as much detail as you can describe requirements ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

No that has nothing to do with resolving things at runtime. Your example
does not work because the language is very specific about looking up
global variables. Your programming error, not Python's shortcoming.

If someone has an old version of Python handy, I suspect that it used
to "work", and the "a" on the right hand side was the global "a" because
a local "a" hadn't been defined until the assignment, which worked to
produce a local binding of "a". Personally, I like that better than
the current way, because it follows the order of accesses implied
by the precedences in expression evaluation and statement execution.
But maybe I don't RC ;-)

Regards,
Bengt Richter

 

[Paul Rubin]

Hm, "the" fix? Why wouldn't e.g. treating augassign as shorthand for
a source transformation (i.e., asstgt <op>= expr becomes by simple
text substitution asstgt = asstgt <op> expr) be as good a fix? Then
we could discuss what

Consider "a[f()] += 3". You don't want to eval f() twice.

 

[Bengt Richter]

Take the code:

lst[f()] += 1

Now let f be a function with a side effect, that in succession
produces the positive integers starting with one.

What do you think this should be equivallent to:

t = f()
lst[t] = lst[t] + 1

or

lst[f()] = lst[f()] + 1

If you think the environment can change between references then I
suppose you prefer the second approach.

I am quite sympathetic to your probe of python semantics, but I
don't think the above is an argument that should be translated
to attribute assignment. BTW, ISTM that augassign (+=) is
a red herring here, since it's easy to make a shared class variable
that is augassigned apparently as you want, e.g.,
... def __init__(self, v=0): self.v=v
... def __get__(self, *any): return self.v
... def __set__(self, _, v): self.v = v
... ... a = shared(1)
...

>>> b=B()
>>> b.a 1
>>> B.a 1
>>> b.a += 2
>>> b.a 3
>>> B.a 3
>>> vars(b) {}
>>> vars(b)['a'] = 'instance attr'
>>> vars(b) {'a': 'instance attr'}
>>> b.a 3
>>> b.a += 100
>>> b.a 103
>>> B.a 103
>>> B.a = 'this could be prevented'
>>> b.a 'instance attr'
>>> B.a

'this could be prevented'

The spelled out attribute update works too 'alpha beta'

But the instance attribute we forced is still there {'a': 'instance attr'}

You could have shared define __add__ and __iadd__ and __radd__ also,
for confusion to taste ;-)

Regards,
Bengt Richter

 

[Mike Meyer]

Hm, "the" fix? Why wouldn't e.g. treating augassign as shorthand for a source transformation
(i.e., asstgt <op>= expr becomes by simple text substitution asstgt = asstgt <op> expr)
be as good a fix? Then we could discuss what

b.a = b.a + 2

should mean ;-)

The problem with += is how it behaves, not how you treat it. But you
can't treat it as a simple text substitution, because that would imply
that asstgt gets evaluated twice, which doesn't happen.

OTOH, we could discuss how you can confuse yourself with the results of b.a += 2
after defining a class variable "a" as an instance of a class defining __iadd__ ;-)

You may confuse yourself that way, I don't have any problems with it
per se.

Or point out that you can define descriptors (or use property to make it easy)
to control what happens, pretty much in as much detail as you can describe requirements ;-)

I've already pointed that out.

<mike