Class Variable Access and Assignment

[Paul Rubin]

Then you don't approve of inheritance? That's fine, it is your choice, but
as far as I know, all OO languages include inheritance.

Some OO languages only implement inheritance for method calls. Class
variables don't get inherited.

 

[Mike Meyer]

I would expect a result consistent with the fact that both times
b.a would refer to the same object.

Except they *don't*. This happens in any language that resolves
references at run time. Changing that would be changing a fundamental
- and *important* - feature of Python. Arbitrary restrictions to
prevent a single case of this from doing something people who aren't
used to suvh behavior are kludges, and would constitute a wart on the
language, pure and simple.

If you think this is bad, you should consider Jensen's device. It uses
call-by-name, which Python doesn't have. This defers evauation of an
argument until the actual use of the argument. You can fake it in
python by passing strings that you eval and/or lambdas that you call -
I'm not going to work out the details.

Here's the function:

real procedure SIGMA(x, i, n);
value n;
real x; integer i, n;
begin
real s;
s := 0;
for i := 1 step 1 until n do
s := s + x;
SIGMA := s;
end

The object referred to by "x" in the body of the loop is different
*every pass through the loop*, at least in the expected usage. That
usage is SIGMA(a(i), i, 10). Call-by-name evaluates a(i) when x is
mentioned. Since i changes with each pass through the loop, a(i) is a
different element in the array. So SIGMA(a(i), i, 10) is sum(a[:10])
(give or take). But SIGMA(a(i) * b(i), i, 10) is sum([a(i) * b(i) for
i in xrange(10)]) (more or less).

This isn't insane, it's a feature. It's a *very powerful*
feature. Yes, it causes behavior that's unexpected and appears to be
wrong to people who aren't used to resolving names at run time. They
need to get used to it. Then they can start taking advantage of it.

I think it even less sane, if the same occurce of b.a refers to two
different objects, like in b.a += 2

That's a wart in +=, nothing less. The fix to that is to remove +=
from the language, but it's a bit late for that.

<mike

 

[Graham]

Thanks to all of you for your reply's i had no idea it would get this
sort of response,
i've read through most of the posts and it seems that its a active
topic.

My question remains however, i suppose i'm not familiar with how this
functions in
other languages, but what is the common way of referring to a class
variable.

is <class>.<var> the norm?
or <instance>.<var> the norm.

I just seems to me that <instance>.<var> shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

Is my thinking way off here?

Graham

 

[Mike Meyer]

I just seems to me that <instance>.<var> shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

Is my thinking way off here?

Yes. This behavior is how you get inheritted access to class
variables. Consider:

class Counter(object):
"A mutable counter."
# implementation elided

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_count.increment()

class B(A):
instance_count = Counter()

This is sufficient to count instances of A and B. If
self.instance_count raised an exception, you'd have to do something
like A.instance_count in A.__init__, which would mean the __init__ B
inherited from A would do the wrong thing. Currently, you can either
reference class variables by explicit class, or via inheritance, and
both behaviors are desirable. If you're going to disallow
self.class_variable, you need to come up with a mechanism to replace
the latter behavior.

<mike

 

[Steven D'Aprano]

Some OO languages only implement inheritance for method calls. Class
variables don't get inherited.

Fascinating. Which languages, and what is the reasoning behind that?

I bet they are languages that force the class designer to spend half their
time writing setters and getters. Am I right?

 

[Graham]

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.
Given your previous example:

class Counter(object):
"A mutable counter."
# implementation elided

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_count.increment()

if you changed

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_count.increment()
to

class A(object):
instance_count = 0
def __init__(self):
self.instance_count = self.instance_count + 1

It would not work as planned. I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect. It seems to me that
self.instance_count should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

Does that make sense?

Again thank you for explaination.

graham

 

[Bengt Richter]

I understand what happens too, that doesn't make it sane behaviour.


I don't care what should be different. But a line with only one
referent to an object in it, shouldn't be referring to two different
objects.

In the line: b.a += 2, the b.a should be refering to the class variable
or the object variable but not both. So either it could raise an
attribute error or add two to the class variable.

Sure one could object to those sematics too, but IMO they are preferable
to what we have now.

A somewhat similar name space problem, where you could argue
that "a" prior to += should be seen as defined in the outer scope,
but lookahead determines that a is local to inner, period, so that
is the reference that is used (and fails).
... a = 1
... def inner():
... a += 2
... print a
... print 'outer a', a
... inner()
... print 'outer a', a
... outer a 1
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in outer
File "<stdin>", line 4, in inner
UnboundLocalError: local variable 'a' referenced before assignment

Regards,
Bengt Richter

 

[Steven D'Aprano]

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.

You mean, like this?

# access the value of an instance attribute:
x = instance.name

# assign the value of an instance attribute:
x = instance.name

No, of course that's not what you *meant* -- but it is
what you said. So what do you actually mean? I'll
assume you mean something like "the scoping rules for
access and assignment should be the same".

The problem is, they are. When you access an attribute,
Python looks in the instance scope, and if that lookup
fails, it looks in the class scope. When you assign an
attribute, at least ignoring the complication of slots,
Python uses the same scoping rules: assign to the
instance scope, WHICH ALWAYS SUCCEEDS, and if it fails,
assign to the class scope.

In fact, Python may not even bother to include code to
push the assignment to the class, since the assignment
at the instance level is guaranteed to either succeed,
or fail in such a way that you have no choice but to
raise an exception. But conceptually, assignment and
access are using the same scoping rules.

(Again, I stress that new-style classes with slots may
do things differently.)

Now that I've demonstrated that what you want is not
either "assignment and access should be the same", nor
"the scoping rules should be the same", can you
describe precisely what you do want?


See below for further details.

Given your previous example:




if you changed



It would not work as planned.

Why not?

For starters, what is the plan? Do you want all
instances of class A to share state? Are all instances
of Counter supposed to share state?

Depending on whether you want the answers of those to
be Yes or No, you would pick one technique or the
other. Sometimes you want to increment mutables in
place, and sometimes you don't.

But I would suggest very strongly that in general, you
usually don't want instances to share state.

I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect. It seems to me that
self.instance_count should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

But that would stop inheritance from working the
expected way.

In standard OO programming, you expect instances to
inherit behaviour from their class (and superclasses)
unless over-ridden. This lets you do something like this:

class Paper:
size = A4

Now all instances of Paper are created with a default
size of A4 -- they inherit that size from the class.

If you are localising your application for the US
market, you simply change the class attribute:

Paper.size = USLetter

and all the instances that inherit from the class will
now reflect the new default.

Now suppose you have a specific instance that needs a
different paper size:

instance = Paper()
instance.size = Foolscap

What do you expect should happen? Should all Paper
instances suddenly be foolscap size, or just the one?
If you say "just the one", then you want the current
behaviour. If you say "all of them", then you want
shared state -- but do you really want all class
instances, all the time, to have shared state?


You can get ride of that behaviour by getting rid of
inheritance, or at least inheritance of non-method
attributes. Then you have to write code like this:

class PrintableThing:
"""Prints a Thing object with prefix and suffix.
Customize the prefix and suffix by setting the
appropriate instance attributes.
"""

prefix = "START "
suffix = " STOP"

def __str__(self):
try:
# access the instance attributes,
# if they exist
prefix = self.prefix
suffix = self.suffix
except AttributeError:
# fall back to class attributes
prefix = self.__class__.prefix
suffix = self.__class__.suffix
# have you spotted the subtle bug in this code?
return prefix + self.thing + suffix

instead of:

class PrintableThing:
def __str__(self):
return self.prefix + self.thing + self.suffix


Even worse would be the suggestion that Python allowed
accessing instance.attribute to refer to either a class
or instance attribute, decided at runtime as it does
now, but *remembered* which it was so that assignment
went back to the same object.

That would mean that class attributes would mask
instance attributes -- or vice versa, depending on
which was created first. I assume that in general,
class attributes would be created before instances.

If you had a class with a default attribute, like
Paper.size above, you couldn't over-write it at the
instance level because instance.size would always be
masked by class.size. You would need to write code like
this:

class Paper:
default_size = A4

def __init__(self, size=None):
self.size = size

def print(self):
if self.size is None:
papersize = self.__class__.default_size
else:
papersize = self.size
do_something(papersize)


The standard inheritance model used by Python and all
OO languages I know of is, in my opinion, the optimal
model. It gives you the most convenient behaviour for
the majority of cases, and in those few cases where you
want non-standard behaviour (e.g. shared state) it is
easy to do with some variant of self.__class__.attribute.

 

[Mike Meyer]

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.

That's not true in lots of cases, and would be a serious restriction
on the language. The rules for where a binding takes place are the way
the are for good reason. Restricting lookups to those rules would make
a number of things more difficult, and would make some features
(cough-closures-cough) nearly useless.

Given your previous example:




if you changed

It would not work as planned. I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect.

No, it acts in a different way than *you* expect. It does exactly what
I expect, which is why I didn't write it that way.

It seems to me that
self.instance_count should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

Does that make sense?

Yes, but such behavior would make Python worse, not better. Right
now, binding instance.name *always* works(*) on the value of the name
attribute of instance. Creating a special case for when one of the
classes instance belongs to happens to have an attribute "name" would
be less consistent than the current behavior. Yes, the current
behavior is surprising to people who aren't used to dynamic
languages. But there are lots of such things - they're part of the
power of dynamic languages. People who want to program in dynamic
languages just have to get used to those things. That's a lesser price
to pay than having a language cluttered with special cases just to
avoid surprising people who aren't used to the language yet.

Again thank you for explaination.

You're welcome.

<mike

*) I almost said "binds", but that's not true. Attempting to bind an
attribute can invoke arbitrary code - but it's got the instance and
attribute name to work with. If you really wanted to, you could create
a class for which trying to set an attribute behaved as you wanted. Be
warned - it's not as easy as it looks.

 

[Steve Holden]

Yes it does. If the b.a refers to the instance variable, then an
AttributeError should be raised, because the instance variable doesn't
exist yet, so you can't add two to it.

Excuse me. The statement

a += 2

causes a to refer to a different object after the assignment than it did
before. So does the statement

self.a += 2

So why are you so concerned that the pre-assignment reference comes from
a different scope to the post-assignment reference? The fact remains
that after both assignments the rebound name can no longer (ever) be
used to refer to its former referent without a further rebinding taking
place.

If the b.a refers to the class variable then two should be added to it.

Wring, wring, wring. (Sorry, got that wrong

Neither happens instead we get some hybrid in which an instance varible
is created that gets the value of class variable incrented by two.

Yes. So does this mean you also have a problem with

def f(x):
x += 2

g = 3
print f(g)

When the function call executes, the name x is bound to an object in the
call's containing scope. Is it then your contention that the augmented
assignment in the function should add two to that object, changing the
value of g?

For extra marks please explain the difference between augmented
assignment to a function argument and augmented assignment to a class
variable referenced through self.I'm not sure where this moral imperative comes from, and your arguments
singularly fail to convince me.

There is no instance variable at that point. How can it add 2, to
something that doesn't exist at the moment.

It doesn't, it simply proceeds along the lines of all Python assignments
and resolves the name as a reference to a specific object. It then
computes a new value from the referenced object and the augmented
assignment operator's right operand, and rebinds the name to the
newly-computed value.

Please stop talking about variables.

Although augmented assignment operators have the *option* of updating
objects in place, surely not even you can require that they do so when
they are bound to an immutable object such as an integer.Why "should" it? Why, why, why? And gain, just for good measure, why?
Augmented assignment to a function argument doesn't modify the passed
object when immutable, and you have no problem with that (I know as I
write that this is just asking for trouble, and it will turn out that
you also find that behavior deeply controversial ...)

But it accesses the class variable.

Repeat after me: "Python assignment binds values to names".

When I write

class something:
a = 1
def __init__(self, val=None):
if val:
self.a += val

then in the last statement the augmented assignment rebinds "self.a"
from the class "variable" to a newly-created instance "variable". I am
of course using the word "variable" here in a Pythonic sense, rather
than in the sense that, say, a C programmer would use. In Python I
prefer to talk about binding names because talking of variables leads
people to expect that a name is bound to an area of memory whose value
is modified by assignment, but this is in fact not so.

The initial access to self.a uses the defined name resolution order to
locate a value that was bound to the name "a" in class scope. So what?
This is a long-documented fact of Python life. It's *supposed* to be
that way, dammit.

I fail to understand why this is such a problem for you. But then it's
clear from long past experience that our perceptions of Python's
execution model differ quite radically, and that I seem to find it quite
satisfactory overall, whereas you are forever banging on about what
"should" be true of Python and what Python "should" do. Which, as is
probably obvious by now, I sometimes find just a teeny bit irritating.
Kindly pardon my tetchiness.

I suppose ultimately I'm just more pragmatic than you. Plus I started
using Icon, whose assignment semantics are very similar, back in the
1970's, so Python's way of doing things fits my brain quite nicely,
thank you.

regards
Steve

PS As a total non-sequitur added for light relief at the end of what
seems even to me to be a slightly tedious post, I discover I managed to
misspell "assignment" in four distinct ways during the composition of
the above.

 

[Steve Holden]

Thanks to all of you for your reply's i had no idea it would get this
sort of response,
i've read through most of the posts and it seems that its a active
topic.

My question remains however, i suppose i'm not familiar with how this
functions in
other languages, but what is the common way of referring to a class
variable.

is <class>.<var> the norm?
or <instance>.<var> the norm.

Let me try to give a simple answer to a simple question before you begin
to feel that we are all barmy and maybe Perl would be an easier
alternative

Under normal circumstances, yes, if you want to reference and/or modify
a value that's intended to be shared amongst all instances of a given
class you would normally refer to that as

classname.var

and such a reference will work inside any of the class's methods.

Inside the class body (but outside any method body) the same reference
can be written as

var

I just seems to me that <instance>.<var> shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

Is my thinking way off here?

It's a long way from how Python is designed, because name resolution in
Python will look in an instance's class (and then the class's
superclass, and so on until object is reached, which is at the top of
all modern class hierarchies) unless the name is first found in the
instance's namespace.

That's just the way Python was designed. Like it or loathe it, it's too
late to change now. <shrug>. Some languages only use such a technique
for resolving the names associated with methods, but this isn't
comp.lang.someotherlanguage

pragmatical-ly y'rs - steve

 

[Steve Holden]

Op 2005-11-03, Stefan Arentz schreef <[email protected]>: [...]

It is all according to how things have been in Python for a long time.

Unsane behaviour for a long time is still unsane behaviour.

As your continued contributions on this newsgroup so adequately
demonstrate .

Sorry, I *couldn't* resist. You asked for it. It was hanging there (in a
containing namespace?) waiting to be posted. If I hadn't said it someone
else would have. And other justifications for what I hope doesn't seem
like too unpleasant a personal attack.

The fact that this can be regarded as unwise coding, doesn't imply
it is sane behaviour of python. Variable shadowing happens. I don't
consider it sane behaviour if the same reference in a line gets
resolved in different name spaces

Well I'm sure Guido will be happy to know you think his design is
insane. Now who's calling who names?

regards
Steve

 

[Antoon Pardon]

That is an explanation, not a reason.

I'm just following the

1. get 't'b.a and store it in a temporary 't' (found the instance)
2. add 2 to 't'
3. store 't' in 'b.a'

The last operation stores it into an instance variable.

[ I think you mean '(found the class variable)' in line 1 ]

All you are doing here is explain how the current implemantation treats
this. You are not giving arguments for why the result of this
implementation should be considered sane behaviour.

Ah yes. Well, good luck with that. You seem to have decided that it is not
sane and who am I to argue with that. It depends on your state of mind

I can just say the opposite, that you seem to have decided that it is
sane.

The model makes sense in my opinion. If you don't like it then there are
plenty of other languages to choose from that have decided to implement
things differently.

And again this argument. Like it or leave it, as if one can't in general
like the language, without being blind for a number of shortcomings.

It is this kind of recations that make me think a number of people is
blindly devoted to the language to the point that any criticism of
the language becomes intollerable.

 

[Graham]

Once again, many thanks, your explainations are very detailed and i
think i'm in full understanding of the what/when/why of it all.

And with further introspection i can see why its done this way from a
language processing point of view rather than programming one. I also
now realize that <instance>.<classvarname> is there so that you dont
have to type <instance>.__class__.<varname> all the time.

Once again many thanks, i hadn't expected nearly this type of response.


Graham.

 

[Antoon Pardon]

Because b.a += 2 expands to b.a = b.a + 2. Why would you want b.a =
<something> to correspond to b.__class__.a = <something>?

That is an implemantation detail. The only answer that you are given
means nothing more than: because it is implemented that way.

I'm not saying that it couldn't, if that was the model for inheritance you
decided to use. I'm asking why would you want it? What is your usage case
that demonstrates that your preferred inheritance model is useful?

It has nothing to do with a model for inheritance, but with a model of
name resolution.

The hierarchie of searching an instance first in an object and then in
a class isn't that different from searching first in a local namespace
and then in a more global namespace.

When we search names in a function we don't resolve the same name in
different name spacese each occurence of the same name in the same
function occurs in the same namespace.

But with class variables we can have that one and the same name
on a line refers to two different namespaces at the same time.
That is IMO madness. You may argue that the madness is of little
importance, you can argue that because of the current implementation
little can be done about it. But I don't see how one can defend
it as sane behaviour.

 

[Antoon Pardon]

Op 2005-11-03, Stefan Arentz schreef <[email protected]>: [...]

It is all according to how things have been in Python for a long time.

Unsane behaviour for a long time is still unsane behaviour.

As your continued contributions on this newsgroup so adequately
demonstrate .

Sorry, I *couldn't* resist. You asked for it. It was hanging there (in a
containing namespace?) waiting to be posted. If I hadn't said it someone
else would have. And other justifications for what I hope doesn't seem
like too unpleasant a personal attack.

Well I would argue that a lot of the defenders of python are not
reacting very sane. My impression is that a lot of them react like
zealots, blindly devoted to the language, rather intollerant of
every criticism and prepared to defend anything as long as it happens
to be a current characteristic of the language and where any such
criticism sooner or later is met with something like: "If you don't
like it, use a different language", as if only those who are 100%
perfectly happy with the language as it is, should use it.

Well I'm sure Guido will be happy to know you think his design is
insane. Now who's calling who names?

I'm not calling anyone names. I'm just pointing to one specific
behaviour in python and call that behaviour unsane. If you want
to interpret that as me calling his (entire) design insane, I
suggest you are two defensive with regards to python.

 

[Antoon Pardon]

class A:
a = 1
b = A()
b.a += 2
print b.a
print A.a
Which results in
3
1

I don't suppose you'd care to enlighten us on what you'd regard as the
superior outcome?

class A:
a = []
b = A()
b.append(3)
print b.a
print a.a

Compare and contrast.

I take it then that you believe that ints like 1 should be mutable like
lists? Because that is what the suggested behaviour implies.

No it isn't.

One other way, to implement the += and likewise operators would be
something like the following.

Assume a getnsattr, which would work like getattr, but would also
return the namespace where the name was found. The implementation
of b.a += 2 could then be something like:

ns, t = getnsattr(b, 'a')
t = t + 2
setattr(ns, 'a')


I'm not arguing that this is how it should be implemented. Just
showing the implication doesn't follow.

 

[Antoon Pardon]

It isn't broken, there is nothing to fix. The code does precisely what the
inheritance model promises to do.

That is not a contra argument. Delivering what is promissed says nothing
about the sanity of what is promissed. It is even possible that a number
of things that are sane in itself, produce something unsane when
combined or in certain circumstances.

 

[Antoon Pardon]

Because 'a += 1' is only a shorthand for 'a = a + 1' if a is an
immutable object? Anyway, the behaviour is well documented.

http://docs.python.org/ref/augassign.html says:

An augmented assignment expression like x += 1 can be rewritten as x = x
+ 1 to achieve a similar, but not exactly equal effect. In the augmented
version, x is only evaluated once.

Then couldn't we expect that the namespace resolution is also done
only once?

I say that if the introduction on += like operators implied that the
same mentioning of a name would in some circumstances be resolved to
two different namespaces, then such an introduction would better have
not occured.

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

 

[Stefan Arentz]

....

I can just say the opposite, that you seem to have decided that it is
sane.

I have. I like the Python model.

And again this argument. Like it or leave it, as if one can't in general
like the language, without being blind for a number of shortcomings.

Personally I don't see it as a shortcoming.

It is this kind of recations that make me think a number of people is
blindly devoted to the language to the point that any criticism of
the language becomes intollerable.

No not at all. Just look at all the PEPs and the changes in the language
that have been made in the past. Python is very much community driven and
that shows in it's progress.

You on the other hand keep talking about emo things like 'sane' and
'madness' without giving any technical backing about these problems
that you are having with the language.

Snap out of that, make it a real discussion and maybe something good
will happen. Or not

S.