initializing mutable class attributes

[Dan Perl]

There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is already awkward because there is such a difference between attr1 and
attr2. But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

I find this even more awkward because many people will forget to do it.
Clearly, this is then a more general issue with __init__, but I think it is
accentuated by the fact that you HAVE TO HAVE __init__ in order to
initialize attributes that are mutable.

Is there something I don't know here and there is a better way to do this in
Python? I would like to get a better solution or otherwise start a
discussion.

 

[Benjamin Niemann]

That's the way it is supposed to work. Instance attributes have to be
initialized via self.foo=..., usually in __init__() which in turn is
*STRONGLY* advised to class its parents __init__() - or you create
instance attributes 'on-the-fly' when they are used for the first time
(useful technique for mixin classes without constructor). Class
attributes are initialized once for the class and are shared between
instances.
"self.attr1=data" in Father.foo() doesn't "override" the Father.attr1
attribute you defined before. It creates an instance attribute that
shadows Father.attr1!
Both attribute assignments in Father are OK - if you treat them as class
attributes. They won't become instance attributes by hidden magic.

 

[Alex Martelli]

There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

I disagree: good Python programmers often use mutable class attributes,
and use them to good effect. I've done my share of Python teaching,
consulting and mentoring, and I definitely do *not* think this usage "is
desired sometimes but usually it is just a bug".

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is the canonical way, sure.

This is already awkward because there is such a difference between attr1 and
attr2.

One is a class attribute (which nobody forced you to have), the other is
an instance attribute. If you want both to be instance attributes,
initialize both in __init__ -- that's all there is to it. Don't use
class attributes unless there's a reason for them to be class
attributes, that seems like a good and sensible rule of thumb to me.

But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

Assuming an instance of Child needs to have both attributes attr2 and
attr3, yes.

I find this even more awkward because many people will forget to do it.

Forget to do what -- call the parent class's __init__? Again, this is
rare -- not as rare as the other "usually just a bug" you opined about,
but not common. A class normally needs __init__ for a large number of
purposes, of course, not just to assign mutable attributes to each
instance, therefore people who learn subclassing do learn to call the
parent class __init__ very early on, to avoid everything breaking.

Clearly, this is then a more general issue with __init__, but I think it is
accentuated by the fact that you HAVE TO HAVE __init__ in order to
initialize attributes that are mutable.

Making __init__ more common means people are more likely to get used to
it, and thus makes the bug of not calling a parent class's __init__
rarer, not "accentuated".

Is there something I don't know here and there is a better way to do this in
Python? I would like to get a better solution or otherwise start a
discussion.

There is no BETTER way, IMHO, than normal Python coding, which I believe
is quite good -- but you appear to disagree, therefore it's hard for me
to gauge what you may consider "better". There is definitely a
DIFFERENT way -- coding a custom metaclass which will tweak the
instantiation of all classes belonging to it, in order to insert some
funky initialization of mutable attributes, and perhaps automate calls
to superclasses' __init__ methods, and so on. Such automation is a
ticklish issue, in the general case, so let's assume initializing
mutable attributes is all we ever want to do. Then, for example...:

import copy
class metaImu(type):
def __call__(cls, *a, **k):
instance = type.__call__(cls, *a, **k)
for sup in cls.mro():
muts = gerattr(sup, '__mutables__', ())
for mutname in muts:
if hasattr(instance, mutname): continue
mutvalue = muts[mutname]
setattr(instance, mutname, copy.copy(mutvalue))
return instance

class Father:
__metaclass__ = metaImu
__mutables__ = dict(attr1=None, attr2={})

class Child(Father):
__mutables__ = dict(attr3=[])

I haven't tested this, but I believe it should behave as you appear to
desire (assuming you want shallow rather than deep copying of the
values, of course: I can't read your mind!). I specifically showed that
you can use __mutables__ also for attributes which are in fact not
mutable, since copy.copy should be innocuous (identity) on those, just
in case you insist on having the same form for all such attributes. I
have also assumed that an instance's __init__ may be allowed to set its
own value for an attribute in __mutables__ and if it does so then that
should be left alone. But of course you can play with all of these
aspects at will. You could even have that __call__ method examine all
attributes defined in the class (or any superclass) and force copies of
them into instance attributes for all whose types are [[whatever
criteria you like...]].


Alex

 

[Dan Perl]

I will clarify something that was probably not clear enough in my initial
posting. I am interested in instance attributes, but I am using the class
attributes to provide a default, "null", value for the instance attributes.
Yes, the instance attributes only mask the class ones, but a class attribute
and an instance attribute can be used like this:
class WorksForMe:
attr=None # used just as a default value for the instance
attribute self.attr
def foo(self):
self.attr=SomethingUseful( )
def bar(self):
if self.attr:
useIt(self.attr)
else:
ignoreIt(self.attr)
This makes it safe for an instance of WorksForMe to invoke bar( ) even if it
never invoked foo( ).

That's the way it is supposed to work. Instance attributes have to be
initialized via self.foo=..., usually in __init__() which in turn is
*STRONGLY* advised to class its parents __init__() - or you create
instance attributes 'on-the-fly' when they are used for the first time
(useful technique for mixin classes without constructor). Class
attributes are initialized once for the class and are shared between
instances.

You are confirming the code I suggested but you don't share my view that
such code is awkward. I think I I was actually conservative calling it
awkward, I would rather call it unsafe. If I were a third-party library
vendor, I would not be comfortable providing a mixin class that forces users
to either invoke the parent's __init__ in their constructors or to
initialize the instance attributes on-the-fly. The latter would even be
against the principles of encapsulation, users should not need to know about
the parent's attributes, especially because they may change in future
releases. Both ways of restricting the use of the mixin class are unsafe
because they are easy to be ignored by users. And it is not only the users'
fault if they do not follow the restrictions, it is also my fault to put
such restrictions in the first place.

I think this points to a weakness in Python as an OO language. I'm not
trying to bash it, but my understanding is that Python is still evolving in
that respect and I think that this should be improved in the future. Then,
on the other hand, maybe I'm the only one feeling this way and there's not
going to be a change.

"self.attr1=data" in Father.foo() doesn't "override" the Father.attr1
attribute you defined before. It creates an instance attribute that
shadows Father.attr1!
Both attribute assignments in Father are OK - if you treat them as class
attributes. They won't become instance attributes by hidden magic.

There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is already awkward because there is such a difference between attr1 and
attr2. But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

I find this even more awkward because many people will forget to do it.
Clearly, this is then a more general issue with __init__, but I think it is
accentuated by the fact that you HAVE TO HAVE __init__ in order to
initialize attributes that are mutable.

Is there something I don't know here and there is a better way to do this in
Python? I would like to get a better solution or otherwise start a
discussion.

 

[Peter Otten]

something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

I find this even more awkward because many people will forget to do it.

You have to call base-class-__init__() in every non-trivial inheritance
scheme. Should you forget it, pychecker is always there to remind you

<sloppy-code>
class X:
def __init__(self):
self.x = 2

class Y(X):
pass

class Z(Y):
def __init__(self):
pass
</sloppy-code>

And here's what pychecker says:

$ pychecker t1.py
Processing t1...

Warnings...

t1.py:9: Base class (t1.Y) __init__() not called

Clear enough.

Peter

 

[Larry Bates]

Dan,

I too use class attributes to assign defaults, but do
it like following:

import copy

class Father:
_attr1default=None
_attr2default=[]

def __init__(self):
self.attr1=self._attr1default
# Get copy of attr2default
self.attr2=list(self._attr2default)

def foo(self, data):
self.attr1=data
self.attr2.append(data)


class Child (Father):
def __init__(self):
Father.__init__(self)
#
# At this point self.attr1 (instance)
# and self.attr2 (instance) hold their defaults
# while _attr1/_attr2default (class) still hold
# their default values for initialization of more
# class instances.
#

HTH,
Larry Bates
Syscon, Inc.

I will clarify something that was probably not clear enough in my initial
posting. I am interested in instance attributes, but I am using the class
attributes to provide a default, "null", value for the instance attributes.
Yes, the instance attributes only mask the class ones, but a class attribute
and an instance attribute can be used like this:
class WorksForMe:
attr=None # used just as a default value for the instance
attribute self.attr
def foo(self):
self.attr=SomethingUseful( )
def bar(self):
if self.attr:
useIt(self.attr)
else:
ignoreIt(self.attr)
This makes it safe for an instance of WorksForMe to invoke bar( ) even if it
never invoked foo( ).

That's the way it is supposed to work. Instance attributes have to be
initialized via self.foo=..., usually in __init__() which in turn is
*STRONGLY* advised to class its parents __init__() - or you create
instance attributes 'on-the-fly' when they are used for the first time
(useful technique for mixin classes without constructor). Class
attributes are initialized once for the class and are shared between
instances.

You are confirming the code I suggested but you don't share my view that
such code is awkward. I think I I was actually conservative calling it
awkward, I would rather call it unsafe. If I were a third-party library
vendor, I would not be comfortable providing a mixin class that forces users
to either invoke the parent's __init__ in their constructors or to
initialize the instance attributes on-the-fly. The latter would even be
against the principles of encapsulation, users should not need to know about
the parent's attributes, especially because they may change in future
releases. Both ways of restricting the use of the mixin class are unsafe
because they are easy to be ignored by users. And it is not only the users'
fault if they do not follow the restrictions, it is also my fault to put
such restrictions in the first place.

I think this points to a weakness in Python as an OO language. I'm not
trying to bash it, but my understanding is that Python is still evolving in
that respect and I think that this should be improved in the future. Then,
on the other hand, maybe I'm the only one feeling this way and there's not
going to be a change.

"self.attr1=data" in Father.foo() doesn't "override" the Father.attr1
attribute you defined before. It creates an instance attribute that
shadows Father.attr1!
Both attribute assignments in Father are OK - if you treat them as class
attributes. They won't become instance attributes by hidden magic.

There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is already awkward because there is such a difference between

attr1

and

attr2. But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

I find this even more awkward because many people will forget to do it.
Clearly, this is then a more general issue with __init__, but I think

it

is

 

[Dan Perl]

There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

I disagree: good Python programmers often use mutable class attributes,
and use them to good effect. I've done my share of Python teaching,
consulting and mentoring, and I definitely do *not* think this usage "is
desired sometimes but usually it is just a bug".

My mistake. I shouldn't have made that statement. I don't really know
which case happens more often. Having said that, that is probably a mistake
that many beginners make. Okay, I have no real way of knowing that either,
but I think it's a good assumption.

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is the canonical way, sure.

This is already awkward because there is such a difference between attr1 and
attr2.

One is a class attribute (which nobody forced you to have), the other is
an instance attribute. If you want both to be instance attributes,
initialize both in __init__ -- that's all there is to it. Don't use
class attributes unless there's a reason for them to be class
attributes, that seems like a good and sensible rule of thumb to me.

I was using the class attribute as a default, "null", value for the instance
attributes. That works just fine for immutable attributes and it allows me
not to implement an __init__.

But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

Assuming an instance of Child needs to have both attributes attr2 and
attr3, yes.

I find this even more awkward because many people will forget to do it.

Forget to do what -- call the parent class's __init__? Again, this is
rare -- not as rare as the other "usually just a bug" you opined about,
but not common. A class normally needs __init__ for a large number of
purposes, of course, not just to assign mutable attributes to each
instance, therefore people who learn subclassing do learn to call the
parent class __init__ very early on, to avoid everything breaking.

Clearly, this is then a more general issue with __init__, but I think it is
accentuated by the fact that you HAVE TO HAVE __init__ in order to
initialize attributes that are mutable.

Making __init__ more common means people are more likely to get used to
it, and thus makes the bug of not calling a parent class's __init__
rarer, not "accentuated".

I'm leaving the rest of your reply out, but thanks for the suggestion of a
metaclass, I'll look into it.

After seeing a few replies (yours, Benjamin Niemann's and Peter Otten's) to
my initial posting, I think I am getting the picture that there is a
conscious decision to keep the use of __init__ the way it is and just make
people learn it and learn it early enough. That's a valid approach and I'll
accept it.

I can also understand your frustration with people who are new to Python,
like me, coming from a background in other OO languages, who are not yet
comfortable with "the Python way" and feel that there is a "better way" and
who suggest changing Python. But you also have to be reallistic and accept
that there will always be people coming to Python from other languages and
that the adjustment is rather difficult in some areas. You may just say
'Tough!', or you may help them to make that adjustment (I think you are
doing that), or you may even accommodate them (you should at least consider
that).

No one, including you, has given me a reason WHY __init__ is implemented
this way. I am not bashing you for that, I would just still like to hear
that 'WHY'. I'm sure that this implementation has some advantages. But,
coming from a C++ and Java background, where parent default constructors are
automatically invoked (well, not always, and that is something that users
have to learn too), I find that that approach has some clear advantages.

Those are my 2 cents.

Dan
PS: Does my last name attract the wrong kind of attention from people in
this newsgroup? It's really my name, it's not fake, BTW.

 

[Dan Perl]

Larry,

I don't think it is very different from what I suggested. You still
implement an __init__ and you initialize the instance attributes there.

Thanks, though. It's a nice trick and it's useful because you have a
default value that can be used also in other places in the code (like
restoring the default). And it's probably a good use of pseudo-private
class attributes.

Dan

Dan,

I too use class attributes to assign defaults, but do
it like following:

import copy

class Father:
_attr1default=None
_attr2default=[]

def __init__(self):
self.attr1=self._attr1default
# Get copy of attr2default
self.attr2=list(self._attr2default)

def foo(self, data):
self.attr1=data
self.attr2.append(data)


class Child (Father):
def __init__(self):
Father.__init__(self)
#
# At this point self.attr1 (instance)
# and self.attr2 (instance) hold their defaults
# while _attr1/_attr2default (class) still hold
# their default values for initialization of more
# class instances.
#

HTH,
Larry Bates
Syscon, Inc.

I will clarify something that was probably not clear enough in my initial
posting. I am interested in instance attributes, but I am using the class
attributes to provide a default, "null", value for the instance attributes.
Yes, the instance attributes only mask the class ones, but a class attribute
and an instance attribute can be used like this:
class WorksForMe:
attr=None # used just as a default value for the instance
attribute self.attr
def foo(self):
self.attr=SomethingUseful( )
def bar(self):
if self.attr:
useIt(self.attr)
else:
ignoreIt(self.attr)
This makes it safe for an instance of WorksForMe to invoke bar( ) even

if

it
never invoked foo( ).

That's the way it is supposed to work. Instance attributes have to be
initialized via self.foo=..., usually in __init__() which in turn is
*STRONGLY* advised to class its parents __init__() - or you create
instance attributes 'on-the-fly' when they are used for the first time
(useful technique for mixin classes without constructor). Class
attributes are initialized once for the class and are shared between
instances.

You are confirming the code I suggested but you don't share my view that
such code is awkward. I think I I was actually conservative calling it
awkward, I would rather call it unsafe. If I were a third-party library
vendor, I would not be comfortable providing a mixin class that forces users
to either invoke the parent's __init__ in their constructors or to
initialize the instance attributes on-the-fly. The latter would even be
against the principles of encapsulation, users should not need to know about
the parent's attributes, especially because they may change in future
releases. Both ways of restricting the use of the mixin class are unsafe
because they are easy to be ignored by users. And it is not only the users'
fault if they do not follow the restrictions, it is also my fault to put
such restrictions in the first place.

I think this points to a weakness in Python as an OO language. I'm not
trying to bash it, but my understanding is that Python is still evolving in
that respect and I think that this should be improved in the future. Then,
on the other hand, maybe I'm the only one feeling this way and there's not
going to be a change.

"self.attr1=data" in Father.foo() doesn't "override" the Father.attr1
attribute you defined before. It creates an instance attribute that
shadows Father.attr1!
Both attribute assignments in Father are OK - if you treat them as class
attributes. They won't become instance attributes by hidden magic.

Dan Perl wrote:
There is something with initializing mutable class attributes that I am
struggling with. I'll use an example to explain:
class Father:
attr1=None # this is OK
attr2=[ ] # this is wrong
def foo(self, data):
self.attr1=data
self.attr2.append(data)
The initialization of attr1 is obviously OK, all instances of Father
redefine it in the method foo. But the initialization of attr2 is wrong
because all the instances of Father end up sharing the same value. Maybe
that is desired sometimes, but usually it is just a bug.

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is already awkward because there is such a difference between

attr1

and
attr2. But moreover, I think this forces subclasses of Father to do
something like this:
class Child (Father):
def __init__(self):
Father.__init__(self)
self.attr3=[ ]

I find this even more awkward because many people will forget to do it.
Clearly, this is then a more general issue with __init__, but I

think

it

 

[Dan Perl]

Thanks, Peter. pychecker detecting the wrong use makes the argument that
there is no excuse for making the mistake of not calling the parent's
__init__ in subclasses. I am regularly using pychecker on my code, but I
never saw that warning because I did use __init__ correctly.

But I did make the mistake once of using a class attribute with an empty
dictionary to initialize an instance attribute just because I was doing the
same thing with immutable attributes. Fortunately, I caught it soon enough
and I will not make that mistake again. But it was kind of an embarrassing
mistake and I learned my lesson only after that.

Dan

 

[F. Petitjean]

So the only solution I see to this is to initialize attr2 in __init__:
class Father:
attr1=None
def __init__(self):
self.attr2=[ ]

This is the canonical way, sure.

Snip

After seeing a few replies (yours, Benjamin Niemann's and Peter Otten's) to
my initial posting, I think I am getting the picture that there is a
conscious decision to keep the use of __init__ the way it is and just make
people learn it and learn it early enough. That's a valid approach and I'll
accept it.
Snip

No one, including you, has given me a reason WHY __init__ is implemented
this way. I am not bashing you for that, I would just still like to hear
that 'WHY'. I'm sure that this implementation has some advantages. But,

Explicit is better than implicit

import this

Dan
PS: Does my last name attract the wrong kind of attention from people in
this newsgroup? It's really my name, it's not fake, BTW.

My nick littlejohn is the direct translation of my last name and I'm
far from big. So what ? :=)

 

[Shalabh Chaturvedi]

After seeing a few replies (yours, Benjamin Niemann's and Peter Otten's) to
my initial posting, I think I am getting the picture that there is a
conscious decision to keep the use of __init__ the way it is and just make
people learn it and learn it early enough.

I'd agree.

> That's a valid approach and I'll
accept it.

That's nice

No one, including you, has given me a reason WHY __init__ is implemented
this way. I am not bashing you for that, I would just still like to hear
that 'WHY'. I'm sure that this implementation has some advantages. But,
coming from a C++ and Java background, where parent default constructors are
automatically invoked (well, not always, and that is something that users
have to learn too), I find that that approach has some clear advantages.

'Why' is a very subjective question. People from different backgrounds
may accept very different answers for why since it may always be in a
specific context (why this instead of *that*?). The following is my idea
of why:

In Python you either have an __init__ or you don't. There is no 'default
constructor' - or if there is, it does nothing. Since attributes can be
dynamically added to an instance (not just in __init__ but in any
method), it follows that the standard practice is to initialize instance
members in __init__ as it is always called before any other instance
method. Now that there is one way to do a thing, Python avoids the
introduction of another way (unless it is notably more productive). That
would lead to a whole set of questions about 'which way is better?
__init__ or the other way?'.

Btw, now that there are descriptors, you can create descriptors that
initialize instance members with default values when they are accessed.
However, unless there is great advantage in your specific case, it might
be just better to follow standard practice and make it easier for
everyone else reading your code.

Dan
PS: Does my last name attract the wrong kind of attention from people in
this newsgroup?

I don't think so.

 

[Dan Perl]

'Why' is a very subjective question. People from different backgrounds
may accept very different answers for why since it may always be in a
specific context (why this instead of *that*?). The following is my idea
of why:

In Python you either have an __init__ or you don't. There is no 'default
constructor' - or if there is, it does nothing. Since attributes can be
dynamically added to an instance (not just in __init__ but in any
method), it follows that the standard practice is to initialize instance
members in __init__ as it is always called before any other instance
method. Now that there is one way to do a thing, Python avoids the
introduction of another way (unless it is notably more productive). That
would lead to a whole set of questions about 'which way is better?
__init__ or the other way?'.

It is a subjective issue indeed. But I'll restrict my question to a simpler
question. By analogy to C++ and Java, why is __init__ not implemented to
automatically invoke the __init__(self) of the parent classes? I'm
beginning to see a reason here, which is that C++ and Java allow you to
overload the constructor and have a default constructor at the same time as
another constructor with arguments. C++ and Java then force you to create a
default constructor for a parent class, even if it's empty, if you create a
non-default constructor for it and you subclass the parent class.

In contrast, Python does not allow method overloading and I'm OK with that.
It's a simplification that allows using Python in ways that are not possible
with C++ and Java. So Python cannot force you to have a 'default' __init__
and allow you to have 'non-default '__init__'s at the same time. I am
answering my own question here, but that's OK and it would not have happened
without this discussion.

Btw, now that there are descriptors, you can create descriptors that
initialize instance members with default values when they are accessed.
However, unless there is great advantage in your specific case, it might
be just better to follow standard practice and make it easier for
everyone else reading your code.

I'll have to look more into descriptors, I haven't used them yet and I see
there are many discussions on the topic in this newsgroup. I found your
"Python Attributes and Methods" page on cafepy.com and I'll take a look
there. Thanks!

Dan

 

[Shalabh Chaturvedi]

It is a subjective issue indeed. But I'll restrict my question to a simpler
question. By analogy to C++ and Java, why is __init__ not implemented to
automatically invoke the __init__(self) of the parent classes? I'm
beginning to see a reason here, which is that C++ and Java allow you to
overload the constructor and have a default constructor at the same time as
another constructor with arguments. C++ and Java then force you to create a
default constructor for a parent class, even if it's empty, if you create a
non-default constructor for it and you subclass the parent class.

In contrast, Python does not allow method overloading and I'm OK with that.
It's a simplification that allows using Python in ways that are not possible
with C++ and Java. So Python cannot force you to have a 'default' __init__
and allow you to have 'non-default '__init__'s at the same time. I am
answering my own question here, but that's OK and it would not have happened
without this discussion.

Yes, if the parent class __init__ accepts parameters, how can Python
call it automatically? Should it be automatic in some cases and
non-automatic in others? (Answer: no, it's too ambiguous). Also what if
there are multiple parent classes? As the Quixote folks put it, in the
face of ambiguity, refuse the temptation to guess. (Btw, Python's
solution for multiple parent classes completely knocks the socks off C++
[1])

Cheers,
Shalabh

[1] http://www.python.org/2.2.3/descrintro.html#cooperation

 

[Aahz]

In Python you either have an __init__ or you don't. There is no 'default
constructor' - or if there is, it does nothing.

Actually, that's not quite true. Python does indeed have a default
constructor that returns the object. This isn't particularly visible
from classic classes because there's no way to get access to a
constructor. New-style classes make the constructor available through
__new__(). __init__() is an initializer.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"To me vi is Zen. To use vi is to practice zen. Every command is a
koan. Profound to the user, unintelligible to the uninitiated. You
discover truth everytime you use it." (e-mail address removed)

 

[Alex Martelli]

I will clarify something that was probably not clear enough in my initial
posting. I am interested in instance attributes, but I am using the class
attributes to provide a default, "null", value for the instance attributes.

Quite a reasonable approach for immutables, because you don't have to
worry about how to copy them -- only about rebinding, which only happens
on the instance anyway. Not feasible for mutables, obviously, because
you WOULD have to worry about how to copy them -- shallow or deep, for
example?

awkward, I would rather call it unsafe. If I were a third-party library
vendor, I would not be comfortable providing a mixin class that forces users
to either invoke the parent's __init__ in their constructors or to

You're out of luck, then -- most well-designed classes, mixin or not, do
need to have their __init__ called in order to set a consistent state on
a new instance which will allow that instance to fulfill its
responsibilities. Just copying (shallowly or deeply...?) some class
attributes ain't gonna cut it.

I think this points to a weakness in Python as an OO language. I'm not

I think that Python's having as few "hidden black magic thing that
happens when you're not looking" aspects is a strength, not a weakness.

trying to bash it, but my understanding is that Python is still evolving in
that respect and I think that this should be improved in the future. Then,

Python's OO is still settling down after the huge changes in the passage
from 2.1 to 2.2 and it's unlikely that any change to come will include
making the calls to superclasses' __init__ methods somehow magical and
automatic rather than explicit -- I can't speak with certainty, only
Guido could, but that is my best guess.

However a custom metaclass that does automagically call some
initializers is surely a possibility, since it's a rather modest
investment of time to develop one, if anybody with a good understanding
of Python OO shares your opinion. Calling initializer methods appear
self-obviously (to me) to be head and shoulders above your idea about
copying (deeply or shallowly?) some class attributes into instances, of
course. And I would not call the automagically called initializers
__init__ to avoid conflicting with __init__'s very deliberately
NON-automagical behavior -- maybe something like __preinit__ or
__postinit__ depending on when you want these black magic methods
called, or __autoinit__ if you want to keep that aspect undefined, &c.

on the other hand, maybe I'm the only one feeling this way and there's not
going to be a change.

It's likely that others share your preferences, but if they share them
strongly enough to want to actually do something about them I think
they're more likely to be considering other languages, more suitable to
their overall philosophy -- it's rare, though not unheard of, for
somebody to stick with a language on which they differ on fundamental
points of design philosophy, and strive to change that language to their
own philosophy, when there are so many alternatives with different
philosophies that one might switch to instead. Similarly, it would be
unusual, though far from impossible, for somebody to both have a
complete and perfect grasp of Python's object model (so they'd be the
kind of person who could implement a custom metaclass for your purposes
with comparatively little effort) and at the same time loathe
fundamental aspects of that object model enough to want to strive to
change them. Nevertheless, these are only 'trends', not certainties.


Alex

 

[Alex Martelli]

My mistake. I shouldn't have made that statement. I don't really know
which case happens more often. Having said that, that is probably a mistake
that many beginners make. Okay, I have no real way of knowing that either,
but I think it's a good assumption.

I may not have enough experience with beginners, I guess -- my wife,
Anna, does have it, and I hope she sees this tidbit and shares that
experience with us. In my limited experience with beginners, in a huge
variety of programming languages and other technologies, they make every
possible mistake and many that are patently impossible too; and if you
try to design your technology to be foolproof, you're wasting your time,
because fools are _extremely_ ingenious. I prefer simplicity and
transparency over desperate attempts to safe fools from their
foolishness -- the smaller, simpler and more transparent a language or
other technology is, the earlier beginners cease to be beginners and
stop making most of their earlier, incredible mistakes.

I was using the class attribute as a default, "null", value for the instance
attributes. That works just fine for immutable attributes and it allows me
not to implement an __init__.

Yes, because immutable attributes cannot be mutated -- only by rebinding
that name will anything change, and name rebinding never affects the
object that name previously referred to. But mutable attributes can be
mutated, indeed that's the whole point of having them, rather than just
having their names rebound to refer to other objects. I know you want
some kind of automatic copy of (some?) class attributes to instance
attributes, but I think you just haven't thought clearly enough about
the issues involved, even something as trivial as whether the copy
should be shallow or deep -- the difference will be enormous.

I'm leaving the rest of your reply out, but thanks for the suggestion of a
metaclass, I'll look into it.

You're welcome.

After seeing a few replies (yours, Benjamin Niemann's and Peter Otten's) to
my initial posting, I think I am getting the picture that there is a
conscious decision to keep the use of __init__ the way it is and just make
people learn it and learn it early enough. That's a valid approach and I'll
accept it.
OK.

I can also understand your frustration with people who are new to Python,
like me, coming from a background in other OO languages, who are not yet
comfortable with "the Python way" and feel that there is a "better way" and
who suggest changing Python. But you also have to be reallistic and accept
that there will always be people coming to Python from other languages and
that the adjustment is rather difficult in some areas. You may just say

Sure! If they devoted their energy to understanding things in depth and
complete detail, rather than campaigning to change something because
they don't understand that something well enough yet, their adjustment
would be less difficult and we'd all be happier, of course. But human
nature will always make some people perceive that requiring THEM, their
precious and unique SELVES!, to change (their habits, their minds, etc)
is absurd, while it's obviously all the rest of the universe that has to
change to accomodate their tastes and preferences (that may be a
disaster for millions of existing users, but hey, none of those users is
THEM, so they're obviously not very important). They don't generally
phrase things that way (too politically incorrect) and more often than
not they rationalize even to themselves the changes as being good for
mankind, but, really, they're generally just sticking up for their
existing attachments, habits, and worldviews. I'm quite prone to doing
it myself unless I keep up a substantial guard against my own laziness
-- so I try to avoid suggesting any changes in something complex, if I
possibly can avoid that, until and unless I convince myself that I have
a thorough, complete understanding of that something, of the reason for
being and consequences of what I'm thinking of changing, and of the
impact and consequences of what I'd like to change it into.

'Tough!', or you may help them to make that adjustment (I think you are
doing that), or you may even accommodate them (you should at least consider
that).

It frightens me to consider that Python might one day get designs
decisions made, not because GvR sees them as useful in themselves and
consistent with Python's overall workings, but to "accomodate" people
who are not (yet) familiar with Python. Fortunately I can generally
dispell the fear and calm down, but please don't ask me to consider such
horrid scenarios, they can do nothing but frighten me uselessly.

No one, including you, has given me a reason WHY __init__ is implemented
this way. I am not bashing you for that, I would just still like to hear
that 'WHY'. I'm sure that this implementation has some advantages. But,
coming from a C++ and Java background, where parent default constructors are
automatically invoked (well, not always, and that is something that users
have to learn too), I find that that approach has some clear advantages.

In this like in many other details, Python chooses simplicity and
clarity against automatic, black-magic, "behind the scenes" approaches.
"Explicit is better than implicit" is one part of the Zen of Python that
speaks to this -- at a python interactive prompt do
import this
to read it all. The automatic invocation of default constructors when
they exist, what you have to do instead to get different constructors
for parent classes, etc etc, are all complications. When classes are
designed to execute responsibilities it's not unusual that they can't
really have a useful no-arguments constructor -- and when they don't
have such a constructor, C++'s and Java's rules are nothing BUT overhead
and conceptual complication. In C++ one often works around the burden
of contraints on constructor calls via "two-phase constructors" -- a
default constructor that does not really leave the instance in a usable
state, just to bypass the darn rules you find "have some clear
advantages", and then a normal member function that really does the job
of initialization and has the enormous advantage that YOU decide when
and with what arguments to call it, without all the darn rules in the
way, explicitly. Well, in Python the constructor, __init__, is under
you control in exactly this way -- less complication, more simplicity,
more transparency, fewer rules to learn, and far fewer instances of the
"two-phase constructor" pattern (not zero, mind you -- just 99% fewer).

Those are my 2 cents.

Dan
PS: Does my last name attract the wrong kind of attention from people in
this newsgroup? It's really my name, it's not fake, BTW.

I don't think you need to worry about this!-)


Alex

 

[Dan Perl]

............

In this like in many other details, Python chooses simplicity and
clarity against automatic, black-magic, "behind the scenes" approaches.
"Explicit is better than implicit" is one part of the Zen of Python that
speaks to this -- at a python interactive prompt do
import this
to read it all. The automatic invocation of default constructors when
they exist, what you have to do instead to get different constructors
for parent classes, etc etc, are all complications. When classes are
designed to execute responsibilities it's not unusual that they can't
really have a useful no-arguments constructor -- and when they don't
have such a constructor, C++'s and Java's rules are nothing BUT overhead
and conceptual complication. In C++ one often works around the burden
of contraints on constructor calls via "two-phase constructors" -- a
default constructor that does not really leave the instance in a usable
state, just to bypass the darn rules you find "have some clear
advantages", and then a normal member function that really does the job
of initialization and has the enormous advantage that YOU decide when
and with what arguments to call it, without all the darn rules in the
way, explicitly. Well, in Python the constructor, __init__, is under
you control in exactly this way -- less complication, more simplicity,
more transparency, fewer rules to learn, and far fewer instances of the
"two-phase constructor" pattern (not zero, mind you -- just 99% fewer).

This is the kind of answer I was hoping for. Actually, I was prepared to
see a suggestion for solving the problem in a different way and there were a
couple of them, even if the people suggesting them were themselves advising
to prefer the original solution.

When learning something, I need to understand the 'WHY' behind a design
decision like that, when clearly there are alternatives. I will not be
satisfied with an explanation of "that's the Python way and you just have to
learn it". The explanation of choosing "explicit" instead of "implicit"
makes sense and helps me in better understanding Python. I have to mention
though that I do not accept "explicit is better than implicit" as an
absolute truth. [That's like when someone told me once with pride: "I
always design bottom-up."] Applying a principle like that CONSISTENTLY in
the design of a language can make a very good language. On the other hand,
I can imagine there can be very good languages, with their own uses and
advantages, if the reverse principle were consistently used.

You may see in another message I posted in this thread that I ended up
giving an answer to my own question (the 'WHY' question). Here is my reason
for that 'WHY'. Python does not have method overloading (and I am ok with
that, because it comes with the dynamic typing), so you cannot have a
default constructor and a non-default one at the same time. C++ and Java
have overloading and then can also mandate a default constructor for a
parent class, even if it's empty because you actually use only a non-default
constructor for that class. Python cannot request that you also implement a
default __init__ when you need a non-default one. There would be the
possibility of adding another special method, but there's already __init__
and __new__, so that would be too confusing.

I don't know whether that was an actual reason for the design even if it is
definitely an obstacle. But the principle of "explicit instead of implicit"
(I'll think of it that way) will help me to understand also other aspects of
Python. And thank you for that.

Dan

 

[Alex Martelli]

...
This is the kind of answer I was hoping for. Actually, I was prepared to
see a suggestion for solving the problem in a different way and there were a
couple of them, even if the people suggesting them were themselves advising
to prefer the original solution.

Glad I was helpful (in both ways).

When learning something, I need to understand the 'WHY' behind a design
decision like that, when clearly there are alternatives. I will not be

I absolutely agree: understanding design rationale is crucial to using
complicated systems optimally.

satisfied with an explanation of "that's the Python way and you just have to
learn it". The explanation of choosing "explicit" instead of "implicit"
makes sense and helps me in better understanding Python. I have to mention

I'm glad to hear this.

though that I do not accept "explicit is better than implicit" as an
absolute truth. [That's like when someone told me once with pride: "I
always design bottom-up."] Applying a principle like that CONSISTENTLY in
the design of a language can make a very good language. On the other hand,
I can imagine there can be very good languages, with their own uses and
advantages, if the reverse principle were consistently used.

Perhaps, but from my experience with languages which consistently try to
double-guess programmer intent and "do what's right" implicitly based on
context, it doesn't work -- I'm thinking of PL/I, C++, Perl. They're
all big languages, so perfect consistency is impossible, of course. But
even small languages grow, if they're successful. Do you have specific
examples in mind of very good languages based on "implicit is better
than explicit"?

You may see in another message I posted in this thread that I ended up
giving an answer to my own question (the 'WHY' question). Here is my reason
for that 'WHY'. Python does not have method overloading (and I am ok with
that, because it comes with the dynamic typing), so you cannot have a
default constructor and a non-default one at the same time. C++ and Java

Not with the same name. You can have all alternative constructors you
want if you give them different names -- that's a popular use of
classmethod (in Python as well as in Smalltalk, though there are
differences between the two, of course). Alternative names are more
powerful than any overloading based on type could possibly be.
Consider:
{(1, 2): None, (5, 6): None, (3, 4): None}

Overloading based on type would be powerless here, but classmethod
fromkeys of the dict class is perfectly usable to get a different
alternative constructor.

have overloading and then can also mandate a default constructor for a
parent class, even if it's empty because you actually use only a non-default
constructor for that class. Python cannot request that you also implement a
default __init__ when you need a non-default one. There would be the
possibility of adding another special method, but there's already __init__
and __new__, so that would be too confusing.

Java and C++ _could_ mandate a default ctor, but they don't -- it would
be a very bad design decision for them to do so, and their designers
aren't THAT bad. It's perfectly possible to have a class that just
CANNOT be constructed without some specific arguments.

I'm not sure why different names would be confusing any more than
different signatures for the same name. Indeed in practice C++ builds
names from signatures (with some added complications regarding hiding of
names apart from signatures). So I don't think this rationale is a good
one for Python's design choices regarding object creation &
initialization.

I don't know whether that was an actual reason for the design even if it is
definitely an obstacle. But the principle of "explicit instead of implicit"
(I'll think of it that way) will help me to understand also other aspects of
Python. And thank you for that.

You're welcome. But I think the impossibility of having several methods
with the same name, differentiated only by signature, is not the root of
things in this particular case.


Alex

 

[Dan Perl]

...

though that I do not accept "explicit is better than implicit" as an
absolute truth. [That's like when someone told me once with pride: "I
always design bottom-up."] Applying a principle like that CONSISTENTLY in
the design of a language can make a very good language. On the other hand,
I can imagine there can be very good languages, with their own uses and
advantages, if the reverse principle were consistently used.

Perhaps, but from my experience with languages which consistently try to
double-guess programmer intent and "do what's right" implicitly based on
context, it doesn't work -- I'm thinking of PL/I, C++, Perl. They're
all big languages, so perfect consistency is impossible, of course. But
even small languages grow, if they're successful. Do you have specific
examples in mind of very good languages based on "implicit is better
than explicit"?

No, I do not have an example, but I still stand by my comment. Maybe it's
just impossible to use the principle "implicit is better than explicit"
consistently because you need at least something explicit.

Java

Not with the same name. You can have all alternative constructors you
want if you give them different names -- that's a popular use of
classmethod (in Python as well as in Smalltalk, though there are
differences between the two, of course). Alternative names are more
powerful than any overloading based on type could possibly be.
Consider:

We may have a disagreement in terminology, but I was using "overloading" in
the C++/Java sense, which means strictly samename, different signatures.

Java and C++ _could_ mandate a default ctor, but they don't -- it would
be a very bad design decision for them to do so, and their designers
aren't THAT bad. It's perfectly possible to have a class that just
CANNOT be constructed without some specific arguments.

Try this C++ code:
#include <iostream>
class Class1 {
public:
Class1(int arg )
{
std::cout << "Class1 Constructor" << std::endl;
}
};
class Class2 : public Class1 {
public:
Class2(int arg)
{
std::cout << "Class2 Constructor" << std::endl;
}
};
int main(int argc, char **argv)
{
Class2 c2 = Class2(9);
}

The compiler (I tried gcc) will give an error that the Class1::CLass1( )
constructor is missing. Without the non-default Class1 constructor, the
compiler would have created a default constructor, IMPLICITLY. Note that
even C++ chooses not to create the implicit, default, constructor anymore if
you have a non-default constructor. You have to do it explicitly, even if
it's an empty one.

Anyway, in C++, if I write a library with a class like Class1, I can create
a default constructor to initialize all the members with default values if
there is no need for non-default values. C++ gives me that possibility. A
user of my library (who will never know me) can use this library and does
not need to know anything about the Class1 constructor. This is consistent
with the principle of encapsulation, so subclasses don't need to know
anything about how the superclass is implemented.

Not in Python. A user of my library has to invoke the parent's class
__init__ in their own __init__. What happens if, in a future release, I get
rid of the __init__ in the parent class? Or the other way around. An early
release does not have a parent __init__, the users don't invoke it because
they can't, and then, in a future release, I add the parent __init__ because
I added some attributes. It breaks all the users' code. This is poor
encapsulation.

I think this IS a case where "implicit is better than explicit".

Dan

 

[Jorge Godoy]

Not in Python. A user of my library has to invoke the parent's class
__init__ in their own __init__. What happens if, in a future release, I get
rid of the __init__ in the parent class? Or the other way around. An early
release does not have a parent __init__, the users don't invoke it because
they can't, and then, in a future release, I add the parent __init__ because
I added some attributes. It breaks all the users' code. This is poor
encapsulation.

I'm getting in the middle of the discussion so forgive-me if this example
doesn't apply here.

.... def test(self):
.... print "Just a test"
........ def __init__(self):
.... test.__init__(self)
.... def showit(self):
.... self.test()
....
There was no "__init__" explicitly defined in 'test'.


Be seeing you,