What are super()'s semantics?

[Mike Krell]

I'm reading Alex Martelli's "Nutshell" second edition. In the section
called "Cooperative superclass method calling", he presents a diamond
inheritance hierachy:

class A(object):
def met(self):
print "A.met"
class B(A):
def met(self):
print "B.met"
A.met(self)
class C(A):
def met(self):
print "C.met"
A.met(self)
class D(B,C):
def met(self):
print "D.met"
B.met(self)
C.met(self)


D().met() # essentially "D B A C A"

Martelli says "In this code, when we call D().met(), A.met ends up being
called twice. How can we ensure that each ancestor's implementation of
the method is called once, and only once? The solution is to use
built-in type super. super(aclass, obj), which returns a special
superobject of object obj. When we look up an attribute (e.g., a method)
in this superobject, the lookup begins after class aclass in obj's
MRO. We can therefore rewrite the previous code as:

class A(object):
def met(self):
print "A.met"

class B(A):
def met(self):
print "B.met"
super(B, self).met()

class C(A):
def met(self):
print "C.met"
super(C, self).met()

class D(B,C):
def met(self):
print "D.met"
super(D, self).met()

D().met() # essentially "D B C A"


Now, D().met() results in exactly one call to each class's version of
met."

I see that this is true, but I am confused by the explanation (the bit
about truncated lookup in the class's MRO). In particular:

1. The super() call in D somehow invokes both parent class methods
instead of stopping when the method is resolved in B. This has
nothing to do with truncated lookup per se. Why isn't the output "D
B A"?

2. If I understand correctly, B's MRO is (B, A) and super(B, self) would
have an MRO of (A). Similarly for C. So it seems that by the above
explanation, A.met() would still be invoked twice (assuming both
B.met() and C.met() are invoked).

I guess I can just take it on faith that super() invokes everything once
and only once, but I'd rather understand how. Can someone point me to a
more complete description of super()'s semantics and how they work?

BTW, the official docs are even worse in this regard. AFAICT, they
essentially say that super() returns a superclass with no discussion of
diamond inheritance or any hint of how the semantics of super(B,
self).met() would be any different than those of A.met(self).

This seems like very important functionality to be documented in the
official docs so poorly.

Mike

 

[Michele Simionato]

BTW, the official docs are even worse in this regard. AFAICT, they
essentially say that super() returns a superclass with no discussion of
diamond inheritance or any hint of how the semantics of super(B,
self).met() would be any different than those of A.met(self).

This seems like very important functionality to be documented in the
official docs so poorly.

Well, you are right. I remember being fooled myself. 'super' does NOT
return a
superclass. Actually, there is no meaningful concept of superclass in a
multiple inheritance
world. Anyway, the MRO concept is documented here:

http://www.python.org/download/releases/2.3/mro/

(yes, it is not easy to find this link in python.org).

Michele Simionato

 

[Carl Banks]

class A(object):
def met(self):
print "A.met"

class B(A):
def met(self):
print "B.met"
super(B, self).met()

class C(A):
def met(self):
print "C.met"
super(C, self).met()

class D(B,C):
def met(self):
print "D.met"
super(D, self).met()

D().met() # essentially "D B C A" [snip]

2. If I understand correctly, B's MRO is (B, A) and super(B, self) would
have an MRO of (A).

This is the source of your misunderstanding.

Essentially, it's objects that have MROs, not classes. When you create
an object of class D, the MRO of that object is (D,B,C,A), and it
doesn't change, even when you're executing code defined in class B.
Thus, when self is of type D, super(B,self) does not have and MRO of
(A,), but (C,A). Therefore, super(B,self).__init__() invokes
C.__init__.


Carl Banks

 

[Maric Michaud]

Le lundi 04 septembre 2006 12:25, Mike Krell a écrit :

1. The super() call in D somehow invokes both parent class methods
   instead of stopping when the method is resolved in B. This has
   nothing to do with truncated lookup per se.  Why isn't the output "D
   B A"?

Yes but, super(B, B()) and super(B,D()) are not the same object like the code
below shows.

2. If I understand correctly, B's MRO is (B, A) and super(B, self) would
   have an MRO of (A).  Similarly for C.  So it seems that by the above
   explanation, A.met() would still be invoked twice (assuming both
   B.met() and C.met() are invoked).

super(class_, self), is an instance of super, so its class, "super", have a
mro of [<type 'super'>, <type 'object'>]....
"self" in each method call is the same and indeed its type's mro doesn't
change !


-- In [80]: class A(object) :
....: def sup(self) :
....: print 'A'
....:
....:

In [81]: class B(A) :
....: def sup(self) :
....: print 'B'
....: s(B, self).sup()
....:
....:

In [82]: class C(A) :
....: def sup(self) :
....: print 'C'
....: s(C, self).sup()
....:
....:

In [83]: class D(B,C) :
....: def sup(self) :
....: print 'D'
....: s(D, self).sup()
....:
....:

In [97]: class s(super) :
....: def __new__(*a) :
....: print a
....: return super.__new__(*a)
....:
....:

In [98]: D().sup()
D
(<class '__main__.s'>, <class '__main__.D'>, <__main__.D object at
0xa763186c>)
B
(<class '__main__.s'>, <class '__main__.B'>, <__main__.D object at
0xa763186c>) <--- instance is always the same !!
C
(<class '__main__.s'>, <class '__main__.C'>, <__main__.D object at
0xa763186c>) <--- instance is always the same !!
A

In [100]: super(B, D()).sup()
C
(<class '__main__.s'>, <class '__main__.C'>, <__main__.D object at
0xa763178c>)
A

This shows that C is called following the mro of type(D()) from class B to top
and not the mro of B.

All the magic is in the __getattribute__ of super which retrieve the
following class in the mro given its argument.


In [140]: class A(object) :
.....: def sup(self) : print 'a'
.....:
.....:

In [141]: class B(A) :
.....: def sup(self) : print 'b'
.....:
.....:

In [142]: class C(A) :
.....: def sup(self) : print 'c'
.....:
.....:

In [143]: class D(B,C) : pass
.....:


In [147]: super.__getattribute__(super(D,D()), 'sup')()
b

In [148]: super.__getattribute__(super(B,D()), 'sup')()
c

In [149]: super.__getattribute__(super(C,D()), 'sup')()
a

Hope this is clear.

_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Maric Michaud]

Le lundi 04 septembre 2006 13:48, Carl Banks a écrit :

Essentially, it's objects that have MROs, not classes.

Wrong, __mro__ is an attribute of types (subtypes of type) but like __class__
it is not available in the instances.
mro() is standard a method of type.

In [150]: A.mro()
Out[150]: [<class '__main__.A'>, <type 'object'>]

In [152]: A().mro()
---------------------------------------------------------------------------
exceptions.AttributeError Traceback (most recent
call last)

/home/maric/<ipython console>

AttributeError: 'A' object has no attribute 'mro'

In [154]: type.mro(A)
Out[154]: [<class '__main__.A'>, <type 'object'>]


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Carl Banks]

Le lundi 04 septembre 2006 13:48, Carl Banks a écrit :
Wrong, __mro__ is an attribute of types (subtypes of type) but like __class__
it is not available in the instances.
mro() is standard a method of type.

I agree that was misleading; I should have said something like, "Type
of an object never changes, therefore the MRO used for an object is
fixed."

BTW, __class__ is available to instances. (Were you thinking of
__bases__?)


Carl Banks

 

[Maric Michaud]

Le lundi 04 septembre 2006 22:29, Carl Banks a écrit :

BTW, __class__ is available to instances.  (Were you thinking of
__bases__?)

hmmm, I guess they're not the same, are they ?

but you're right, __bases__ and some others are class attributes not available
in instances, I wonder where is this documented and I'm not enough familiar
with python' source code to find this.

Also this create weird things, like a code posted on this list, which was
very confusing and looked to something like :

In [24]: class A(object) :
....: __class__ = list
....:
....:

In [25]: A.__class__
Out[25]: <type 'type'>

In [26]: A().__class__
Out[26]: <type 'list'>

In [27]: isinstance(A(), list) # ouch !
Out[27]: True

In [29]: type(A())
Out[29]: <class '__main__.A'>

In [30]: type(A()).mro()
Out[30]: [<class '__main__.A'>, <type 'object'>]


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Mike Krell]

[examples snipped]

Hope this is clear.

Yes, I get it now. In a prior section in "Nutshell", Alex M. goes over the
MRO as well as the __mro__ attribute. I remembered the fact that the
__mro__ attribute can be referenced only on classes. Somehow, this
confused me into thinking that the operative MRO for each call to super()
would be dictated by the class where the call was being made rather than by
the single instance of D as specified by the second parameter.

Mike

 

[Mike Krell]

Anyway, the MRO concept is documented here:

http://www.python.org/download/releases/2.3/mro/

A very edifying document. Indeed, in "Nutshell" Alex M. mentions your
paper at the end of his high-level explanation of the MRO. In my infinite
wisdom, I had chosen not to follow up by reading about the nitty-gritty
details

As I mentioned in another post, it wasn't my lack of understanding of the
MRO per se that tripped me up. I somehow managed to know that in Alex's
example, an instance of the D class would have an MRO of essentially (D, C,
B, A), and yet not realize that this was strangly similar to the output of
the example. D'oh!

Thanks,
Mike

 

[Steve Holden]

Le lundi 04 septembre 2006 22:29, Carl Banks a écrit :

BTW, __class__ is available to instances. (Were you thinking of
__bases__?)

hmmm, I guess they're not the same, are they ?

but you're right, __bases__ and some others are class attributes not available
in instances, I wonder where is this documented and I'm not enough familiar
with python' source code to find this.

Also this create weird things, like a code posted on this list, which was
very confusing and looked to something like :

In [24]: class A(object) :
....: __class__ = list
....:
....:

In [25]: A.__class__
Out[25]: <type 'type'>

In [26]: A().__class__
Out[26]: <type 'list'>

In [27]: isinstance(A(), list) # ouch !
Out[27]: True

In [29]: type(A())
Out[29]: <class '__main__.A'>

In [30]: type(A()).mro()
Out[30]: [<class '__main__.A'>, <type 'object'>]

You are beginning to appreciate the meaning of the phrase "Python is a
language for use by consenting adults". The general philosophy is to
provide a coherent and easily-used framework in the expectation that
users will not shoot themselves in the foot (too often).

regards
Steve