Want - but cannot get - a nested class to inherit from outer class

[DBak]

I want - but cannot get - a nested class to inherit from an outer
class. (I searched the newsgroup and the web for this, couldn't find
anything - if I missed an answer to this please let me know!)

I would like to build a class for a data structure such that nodes of
the data structure - of interest only to the data structure
implementation itself and not to the consumer - are instances of one
of two class types. I thought to encapsulate the nodes' classes like
this:

class Tree(object):
....class _MT(Tree):
.......def isEmpty(self): return True
.......def insert(self, X): return Tree._Node(X)
....class _Node(Tree):
.......def isEmpty(self): return False
.......def insert(self, X): return _Node(X, self, Tree._MT())
....def merge(self, T):
.......def __init__(): return _MT()
.......<code for merging tree T into self>

In other words, some methods would be implemented on instances'
classes (like isEmpty and insert) and some on the outer class (like
merge). Users of the data structure never need to know about the
nodes, much less the nodes' classes, so I wanted to encapsulate them

However I can't do this, because, of course, the name Tree isn't
available at the time that the classes _MT and _Node are defined, so
_MT and _Node can't inherit from Tree.

What is the Pythonic thing I should be doing instead?

(Easy answer: Put this code in a module, exposing only a factory
function. I could do that, but wanted to know if I could encapsulate
it as described so I could actually put several similar data
structures into one module.)

Thanks! -- David

 

[DBak]

Sorry - code for the class should read:

class Tree(object):
....class _MT(Tree):
.......def isEmpty(self): return True
.......def insert(self, X): return Tree._Node(X)
....class _Node(Tree):
.......def isEmpty(self): return False
.......def insert(self, X): return _Node(X, self, Tree._MT())
....def __init__(): return _MT()
....def merge(self, T):
.......<code for merging tree T into self>

 

[Jeff]

Use metaclasses?

 

[Chris Mellon]

I want - but cannot get - a nested class to inherit from an outer
class. (I searched the newsgroup and the web for this, couldn't find
anything - if I missed an answer to this please let me know!)

I would like to build a class for a data structure such that nodes of
the data structure - of interest only to the data structure
implementation itself and not to the consumer - are instances of one
of two class types. I thought to encapsulate the nodes' classes like
this:

class Tree(object):
...class _MT(Tree):
......def isEmpty(self): return True
......def insert(self, X): return Tree._Node(X)
...class _Node(Tree):
......def isEmpty(self): return False
......def insert(self, X): return _Node(X, self, Tree._MT())
...def merge(self, T):
......def __init__(): return _MT()
......<code for merging tree T into self>

In other words, some methods would be implemented on instances'
classes (like isEmpty and insert) and some on the outer class (like
merge). Users of the data structure never need to know about the
nodes, much less the nodes' classes, so I wanted to encapsulate them

However I can't do this, because, of course, the name Tree isn't
available at the time that the classes _MT and _Node are defined, so
_MT and _Node can't inherit from Tree.

Not only is the name not defined, the class doesn't even exist yet.

What is the Pythonic thing I should be doing instead?

Don't nest them. The single underscore is all you need to keep any
from using them (don't forget that even if your method worked, they'd
be perfectly visible as attributes of the Tree class, or as the types
of returned values). Worrying too much about visibility is a waste of
time in Python.

(Easy answer: Put this code in a module, exposing only a factory
function. I could do that, but wanted to know if I could encapsulate
it as described so I could actually put several similar data
structures into one module.)

There's no reason to use a factory function. If you do put them in a
module, you can use __all__ to document your exports. As with all
visibility issues in Python, this is advisory only - the only code it
affects is the symbols that are exported by "from module import *".

 

[castironpi]

[a certain way]

Not only is the name not defined, the class doesn't even exist yet.

 What is the Pythonic thing I should be doing instead?

Don't nest them. [ Too much anything is waste.]

You expose a problem with the way I and some others write code.

Humans have enormous "mental stacks"--- the stacks the contexts the
speakers speak in push things they're hearing on to. When writing for
computers, even those that speak Python, you have to say one thing at
a time, and let it get its mind around it. They're not that deep,
they're just at a deep depth. (The depth the ocean the submarine's in
has, not the depth the submarine's -at-.) Marvelous memories, though.

Your robotic snake can also do symbolic calculus, radio to clusters,
and generate arbitrary auditory waveforms. But in a snake, the snake
functions are primary. What's a multiplexer?

 

[DBak]

Not only is the name not defined, the class doesn't even exist yet.

Yes, but, well - it certainly isn't usable yet, but some object (that
will be the class when it is finished) is being built (its __dict__ is
being populated, etc.) - so there's an object pointer available inside
the interpreter that could be put somewhere. But this is pedantic -
you're right, the class really isn't available until after the class
statement.

Don't nest them. The single underscore is all you need to keep any
from using them (don't forget that even if your method worked, they'd
be perfectly visible as attributes of the Tree class, or as the types
of returned values). Worrying too much about visibility is a waste of
time in Python.

Yes, I knew it wouldn't stop the determined coder from finding and
using them.

However, I did actually have a reason for wanted them nested: I
wanted to put more than one similar data structure in the same source
file - and several of them would have "Empty" nodes that would behave
differently. If I used nested classes all those classes could be
named Empty. If flattened to module level then I'd have to invent
different names for each one, like, AVL_Empty, RedBlack_Empty, and it
wouldn't seem as "nice" (for some definition of "nice" that I have in
my head).

There's no reason to use a factory function. If you do put them in a
module, you can use __all__ to document your exports. As with all
visibility issues in Python, this is advisory only - the only code it
affects is the symbols that are exported by "from module import *".

OK.

Thanks! -- David

 

[castironpi]

Yes, but, well - it certainly isn't usable yet, but some object (that
will be the class when it is finished) is being built (its __dict__ is
being populated, etc.) - so there's an object pointer available inside
the interpreter that could be put somewhere.  But this is pedantic -
you're right, the class really isn't available until after the class
statement.

There is no obvious solution-- What do you mean? If there are any at
all, there is significant competition without clear winners.

dict dictA:
membA= 0
membB= 0

dict dictB:
membC= 0

But, if you try to nest them, do you want the rest of the 'dict' at
its outer level evaluated (that's your 'here is the crux'), or only
that point so far?

dict dictO:
membA= 0
dict membB:
membC= 0
membD= membE
membE= 0

So, you can't refer to it at all. Especially if membE is defined in
outer scope.

class Tree(object):
...class _MT(Tree):

Can you use a metaclass--- postponedinstantiating (since a class is an
instance of A PARTICULAR THING.) <throat clear> And one of us could
even lobby the wigs to put a metaclass in the standard library.
What's the first to go? <throat.clear()>. If you could have any one
metaclass in it, what would it be?

The additions with the biggest pull are the ones that can make money
really easily for Python speakers. Yes, money.

How long is the workaround? In this case, if you want _MT in Tree (or
the analogous pair, without loss of generality), you can still get it
done (you know where to find me), thanks to Python. You just
sacrifice how closely the final product (code), resembles the concept.

Yes, that idea would make class definitions more explanitorily
powerful-- have more acute explanations. But you need to assign
meaning to a syntax element, which is done over time. If Python is a
hybrid between C and ML, what is the hybrid between Python and XML?

 

[Steven D'Aprano]

Humans have enormous "mental stacks"--- the stacks the contexts the
speakers speak in push things they're hearing on to.

This is not true.

Human beings have extremely shallow mental stacks, limited by short-term
memory. Most people are capable of keeping seven, plus or minus two,
items in short term memory at once. Any more than that and they are
subject to fading away or being over-written.

Grammatically, it doesn't take many levels to confuse most people, and
even the best and brightest can't keep track of hundreds of levels, let
alone "enormous" numbers. At least, not unless you consider five to be
enormous.

Steven Pinker's "The Language Instinct" has some excellent examples of
such "top heavy" phases, such as:

He gave the girl that he met in New York while visiting
his parents for ten days around Christmas and New
Year's the candy.

She saw the matter that had caused her so much anxiety
in former years when she was employed as an efficiency
expert by the company through.


Fortunately the English grammar, which is very strict about word order
(Pinker describes it as "tyrannical") allows alternate forms that are
easier for our shallow stacks to deal with:

He gave the candy to the girl that he met in New York
while visiting his parents for ten days around
Christmas and New Year's.

She saw the matter through that had caused her so much
anxiety in former years when she was employed as an
efficiency expert by the company through.


Pinker also describes situations where the words can be grouped into
phrases as you go, and so are easy to comprehend:

Remarkable is the rapidity of the motion of the wing
of the hummingbird.

and a counter-example of a perfectly grammatical sentence, with only
THREE levels, which is all but unintelligible:

The rapidity that the motion that the wing that the
hummingbird has has has is remarkable.

These nested "onion sentences" are exceedingly difficult for the human
brain to parse. In fact, I'm betting that at least 90% of people will, on
first reading, will question whether it could possibly be grammatically
correct.

 

[castironpi]

This is not true.

Oh yeah. (See below.)

Human beings have extremely shallow mental stacks, limited by short-term
memory. Most people are capable of keeping seven, plus or minus two,
items in short term memory at once. Any more than that and they are
subject to fading away or being over-written.

    efficiency expert by the company<strike> through</strike>.

    Remarkable is the rapidity of the motion of the wing
    of the hummingbird.

Oh, you mean hummingbird wing motion rapidity remarkability! How -
could- I!

'Stack' may have been the wrong word.

I probably meant 'intermodal recognition': the "Ah yes, I know it"
response.

But whether that's related to our tool-bearer capacity, isn't
something you can test all that easily. What animals encounter
something they had a use for once, and forget what?

The breadth of objects in, say, a leopard's representation table*
(working term), doesn't include chisels and footballs. Nothing is its
favorite football, and a football isn't its favorite anything. What
are its favorites?

* Proposition table is clearly wrong. But given the human ability to
touch an object, then open eyes, and identify which it touched might
suggest 'intermodal recognition table', 'resemblance table', or 'value
table'. How about resemblance-value? It at least would explain our
susceptibility and reaction to greedy traps and shock: it's a the
shock and greedy trap susceptibility and reaction of ours
explanation. That is, how close something you like is to something
you like..... and how much you like it. How much you like and how
close something is... <ahem>. Monotonically decreasing bivariate,
delta> 0 => value( object, prior value+ delta )< value( object, prior
value ) & value( object+ delta, prior value )< value( object, prior
value ).


... which, by the way, should answer Archimedes' riddle: can you step
in the same river? By degree. You're downstream and know it.

Nota bene, how fast we can recall a song we even haven't heard in
years we hear a snippet of, and not even from the overture. Given a
'symbol' that comes over a channel, in baud, our hash function maps it
to a value close to previous hashings of it--- we merely search nearby
memories and it's recognized. Odd!

 

[DBak]

There is no obvious solution-- What do you mean?  If there are any at
all, there is significant competition without clear winners.

dict dictA:
   membA= 0
   membB= 0

dict dictB:
   membC= 0

But, if you try to nest them, do you want the rest of the 'dict' at
its outer level evaluated (that's your 'here is the crux'), or only
that point so far?

dict dictO:
  membA= 0
  dict membB:
    membC= 0
    membD= membE
  membE= 0

So, you can't refer to it at all.  Especially if membE is defined in
outer scope.

Thanks for your answer. To the extent I understand it: There is a
difference between the class statements I was trying to nest, with the
inner inheriting from the outer, and your dict example. The main
thing is that in the class example - when the inner class is being
built (i.e., inside its class statement's block) there is no need (as
I understand it) for the parent class to be functional at all WHILE I
AM DEFINING METHODS on the inner class. Sure, if the inner class had
code to be executed immediately, such as statements setting class
attributes on the inner class, and that code used names such that
attribute lookup needed to be done, then that might or might not work
- it would depend on where the names were defined in the outer class
relative to the placement of the inner class statement - exactly like
the fact that the order of definition matters when executing code when
defining a module: functions defined in a module can, in their body,
name functions not yet defined, but assignment statements to module
attributes cannot (they get a run time error).

But in the case I'm talking about I just want to define methods on the
inner class, using names such that when the method is eventually
called on an instance of the inner class the attribute lookup will
proceed with the outer class being the inner class' parent. Creating
instances of the inner class won't happen until after the inner class
and the outer class are both fully created (and assigned to their
names) so any name lookup using inheritance won't happen until both
class objects are fully created, so ... if you could do it ... it
would work fine.

Anyway, I know it can't be done the way I wanted it - the attribute
with the outer class' name hasn't been assigned yet when I need to
reference it in the inner class' class statement - so I was just
looking to see what the preferred alternative was. Based on the
discussion so far it seems I should just forget about using nested
classes and flatten everything to the module level, using the __all__
attribute to make it clear to the user of the data structure what
pieces of the module he should actually be using.

-- David

 

[castironpi]

Thanks for your answer.  To the extent I understand it:  There is a
difference between the class statements I was trying to nest, with the
inner inheriting from the outer, and your dict example.  The main
thing is that in the class example - when the inner class is being
built (i.e., inside its class statement's block) there is no need (as
I understand it) for the parent class to be functional at all WHILE I
AM DEFINING METHODS on the inner class.  Sure, if the inner class had

An interesting observation.

But in the case I'm talking about I just want to define methods on the
inner class, using names such that when the method is eventually
called on an instance of the inner class the attribute lookup will
proceed with the outer class being the inner class' parent.  Creating

There are two possibilities I'm thinking of. In both, cases you go
back after Tree is finished executing, and inform _Node and _MT of
their parentage. I haven't tested either.

The first is, define a metaclass that allows __mro__ (method
resolution order) to be written to. But does the look-up get
performed on the one you define?

The second is, cache _Node and _MT during creation, and go back and
reinstantiate them.

class Tree(object):
...class _MT( metaclass= placeholder ):
......def isEmpty(self): return True
......def insert(self, X): return Tree._Node(X)
...class _Node( metaclass= placeholder ):
......def isEmpty(self): return False
......def insert(self, X): return _Node(X, self, Tree._MT())
...def __init__(): return _MT()
...def merge(self, T):
......<code for merging tree T into self>
Tree._MT.complete( Tree )
Tree._Node.complete( Tree )

And similarly,

class Tree(object):
@placeholder
...class _MT():
......def isEmpty(self): return True
......def insert(self, X): return Tree._Node(X)
@placeholder
...class _Node( metaclass= placeholder( Tree ) ):
......def isEmpty(self): return False
......def insert(self, X): return _Node(X, self, Tree._MT())
...def __init__(): return _MT()
...def merge(self, T):
......<code for merging tree T into self>
Tree._MT.complete( Tree )
Tree._Node.complete( Tree )

Here the last lines might read:

Tree._MT= Tree._MT.complete( Tree )
Tree._Node= Tree._Node.complete( Tree )

But maybe not. Here's a third, actually:

class Tree( placeholder ): pass
class Tree(object, metaclass= placeholderreplace( Tree ) ):
...class _MT(Tree):
......def isEmpty(self): return True
......def insert(self, X): return Tree._Node(X)
...class _Node(Tree):
......def isEmpty(self): return False
......def insert(self, X): return _Node(X, self, Tree._MT())
...def __init__(): return _MT()
...def merge(self, T):
......<code for merging tree T into self>

If you're using Python -3.0, you might even be able to get away
without metaclasses on the first one, or decorators on the second one:

class Tree(object):
...class _MT():
......def isEmpty(self): return True
......def insert(self, X): return Tree._Node(X)
_MT= placeholder( _MT )
...class _Node( metaclass= placeholder( Tree ) ):
......def isEmpty(self): return False
......def insert(self, X): return _Node(X, self, Tree._MT())
_Node= placeholder( _Node )
...def __init__(): return _MT()
...def merge(self, T):
......<code for merging tree T into self>
Tree._MT.complete( Tree )
Tree._Node.complete( Tree )

And once again I'm still not sure.

 

[marek.rocki]

Everybody has already explained why this doesn't work and a possible
"solution" using metaclasses was suggested. I tried to implement it,
ended up with monkey-patching __bases__, which is certainly an
abomination (will it be possible in python 3.0?) but seems to do the
trick.

class _InheritFromOuterClass(object):
""" This class is used as a marker. """
pass

class MetaFixInheritance(type):
""" This metaclass, along with ordinary class creation, changes base
of
all the inner classes (marked as inheriting from
_InheritFromOuterClass)
to the class being created. """
def __new__(self, name, bases, members):
result = type(name, bases, members)
for member in members.itervalues():
if isinstance(member, type) and issubclass(member,
_InheritFromOuterClass):
member.__bases__ = result,
return result

class Outer(object):
""" Test an inner class inheriting from the outer class. Inheritance
is
monkey-patched upon creation of the outer class. """
__metaclass__ = MetaFixInheritance
class Inner(_InheritFromOuterClass):
pass
def foo(self):
return 'Outer.foo()'

inner = Outer.Inner()
print 'Inner.foo() is in fact', inner.foo()