Order in metaclass

[Nicolas Fleury]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

 

[Jeremy Bowers]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

What order?

Yeah, they are listed in the source in a particular order, but there is
absolutely no difference between the six permutations of that order, in
theory or in fact, since the vars end up in a dict.

If you want a concept of "order", you're going to have do define one
yourself. What do you want to do, exactly?

I'm going to guess you're trying to get the vars out in order upon an
iteration, which may not be right but extends to other cases as well
fairly cleanly.

The easy solution is a list:

class MyClass(object):
classData = [4, 5, 6]
offsets = {'x': 0, 'y': 1, 'z':2}
def getVarMaker(var):
def getVar(self):
return self.classData[self.offsets[var]]
return getVar
def setVarMaker(var):
def setVar(self, value):
self.classData[self.offsets[var]] = value
return setVar
x = property(getVarMaker('x'), setVarMaker('x'))
y = property(getVarMaker('y'), setVarMaker('y'))
z = property(getVarMaker('z'), setVarMaker('z'))
def __iter__(self):
for value in self.classData:
yield value

A metaclass could take classData and offsets and make the access vars for
you.


--------
def getVarMaker(var):
def getVar(self):
return self.classData[self.offsets[var]]
return getVar

def setVarMaker(var):
def setVar(self, value):
self.classData[self.offsets[var]] = value
return setVar

class OrderedClassAttributes(type):
def __init__(cls, name, bases, dict):
super(OrderedClassAttributes, cls).__init__(name, bases, dict)
for name in cls.offsets:
setattr(cls, name, property(getVarMaker(name),
setVarMaker(name)))

class MyClass(object):
classData = [4, 5, 6]
offsets = {'x': 0, 'y': 1, 'z':2}
__metaclass__ = OrderedClassAttributes
def __iter__(self):
for value in self.classData:
yield value

---------
(Assuming the above is copy/pasted into Python[4, 33, 6]
---------

Lightly tested but working as I designed.

Is that what you wanted? If not, feel free to ask more. There are
certainly other options but without more data, it is hard to know what you
need.

This is a pretty good metaclass use, I think.

 

[Michele Simionato]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

Short answer: No.

x, y, z are in the class dictionary. dictionaries have no order.

You could use something like

class MyType(MyBaseType):
__mylist__ = [("x", 4), ("y", 5), ("z",6)]

and use the metaclass to set the class attributes x,y and z for you.


Michele Simionato

 

[Peter Otten]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z
somewhere?

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective . Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

class Meta(type):
def __new__(mcl, name, bases, classdict):
assert "ordered_names" not in classdict

values = []
for (n, v) in classdict.iteritems():
try:
v, i = v.value, v.index
except AttributeError:
pass
else:
values.append((i, n, v))
values.sort()

ordered_names = []
for (i, n, v) in values:
ordered_names.append(n)
classdict[n] = v
classdict["ordered_names"] = ordered_names

return type.__new__(mcl, name, bases, classdict)


class Base:
__metaclass__ = Meta

class Demo(Base):
alpha = 0
beta = OrderedValue(1)
gamma = OrderedValue(17)
delta = OrderedValue(3)

print Demo.ordered_names
print Demo.alpha, Demo.beta


Peter

 

[Bengt Richter]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z
somewhere?

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective . Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

class Meta(type):
def __new__(mcl, name, bases, classdict):
assert "ordered_names" not in classdict

values = []
for (n, v) in classdict.iteritems():
try:
v, i = v.value, v.index
except AttributeError:
pass
else:
values.append((i, n, v))
values.sort()

ordered_names = []
for (i, n, v) in values:
ordered_names.append(n)
classdict[n] = v
classdict["ordered_names"] = ordered_names

return type.__new__(mcl, name, bases, classdict)


class Base:
__metaclass__ = Meta

class Demo(Base):
alpha = 0
beta = OrderedValue(1)
gamma = OrderedValue(17)
delta = OrderedValue(3)

print Demo.ordered_names
print Demo.alpha, Demo.beta

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...
... names = cdict.get('ordic',[])
... names = [name for name in names if name in cdict]
... cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
... return type(cname, cbases, cdict)
... ... __metaclass__ = MC # really a function shortcut
... x = 123
... y = sys
... z = 0
... ordic = getnames()
... {'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}
... __metaclass__ = MC # really a function shortcut
... x = 123
... def m(self): pass
... def sm(): print getnames()
... sm = staticmethod(sm)
... ordic = getnames()
... ('getnames',)

Regards,
Bengt Richter

 

[Michael Hudson]

[snip]

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

I don't know whether to laugh or cry! Nice one.

Cheers,
mwh

 

[Peter Otten]

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

 import sys
 def getnames(): return sys._getframe(1).f_code.co_names ...
 def MC(cname, cbases, cdict):

...     names = cdict.get('ordic',[])
...     names = [name for name in names if name in cdict]
...     cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
...     return type(cname, cbases, cdict)
......     __metaclass__ = MC # really a function shortcut
...     x = 123
...     y = sys
...     z = 0
...     ordic = getnames()
...

 C.ordic

{'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}

A metaclass /function/ and sys._getframe() exercised on a class definition
- I think you have raised the bar for what qualifies as a hack

Peter

 

[Carlos Ribeiro]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Well, some people have already pointed to you about a recent
discussion on this topic. I was the one who started it all What I
have learned is:

-- Due to the way the class statement is handled, it's not possible
(without really black magic) to guarantee ordering in a transparent
way. If you need to put attributes in order, you have either to
provide a list with all attributes in order, or use derive your
attributes from a base class that provide some ordering mechanism of
its own.

-- You have to include an extra member into your class to record the
order. I have chosen to include a list, named "_fields", with all
attribute _names_. Storing the attribute name is guaranteed way to
keep the ordering even when a descendant override an attribute, or
when the user modify its value.

I have written a small package called GenericTemplates, that works all
the magic needed. It's the basis for a much more ambitious package for
'declarative-style' data structures using class statements. It handles
nested classes also, and keeps the ordering of all attributes that
inherit from a AbstractAttribute class (it includes nested classes
that inherit from GenericTemplate, and also OrderedAttributes and
TypedAttributes). I have a page for it in portuguese only at:

http://www.pythonbrasil.com.br/moin.cgi/TemplatesGen%E9ricos

The code is as follows -- still with debug code included, and lots of comments:

"""
metatemplate.py

Template class that can be used to write complex data structures using
nested classes. Template classes can store other template classes (nested)
or user-defined attributes (typed or untyped). The original definition
order information is preserved, allowing for true templating use for
applications such as html templates, data entry forms, and configuration
files.

(c) 2004 Carlos Ribeiro
(e-mail address removed)
http:///pythonnotes.blogspot.com

"""

import sys
from inspect import isclass, isdatadescriptor
from types import StringType, IntType, FloatType, ListType
import itertools

#----------------------------------------------------------------------
# Debug constants. I don't intend to remove them, even from production
# code, but I intend to use the logging module to print the messages

debug_generic_attribute = 0
debug_typed_attribute = 0
debug_auto_instantiation = 0

#----------------------------------------------------------------------
# AbstractAttribute is the ancestor of all classes that can be used
# in the metacontainer framework.

class AbstractAttribute(object):
pass

#----------------------------------------------------------------------
# GenericAttribute is the ancestor of all simple elements that are
# used as attributes of user defined Container subclasses
#
# GenericAttributes are simpler than full containers. They're both
# derived from the same AbstractAttribute class, but GenericAttributes
# have only a single value associated with them.
#
# When referred from a instance, the __get__ method returns the value
# associated with the attribute. If called from the class, the __get__
# method returns the property itself.

class GenericAttribute(AbstractAttribute):
""" Generic attributes for generic containers """
def __init__(self, default = None):
self._seqno = next_attribute_id()
self.value = default
def __repr__(self):
return "<Attr '%s'>" % (self.__class__.__name__)
def __get__(self, instance, owner):
if debug_generic_attribute:
print "GET self:[%s], instance:[%s], owner:[%s]" % \
(self, instance, owner)
if instance:
attrdict = instance.__dict__.setdefault('__attr__', {})
return attrdict.get(self.name, self.value)
else:
return owner
def __set__(self, instance, value):
if debug_generic_attribute:
print "SET self:[%s], instance:[%s], value:[%s]" % \
(self, instance, value)
attrdict = instance.__dict__.setdefault('__attr__', {})
attrdict[self.name] = value

class TypedAttribute(GenericAttribute):
""" Typed attributes for generic containers """
def __init__(self, default = None, mytype = None):
self._seqno = next_attribute_id()
self.value = default
if mytype:
if isclass(mytype):
self.mytype = mytype
else:
raise TypeError("Argument <mytype> expects None "
"or a valid type/class")
else:
self.mytype = type(default)
def __repr__(self):
return "<TypedAttr '%s':%s>" % \
(self.__class__.__name__, self.mytype.__name__)
def __get__(self, instance, owner):
if debug_typed_attribute:
print "GET self:[%s], instance:[%s], owner:[%s]" % \
(self, instance, owner)
if instance:
attrdict = instance.__dict__.setdefault('__attr__', {})
return attrdict.get(self.name, self.value)
else:
return self.value
def __set__(self, instance, value):
if debug_typed_attribute:
print "SET self:[%s], instance:[%s], value:[%s]" % \
(self, instance, value)
if not isinstance(value, self.mytype):
# if it's a string, tries to convert to the correct
# target type (this is needed because most things read
# from files will be strings anyway)
if isinstance(value, StringType):
value = self.mytype(value)
else:
raise TypeError, "Expected %s attribute" % \
self.mytype.__name__
attrdict = instance.__dict__.setdefault('__attr__', {})
attrdict[self.name] = value

#----------------------------------------------------------------------
# auxiliary functions

next_attribute_id = itertools.count().next

def getfields(dct):
"""
takes a dictionary of class attributes and returns a decorated list
containing all valid field instances and their relative position.

"""
for fname, fobj in dct.items():
if isinstance(fobj,GenericAttribute):
yield (fobj._seqno, (fname, fobj))
elif isclass(fobj) and issubclass(fobj,AbstractAttribute):
yield (fobj._seqno, (fname, fobj))
elif (fname[0] != '_'):
# conventional attributes from basic types are just stored
# as GenericAttributes, and put at the end of the list,
# in alphabetical order
if (isinstance(fobj,StringType) or
isinstance(fobj,IntType) or
isinstance(fobj,FloatType) or
isinstance(fobj,ListType)):
yield (sys.maxint, (fname, GenericAttribute(fobj)))
else:
yield (0, (fname, fobj))
else:
yield (0, (fname, fobj))

def makefieldsdict(dct, bases):
# build the field list and sort it
fields = list(getfields(dct))
fields.sort()
# undecorate the list and build a dict that will be returned later
sorted_field_list = [field[1] for field in fields]
field_dict = dict(sorted_field_list)
# finds all attributes and nested classes that are containers
attribute_list = [field for field in sorted_field_list
if (isinstance(field[1],AbstractAttribute) or
(isclass(field[1]) and
issubclass(field[1],AbstractAttribute)
))]
# check baseclasses for attributes inherited but not overriden
# !!WARNING: this code does not checks correctly for multiple
# base classes if there are name clashes between overriden
# members. This is not recommended anyway.
inherited = []
for baseclass in bases:
base_field_list = getattr(baseclass, '_fields', None)
# looks for a valid _fields attribute in an ancestor
if isinstance(base_field_list, ListType):
fnames = [f[0] for f in attribute_list]
for fname, fobj in base_field_list:
# checks for overriden attributes
if (fname in fnames):
# overriden - inherited list contains the new value
newobj = field_dict[fname]
inherited.append((fname, newobj))
# remove attribute and quick check field names list
attribute_list.remove((fname, field_dict[fname]))
fnames.remove(fname)
else:
# copy the original entry into the inherited list
inherited.append((fname, fobj))
field_dict['_fields'] = inherited + attribute_list
return field_dict

#----------------------------------------------------------------------
# MetaTemplate metaclass
#
# Most of the hard work is done outside the class by the auxiliary
# functions makefieldsdict() and getfields()

class MetaTemplate(type):
def __new__(cls, name, bases, dct):
# creates the class using only the processed field list
newdct = makefieldsdict(dct, bases)
newclass = type.__new__(cls, name, bases, newdct)
newclass._seqno = next_attribute_id()
newclass.name = name
return newclass

#----------------------------------------------------------------------
# GenericTemplate superclass

class GenericTemplate(AbstractAttribute):
__metaclass__ = MetaTemplate

def __init__(self):
""" instantiates all nested classes upon creation """

# builds a copy of the field list. this is needed to allow
# customizations of the instance not to be reflected in the
# original class field list.
self._fields = list(self.__class__._fields)

# auto instantiates nested classes and attributes
if debug_auto_instantiation:
print "AutoInstantiation <%s>: fieldlist = %s" % \
(self.name, self._fields)
for fname, fobj in self._fields:
if isclass(fobj) and issubclass(fobj,Container):
# found a nested class
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is a "
"Container Subclass" % (self.name, fname)
fobj = fobj()
setattr(self, fname, fobj)
elif isinstance(fobj, AbstractAttribute):
# found an attribute instance
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is an "
"Attribute Instance" % (self.name, fname)
# removed: parent links are still being thought out,
# and I'm not even sure if they're a good idea
# setattr(fobj, 'parent', self)
setattr(fobj, 'name', fname)
else:
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is "
"unknown" % (self.name, fname)

def iterfields(self):
for fname, fobj in self._fields:
yield getattr(self, fname)

def __repr__(self):
return "<%s '%s'>" % (self.__class__.__name__, self.name,)

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Carlos Ribeiro]

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

import sys
def getnames(): return sys._getframe(1).f_code.co_names ...
def MC(cname, cbases, cdict):

... names = cdict.get('ordic',[])
... names = [name for name in names if name in cdict]
... cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
... return type(cname, cbases, cdict)
...

class C(object):

... __metaclass__ = MC # really a function shortcut
... x = 123
... y = sys
... z = 0
... ordic = getnames()
...

C.ordic

{'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}

A metaclass /function/ and sys._getframe() exercised on a class definition
- I think you have raised the bar for what qualifies as a hack

Hey, I came first with that a couple of weeks ago but in truth, it
was Alex Martelli that pointed to me that a metaclass function would
work... but not only is it not recommended, it's also said to make
Guido shudder ;-) Seriously, although it works, it's not recommended
practice. Metaclasses are supposed to be classes, not functions.

As for the getframe, I have played with it a little bit also. But in
the end I have chosen to use a simple counter, using
itertools.count(). More pythonic IMHO. And no need for clever hacks,
when all that is needed is to order the elements in the order they are
executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen.

-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,c = myfunc(...)

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Nicolas Fleury]

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective . Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

That's the solution I finally did after my post (I think I found the
thread you were referring to). In my case, having to create instances
has not been so bad, as I can use them to put other information:

class SomeClass(BinType):
x = Member(Int32)
y = Member(Int16)

class SomeOtherClass(BinType):
a = ArrayMember(Int16, size=256)
b = Member(SomeClass)
c = Member(Int32)

This way, I can generate abstract documentation of my binary formats
without instantiating my types and can instantiate them to dump binary data.

Thx and Regards,
Nicolas

 

[Nicolas Fleury]

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

I don't know. I was thinking that maybe an ordereddict class would be
useful, something like:

myDict = {'a': 4, 'b': 5 } # normal dict
myDict = o{'a': 4, 'b': 5 } # ordered dict, keeping order ['a', 'b']

The dictionary could only be use in specific cases, like configuration
stuff or temporarily when instantiating a metaclass... I don't know,
maybe I'm just crazy...

Regards,
Nicolas

 

[Carlos Ribeiro]

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

I don't know. I was thinking that maybe an ordereddict class would be
useful, something like:

myDict = {'a': 4, 'b': 5 } # normal dict
myDict = o{'a': 4, 'b': 5 } # ordered dict, keeping order ['a', 'b']

The dictionary could only be use in specific cases, like configuration
stuff or temporarily when instantiating a metaclass... I don't know,
maybe I'm just crazy...

I don't think you're crazy. OTOH, defining 'ordering' is difficult. Is
it the alphabethical ordering, or is it the ordering of insertion?
Either way is arbitrary and it not going to satisfy everyone.

I am beginning to get a glimpse of the theorethical implications of
Python's metaclass design and implementation. It's an amazingly clever
solution (the __metaclass__ magic name, I mean); but I suspect that it
could be more powerful if there were a way to detect the metaclass
*before* the class statement is first processed; in this way, the
metaclass could provide a custom locals() dict, or implement other
types of hook for the class being created. But this is really hard
stuff, with implications beyond my current understanding, and I don't
feel prepared to argue at python-dev about it, really.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Peter Otten]

executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen.

-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,c = myfunc(...)

Do you have an example handy? I had so far no success breaking Bengt's code.

Peter

 

[Nicolas Fleury]

I don't think you're crazy. OTOH, defining 'ordering' is difficult. Is
it the alphabethical ordering, or is it the ordering of insertion?

Insertion. The point is that *there is* an order in the code.

Either way is arbitrary and it not going to satisfy everyone.

The ordereddict object could provide a method to change the order of
keys. Right now, in configuration dictionaries I must do (simplified):

SomeConfig(['y', 'x'], {'y': 1, 'x': 2})
or
SomeConfig(MyOrderedDict([('y', 1), ('x', 2)]))

what I would like is to do:
SomeConfig(o{'y': 1, 'x': 2})

I am beginning to get a glimpse of the theorethical implications of
Python's metaclass design and implementation. It's an amazingly clever
solution (the __metaclass__ magic name, I mean); but I suspect that it
could be more powerful if there were a way to detect the metaclass
*before* the class statement is first processed; in this way, the
metaclass could provide a custom locals() dict, or implement other
types of hook for the class being created. But this is really hard
stuff, with implications beyond my current understanding, and I don't
feel prepared to argue at python-dev about it, really.

I feel the same way. I was thinking about writing a PEP for an
ordereddict, suggesting in it that one of the applications could be the
metaclasses. An ordereddict could also contain a dict, and could be
used only temporarily only during metaclass creation and after just keep
the normal dictionary in it.

Regards,
Nicolas

 

[Carlos Ribeiro]

Insertion. The point is that *there is* an order in the code.

In this case, yes. But I meant to say that a ordereddict would need to
make a choice here. Either way, our particular need doesn't seem to be
common enough to warrant such a patch.

BTW, I'm curious about why are doing it yourself. In my particular
case, I'm interested in declarative idioms for templating code: data
entry forms, configuration files, and similar stuff. I also think
that, whatever you're looking after, we could share some code and
experiences, and perhaps bundle it all into a single library. What do
you think of it?

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Carlos Ribeiro]

Do you have an example handy? I had so far no success breaking Bengt's code.

Ok. Just as an exercise -- at this point we're far from safe Python
land anyway, and it's not recommended to push it beyond this limit...
;-) Pseudo code only:

class Foo:
# loop that creates attributes out of a list
for name, val in list:
locals()[name] = val

or (actual code tested):
.... a,b,c,d = (1,2,3,4)
.... {'a': 1, '__module__': '__main__', 'b': 2, 'd': 4, 'c': 3, '__doc__': None}

The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Nicolas Fleury]

In this case, yes. But I meant to say that a ordereddict would need to
make a choice here. Either way, our particular need doesn't seem to be
common enough to warrant such a patch.

I don't know. Store the order only adds a list that would not affect
lookup performance in the dictionary. Right now, the order is simply
lost; that's not nice. Alphabetical order can be get anytime, so I
really see that as different, since there are a lot of solutions for
these needs, while there's only hacks for our need.

BTW, I'm curious about why are doing it yourself. In my particular
case, I'm interested in declarative idioms for templating code: data
entry forms, configuration files, and similar stuff. I also think
that, whatever you're looking after, we could share some code and
experiences, and perhaps bundle it all into a single library. What do
you think of it?

My solution for now is almost exactly what Peter Otten posted,
consisting only of a metaclass and a base class for attributes. I do
some specific stuff, like instanciate some type stored in the attributes
in each instance. I could inherit from your classes instead, but I
don't have a lot to share for now. But yes, I would like very much to
share what I have once it is completed. What I'm doing looks a lot like
a metastruct module (see example in previous post).

Regards,
Nicolas

 

[Thomas Heller]

like very much to share what I have once it is completed. What I'm
doing looks a lot like a metastruct module (see example in previous
post).

In this case, you could also look at the ctypes module, it implements C
compatible structures and unions.

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

Thomas

 

[Nicolas Fleury]

In this case, you could also look at the ctypes module, it implements C
compatible structures and unions.

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

I just looked at ctypes, and yes, that's exactly what I'm doing. I want
the structures definitions to be modified by C++ programmers and I find
the metaclass mechanism much more readable and easy to modify (and it's
not a lot of code to make everything work). I agree everything can be
done using normal Python code, but my goal here is really to make a
pseudo-language to define our formats, and the pseudo-language is just a
simpler solution than XML in my specific case. Probably all cases of
such using of metaclasses are like that...

Regards,
Nicolas

 

[Peter Otten]

Ok. Just as an exercise -- at this point we're far from safe Python
land anyway, and it's not recommended to push it beyond this limit...
;-) Pseudo code only:

class Foo:
# loop that creates attributes out of a list
for name, val in list:
locals()[name] = val

Agreed.

or (actual code tested):

...         a,b,c,d = (1,2,3,4)
...
{'a': 1, '__module__': '__main__', 'b': 2, 'd': 4, 'c': 3, '__doc__':
{None}

You cannot rely on a vars() because it produces a dict - and dictionaries
don't preserve the insertion order by design.

The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

No, it doesn't:
.... a, b, c, d = "xyzt"
.... print sys._getframe().f_code.co_names
....
('__name__', '__module__', 'a', 'b', 'c', 'd', 'sys', '_getframe', 'f_code',
'co_names')

whereas
['a', '__module__', 'b', 'd', 'c', '__doc__']

Of course there are other ways to garble the co_names order:
.... def __getattr__(self, name): return name
........ x = any.d
.... a, b, c, d = "xyzt"
.... print sys._getframe().f_code.co_names
....
('__name__', '__module__', 'any', 'd', 'x', 'a', 'b', 'c', 'sys',
'_getframe', 'f_code', 'co_names')

All very useless/interesting stuff

Peter