Dynamic Class Creation

[Josh English]

I have a large program with lots of data stored in XML. I'm upgrading
my GUI to use ObjectListView, but with my data in XML, I don't have
regular objects to interact with the OLV. I do have an XML validator
that defines the structure of the XML elements, and I'm trying to
dynamically create a class to wrap the XML element.

So, an element like:

<market code="WotF">
<title>Writers of the Future</title>
</market>

I want to create a class like:

class Market(object):
def __init__(self, elem):
self._elem = elem

def get_code(self):
return self._elem.get('code')

def set_code(self, value):
self._elem.set('code', value)

def get_title(self):
return self._elem.find('title').text

def set_title(self, value):
node = self._elem.find('title')
node.text = value

Naturally, I don't want to hand code this for every interface but
would like to create them dynamically. (The laziness of programming, I
guess.)

I have tried several solutions that I found on various forums, but
they are all several years old.

Questions:

What's the best way to create these helper methods?

How can I attach them to the class and have it run?

I have had a few solutions work where I had a class with three methods
(get_code, get_tier, get_mail) but they all return the same value, not
the individual values.

 

[Chris Rebert]

I have a large program with lots of data stored in XML. I'm upgrading
my GUI to use ObjectListView, but with my data in XML, I don't have
regular objects to interact with the OLV. I do have an XML validator
that defines the structure of the XML elements, and I'm trying to
dynamically create a class to wrap the XML element.

So, an element like:

<market code="WotF">
<title>Writers of the Future</title>
</market>

I want to create a class like:

class Market(object):
   def __init__(self, elem):
       self._elem = elem

   def get_code(self):
       return self._elem.get('code')

   def set_code(self, value):
       self._elem.set('code', value)

   def get_title(self):
       return self._elem.find('title').text

   def set_title(self, value):
       node = self._elem.find('title')
       node.text = value

Naturally, I don't want to hand code this for every interface but
would like to create them dynamically. (The laziness of programming, I
guess.)

I have tried several solutions that I found on various forums, but
they are all several years old.

Questions:

What's the best way to create these helper methods?

Nested functions:

def make_attr_getset(name):
def get(self):
return self._elem.get(name)

def set(self, value):
self._elem.set(name, value)
return get, set

get_code, set_code = make_attr_getset('code')

def make_subelement_getset(name):
def get(self):
return self._elem.find(name).text

def set(self, value):
node = self._elem.find(name)
node.text = value
return get, set

get_title, set_title = make_subelement_getset('title')

How can I attach them to the class and have it run?

Use properties and setattr():

class Market(object):
def __init__(... #same as before

setattr(Market, 'code', property(get_code, set_code))
setattr(Market, 'title', property(get_title, set_title))

m = Market(the_XML)
print m.title #=> Writers of the Future
m.title = "Writers of the Past"
print m.code #=> WotF
m.code = "WofP"

Cheers,
Chris

 

[Jack Diederich]

You can either define a catch-all __getattr__ method to look them up
dynamically, or as Chris kinda-suggested write descriptors for the
individual elements.

class Market():
def __init__(self, elem):
self._elem = elem
def __getattr__(self, name):
try:
# I'm assuming this raises a KeyError when not found
return self._elem.get(name)
except KeyError:
return self._elem.find(name)
def __setitem__(self, name, value):
# like __getitem__ but for setting

Chris' property maker function is almost like a descriptor class (how
properties are implemented under the hood), here's a refactoring
[untested]

class ElemGetProperty():
def __init__(self, name):
self.name = name
def __get__(self, ob, cls):
return ob._elem.get(self.name)
def __set__(self, ob, val):
ob._elem.set(self.name, val)

You could write one property class for each kind of element (get/find)
and then put them in your class like this

class Market():
code = ElemGetProperty('code')
title = ElemFindProeprty('title')

The getattr/setattr method is easier to understand and will handle
arbitrary elements; for the descriptors version you'll have to define
one for each tag that might be used on the class.

-Jack

 

[Chris Rebert]

You can either define a catch-all __getattr__ method to look them up
dynamically, or as Chris kinda-suggested write descriptors for the
individual elements.

class Market():
 def __init__(self, elem):
   self._elem = elem
 def __getattr__(self, name):
   try:
     # I'm assuming this raises a KeyError when not found
     return self._elem.get(name)
   except KeyError:
     return self._elem.find(name)
 def __setitem__(self, name, value):

did you mean __setattr__ here?

The getattr/setattr method is easier to understand and will handle
arbitrary elements;  for the descriptors version you'll have to define
one for each tag that might be used on the class.

Cheers,
Chris

 

[Josh English]

Chris,

Thanks. This worked for the attributes, but I think the tactic is
still misleading. There are child elements I can't quite determine how
to deal with:

<market code='anlg' tier='ProMarket' mail='True'>
<title field="pref">Analog Science Fiction and Fact</title>
<nickname>Analog</nickname>
<keyword>Science Fiction</keyword>
<keyword>First Contact</keyword>
<keyword>Hard Science Fiction</keyword>
<address>
<attnline>Stanley Schmidt, Editor</attnline>
<address1>267 Broadway, 4th Floor</address1>
<address2>New York, NY 10007-2352</address2>
</address>
<website>http://www.analogsf.com</website>
</market>

A child element with text and an attribute or two, for example, pose a
problem. I can call Market.title but should I try Market.title.field
or Market.title_field.

Multiple elements, such as keywords, are allowed in xml but harder to
map to the object. I don't know if I want to go create a list and
methods for accessing those keywords as a list, or redefine the rules
of my XML to not allow multiple child elements with the same tag. I
can't decide.

Then the address is a bit of a bear.

In short, I have to figure out the whole object interface to the XML
and how I want that to work.

Thanks for the suggestion. It is undeniably clever.

Josh

 

[Gerard Flanagan]

Chris,

Thanks. This worked for the attributes, but I think the tactic is
still misleading. There are child elements I can't quite determine how
to deal with:

<market code='anlg' tier='ProMarket' mail='True'>
<title field="pref">Analog Science Fiction and Fact</title>
<nickname>Analog</nickname>
<keyword>Science Fiction</keyword>
<keyword>First Contact</keyword>
<keyword>Hard Science Fiction</keyword>
<address>
<attnline>Stanley Schmidt, Editor</attnline>
<address1>267 Broadway, 4th Floor</address1>
<address2>New York, NY 10007-2352</address2>
</address>
<website>http://www.analogsf.com</website>
</market>

A child element with text and an attribute or two, for example, pose a
problem. I can call Market.title but should I try Market.title.field
or Market.title_field.

Multiple elements, such as keywords, are allowed in xml but harder to
map to the object. I don't know if I want to go create a list and
methods for accessing those keywords as a list, or redefine the rules
of my XML to not allow multiple child elements with the same tag. I
can't decide.

Then the address is a bit of a bear.

In short, I have to figure out the whole object interface to the XML
and how I want that to work.

Have you heard of or considered PyXB (http://pyxb.sourceforge.net/)? Not
that I've much experience with it, but you can use it to generate python
code from XML Schema files, and, I think, WSDL definitions. It's a bit
of a rabbit hole, but then isn't that half the fun!

The following might give you the idea. You start with a Schema
schema.xsd, generate a python module from it, then read in some XML with
the generated classes to get your custom objects.


---------------- schema.xsd ----------------------

<xs:schema targetNamespace="http://www.josh.com/schema/0.1"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:josh="http://www.josh.com/schema/0.1">


<xs:element name="market" type="josh:MarketType"/>

<xs:complexType name="MarketType">
<xs:sequence>
<xs:element name="title" type="josh:TitleType"/>
<xs:element name="nickname" type="xs:string"/>
<xs:element name="keyword" type="xs:string"
maxOccurs="unbounded"/>
<xs:element name="address" type="josh:USAddress"/>
<xs:element name="website" type="xs:string"/>
</xs:sequence>
<xs:attributeGroup ref="josh:MarketAttributes"/>
</xs:complexType>

<xs:complexType name="TitleType">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="field" type="xs:string"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>

<xs:attributeGroup name="MarketAttributes">
<xs:attribute name="code" type="xs:string"/>
<xs:attribute name="tier" type="xs:string"/>
<xs:attribute name="mail" type="xs:string"/>
</xs:attributeGroup>

<xs:complexType name="Address" abstract="true">
<xs:sequence>
<xs:element name="name" type="xs:string"/>
<xs:element name="street" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
</xs:sequence>
</xs:complexType>

<xs:complexType name="USAddress">
<xs:complexContent>
<xs:extension base="josh:Address">
<xs:sequence>
<xs:element name="state" type="josh:USState"
minOccurs="0"/>
<xs:element name="zip" type="xsositiveInteger"
minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>

<xs:simpleType name="USState">
<xs:restriction base="xs:string">
<xs:enumeration value="AK"/>
<xs:enumeration value="AL"/>
<xs:enumeration value="AR"/>
<!-- and so on ... -->
</xs:restriction>
</xs:simpleType>

</xs:schema>

---------------- genbindings.sh ---------------------------

#!/bin/sh

pyxbgen -u schema.xsd -m market_binding -r

---------------- demo.py -----------------------------------

s = """<?xml version="1.0"?>
<josh:market
xmlns:josh="http://www.josh.com/schema/0.1"
code="anlg"
tier="ProMarket"
mail="True">

<title field="pref">Analog Science Fiction and Fact</title>
<nickname>Analog</nickname>
<keyword>Science Fiction</keyword>
<keyword>First Contact</keyword>
<keyword>Hard Science Fiction</keyword>
<address>
<name>Stanley Schmidt, Editor</name>
<street>267 Broadway, 4th Floor</street>
<city>New York</city>
</address>
<website>http://www.analogsf.com</website>
</josh:market>
"""

import xml.dom.minidom
import market_binding

market =
market_binding.CreateFromDOM(xml.dom.minidom.parseString(s).documentElement)

print market.tier
print market.code
print market.keyword
assert len(market.keyword) == 3
print market.address.name
print market.toxml()

 

[Aahz]

<market code='anlg' tier='ProMarket' mail='True'>
<title field="pref">Analog Science Fiction and Fact</title>
<nickname>Analog</nickname>
<keyword>Science Fiction</keyword>
<keyword>First Contact</keyword>
<keyword>Hard Science Fiction</keyword>
<address>
<attnline>Stanley Schmidt, Editor</attnline>
<address1>267 Broadway, 4th Floor</address1>
<address2>New York, NY 10007-2352</address2>
</address>
<website>http://www.analogsf.com</website>
</market>

A child element with text and an attribute or two, for example, pose a
problem. I can call Market.title but should I try Market.title.field
or Market.title_field.

Multiple elements, such as keywords, are allowed in xml but harder to
map to the object. I don't know if I want to go create a list and
methods for accessing those keywords as a list, or redefine the rules
of my XML to not allow multiple child elements with the same tag. I
can't decide.

You should always think "LIST!" any time you have the potential for
multiple elements with the same semantic tag. Whether you do it as a
list for all elements or as a combo dict/list is something you need to
decide; the latter is faster in many ways but is more cumbersome:

Market.title.keyword[1]

(Instance attributes and dict keys are almost trivially convertible.)

You would probably get some mileage out of looking at the ElementTree
implementation.