Adding properties to an instance

[dg.google.groups]

Hi all,

So I understand that properties belong to a class not an instance, but
nonetheless I want to add properties to an instance. I have a class
which when an instance is created runs some fairly complicated code
and produces a set of names which I'd like to be able to access via
properties. At the moment, I'm using something like obj.getvar(name)
but I'd like to be able to write obj.name. (Note that they can't be
just standard attributes because they only get computed when they are
accessed.) I could generate functions like obj.name() but I want it to
be obj.name instead.

The solution I've come up with is to create a new class for each
object which is just the real class with some extra properties, and
then dynamically change the class of the object to this new class.
This seems to work, but I wonder if (a) there is a nicer solution than
the one I'll post below, (b) if there are any dangers or pitfalls of
this approach. The obvious difficulty is with derived classes. At the
moment, I'm insisting that a derived class has to call a makeprops()
method to create the properties.

It's kind of similar to this recipe:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/197965

but that recipe has a much simpler situation in which the properties
and values are known at the time of the creation of the object (by
contrast, I don't know what the properties are until the end of the
__init__ method).

Any thoughts?

Code below to illustrate my approach.

import warnings
from operator import itemgetter

class A(object):
def __init__(self,**kwds):
self._kwds = kwds
self.makeprops()
def __getitem__(self,i):
return self._kwds
def makeprops(self):
if not hasattr(self,'_madeprops'):
self._madeprops = set()
self._failedprops = set()
class _A(self.__class__):
pass
for k,v in self._kwds.items():
if not k in self._madeprops and k in dir(self):
if not k in self._failedprops:
warnings.warn("Cannot create property "+k+",
already used in object "+str(self),RuntimeWarning)
self._failedprops.add(k)
else:
setattr(_A,k,property(fget=itemgetter(k)))
self._madeprops.add(k)
self.__class__ = _A

class B(A):
def __init__(self,**kwds):
super(B,self).__init__(**kwds)
self.makeprops()

class C(A):
def __init__(self,**kwds):
self._kwds = kwds

a = A(x=1)
b = B(x=2,makeprops=3)
c = C(x=3)
print isinstance(a,A), isinstance(a,B), isinstance(a,C) # True False
False
print isinstance(b,A), isinstance(b,B), isinstance(b,C) # True True
False
print isinstance(c,A), isinstance(c,B), isinstance(c,C) # True False
True
print a.__class__ # <class '__main__._A'>
print b.__class__ # <class '__main__._A'>
print c.__class__ # <class '__main__.C'>
print a.x # 1
print b.x # 2
print b.makeprops # <bound method _A.makeprops of <__main__._A object
at 0x00A86810>>
try:
print c.x # raises exception
except AttributeError:
print "c has no element x"
c.makeprops()
print c.x # 3
print a.__class__ # <class '__main__._A'>
print b.__class__ # <class '__main__._A'>
print c.__class__ # <class '__main__._A'>

 

[Gabriel Genellina]

So I understand that properties belong to a class not an instance, but
nonetheless I want to add properties to an instance. I have a class
which when an instance is created runs some fairly complicated code
and produces a set of names which I'd like to be able to access via
properties.

I'd suggest a small improvement: _A as a class name isn't very nice.
Replace the inner class statement with:
_A = type(self.__class__.__name__ + '_autoprops', (self.__class__,), {})

A problem with this approach is that instances aren't pickleable (perhaps
that could be solved using __reduce__)

 

[bruno.desthuilliers]

Hi all,

So I understand that properties belong to a class not an instance, but
nonetheless I want to add properties to an instance.

While this is technically possible (I tried a couple years ago), it
requires hacking the __getattribute__ method, which is something I
would not recommand, not only because it can be tricky, but mostly
because this is a very critical path wrt/ perfs. (IIRC it also
required using custom descriptors, but I'm not really sure about this
last point).

I have a class
which when an instance is created runs some fairly complicated code
and produces a set of names which I'd like to be able to access via
properties. At the moment, I'm using something like obj.getvar(name)
but I'd like to be able to write obj.name.

Before new-style classes, we used the __getattr__/__setattr__ hooks
for computed attributes. While this approach is now a bit abandonned
in favor of descriptors (properties or custom ones), it still works
fine, and is probably the best solution to your problem.

HTH

 

[dg.google.groups]

I'd suggest a small improvement: _A as a class name isn't very nice.
Replace the inner class statement with:
_A = type(self.__class__.__name__ + '_autoprops', (self.__class__,), {})

Ah yes, that's much nicer.

A problem with this approach is that instances aren't pickleable (perhaps
that could be solved using __reduce__)

I think because the properties can be computed from knowing the rest
of the data of the object, it would be safe when pickling to just
pickle a copy of the object with the __class__ changed back to A (and
then when you load it again, you can just generate the properties
anew). I haven't really thought much about pickling but it would
certainly be a nice feature to have.

 

[dg.google.groups]

While this is technically possible (I tried a couple years ago), it
requires hacking the __getattribute__ method, which is something I
would not recommand, not only because it can be tricky, but mostly
because this is a very critical path wrt/ perfs. (IIRC it also
required using custom descriptors, but I'm not really sure about this
last point).

Performance is pretty important for the class I'm designing so I think
__getattribute__ is probably out. The computed properties are only
infrequently accessed, but using __getattribute__ slows everything
down, right?

Before new-style classes, we used the __getattr__/__setattr__ hooks
for computed attributes. While this approach is now a bit abandonned
in favor of descriptors (properties or custom ones), it still works
fine, and is probably the best solution to your problem.

Ah, I didn't know about these - it looks as though they might be just
the thing since it seems they're only called after all the other
methods fail. That looks like there would be no performance hit, I
wouldn't need to mess around with dynamically changing the class, and
it would automatically deal with the (irritating) feature of having to
check if there is already something in the object's dir() with that
name. I'll look into this tomorrow - I hope this feature isn't going
to be removed in future versions of Python?

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

Performance is pretty important for the class I'm designing so I think
__getattribute__ is probably out. The computed properties are only
infrequently accessed, but using __getattribute__ slows everything
down, right?

Indeed - it *is* the attribute lookup mechanism. Better to leave it alone.

Ah, I didn't know about these - it looks as though they might be just
the thing since it seems they're only called after all the other
methods fail.
Right.

That looks like there would be no performance hit, I
wouldn't need to mess around with dynamically changing the class, and
it would automatically deal with the (irritating) feature of having to
check if there is already something in the object's dir() with that
name. I'll look into this tomorrow - I hope this feature isn't going
to be removed in future versions of Python?

I really don't think so. It's still has it's use for automatic
delegation. And, as is the case here, for per-instance computed attributes.

As a side note: the naming symetry between __getattr__ and __setattr__
is a gotcha, since __setattr__ is mostly symetric to __getattribute__ -
IOW, customizing __setattr__ is a bit tricky. The naive approach, ie:

class Parrot(object):
def __setattr__(self, name, val):
self.name = val

will indeed go into infinite recursion (or would, if the interpreter
didn't stop it after a while)

The solution is of course to call on the parent class's __setattr__:

class Ni(object):
def __setattr__(self, name, val):
object.__setattr__(self, name, value)

HTH

 

[dg.google.groups]

As a side note: the naming symetry between __getattr__ and __setattr__

is a gotcha, since __setattr__ is mostly symetric to __getattribute__ -
IOW, customizing __setattr__ is a bit tricky. The naive approach, ie:

Ah I see - so __setattr__ is called immediately whereas __getattr__ is
only called if the other methods fail. Does this mean that __setattr__
incurs the same performance penalty that overriding __getattribute__
would? Possibly I can live with this because I think that most of what
I'm doing is getting attributes, or modifying mutable ones, rather
than setting them.

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

Ah I see - so __setattr__ is called immediately whereas __getattr__ is
only called if the other methods fail.
Yes.

Does this mean that __setattr__
incurs the same performance penalty that overriding __getattribute__
would?

Not quite AFAICT - there's less going on here. Also, getting an
attribute is (usually at least) more common than setting it.

Possibly I can live with this because I think that most of what
I'm doing is getting attributes, or modifying mutable ones, rather
than setting them.

Well... Using the __setattr__/__getattr__ hooks is IMHO the simplest
solution that can possibly work - far simpler than your previous one at
least. As far as I'm concerned, and unless some other point of your
specs make this unusable or unpractical, I'd go for this solution first
and run a couple benchs on real-life-or-close-to conditions to check if
the performance hit is acceptable.

 

[dg.google.groups]

Does this mean that __setattr__

Not quite AFAICT - there's less going on here. Also, getting an
attribute is (usually at least) more common than setting it.


Well... Using the __setattr__/__getattr__ hooks is IMHO the simplest
solution that can possibly work - far simpler than your previous one at
least. As far as I'm concerned, and unless some other point of your
specs make this unusable or unpractical, I'd go for this solution first
and run a couple benchs on real-life-or-close-to conditions to check if
the performance hit is acceptable.

I think you're right - I've just tried implementing a simple version
of this in my code and it seems that in the time critical parts of it
__setattr__ isn't called even once. So I think I'll go with this
solution (after having run a few tests).

Thanks for your suggestion!