Per instance descriptors ?

[bruno at modulix]

Hi

I'm currently playing with some (possibly weird...) code, and I'd have a
use for per-instance descriptors, ie (dummy code):

class DummyDescriptor(object):
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, 'bar', 'no bar')

class MyClass1(object):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
self.baaz = DummyDescriptor()

mc1 = MyClass1(bar='back')
mc1.baaz
-> <__main__.DummyDescriptor object at 0x2aaaabc6c390>

Which is of course what one would expect... Now I tried the following
hack^Mworkaround:

class MyClass2(MyClass1):
def __getattribute__(self, key):
v = MyClass1.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(self, self.__class__)
return v

And it *seems* to work just fine:

mc2 = MyClass2(bar='foo')
mc2.baaz
-> 'foo'

Now the question: is there any obvious (or non-obvious) drawback with
this approach ?

A bit of context:
1/ the real class is a decorator for controller functions in a
mod_python application, and given previous experiences with mod_python,
I'd prefer not mess with the class itself at runtime... still I'd like
to abstract some gory details, and descriptors would be an obvious
choice here.

2/ this is for production code, so robustness is more important than
syntactic sugar - but having this syntactic sugar would be nice...

Any hint ?

TIA

 

[Ziga Seilnacht]

Hi

I'm currently playing with some (possibly weird...) code, and I'd have a
use for per-instance descriptors, ie (dummy code):

Now the question: is there any obvious (or non-obvious) drawback with
this approach ?

Staticmethods won't work anymore:
.... @staticmethod
.... def foo():
.... pass
.... def __getattribute__(self, name):
.... v = object.__getattribute__(self, name)
.... if hasattr(v, '__get__'):
.... return v.__get__(self, self.__class__)
.... return v
....Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: foo() takes no arguments (1 given)

TIA

Ziga

 

[bruno at modulix]

Staticmethods won't work anymore:



... @staticmethod
... def foo():
... pass
... def __getattribute__(self, name):
... v = object.__getattribute__(self, name)
... if hasattr(v, '__get__'):
... return v.__get__(self, self.__class__)
... return v
...


Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: foo() takes no arguments (1 given)

Hmmm.... Well, I almost never use staticmethods, but I need to check
this out.

(a few minutes later)

Ok, no apparent impact on classmethods. For staticmethod, a quick fix is:

import types

... def __getattribute__(self, name):
... v = object.__getattribute__(self, name)
... if not isinstance(v, types.FunctionType) \

and hasattr(v, '__get__'):

... return v.__get__(self, self.__class__)
... return v

Thanks Ziga.


Anyone else ? Any good reason to *not* do this ? Or is presumably safe ?

 

[Michael Spencer]

I may be missing the subtlety of what you're up to, but why is overriding
__getattribute__ more desirable than simply defining the descriptor in a subclass?
i.e.,
class MyClass3(MyClass1):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
baaz = DummyDescriptor()


Michael

 

[Steven Bethard]

Hi

I'm currently playing with some (possibly weird...) code, and I'd have a
use for per-instance descriptors, ie (dummy code):

class DummyDescriptor(object):
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, 'bar', 'no bar')

class MyClass1(object):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
self.baaz = DummyDescriptor()

mc1 = MyClass1(bar='back')
mc1.baaz
-> <__main__.DummyDescriptor object at 0x2aaaabc6c390>

Which is of course what one would expect... Now I tried the following
hack^Mworkaround:

class MyClass2(MyClass1):
def __getattribute__(self, key):
v = MyClass1.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(self, self.__class__)
return v

And it *seems* to work just fine:

mc2 = MyClass2(bar='foo')
mc2.baaz
-> 'foo'

Now the question: is there any obvious (or non-obvious) drawback with
this approach ?

Don't know if this matters, but if you override __getattribute__, you'll
slow down all attribute accesses to this object. If this matters, you
could write something like:

class MyClass(object):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
def __getattr__(self, name):
if name == 'baaz':
return self.bar
elif name == 'bar':
return 'no bar'

Then you only incur the penalty on the ``baaz`` lookup and the ``bar``
lookup when it's missing -- not on all attribute lookups.

Could you explain again why you don't want baaz to be a class-level
attribute?

STeVe

 

[Michael Spencer]

Michael Spencer a écrit :

The code snippet I gave as an example was not supposed to reflect how I
effectively mean to use per-instance descriptors, it was just a kind of
Minimal Working Code (tm). The real life code is about 500 LOC, and
explaining the whole thing would take far too long. Also, as I said,
this is mostly syntactic sugar - there are simpler, less 'hackish' (but
also less elegant) solutions to the actual 'problem'.So, to answer your
question, no, subclassing would not be a solution - I'd almost need a
subclass per controller function, which would reintroduce the
boilerplate I'm trying to get rid of.

BTW, there may be other use case for per-instance descriptors...

Agreed. Per-instance descriptors could be interesting (that's why the subject
line caught my attention).
But your solution involves a custom __getattribute__ in the class, which I would
always avoid if possible (and I gather you're uneasy about it too).
Here, I don't see why that's better than having a descriptor in the class and,
if it needs per-instance behavior, then make it dependent on something provided
by the instance.

e.g.,
class DummyDescriptor1(object):
def __get__(self, obj, objtype=None):
if isinstance(obj, objtype):
return obj.foo.__get__(obj)
else:
return self

class MyClass4(object):
baaz = DummyDescriptor1()
def __init__(self, foo, bar = None):
self.foo = foo
self.bar = bar

Python
is so dynamic that you can almost use it like a prototype-based language.

Almost, yes.

Michael

 

[Bruno Desthuilliers]

Michael Spencer a écrit :

I may be missing the subtlety of what you're up to, but why is
overriding __getattribute__ more desirable than simply defining the
descriptor in a subclass?

The code snippet I gave as an example was not supposed to reflect how I
effectively mean to use per-instance descriptors, it was just a kind of
Minimal Working Code (tm). The real life code is about 500 LOC, and
explaining the whole thing would take far too long. Also, as I said,
this is mostly syntactic sugar - there are simpler, less 'hackish' (but
also less elegant) solutions to the actual 'problem'. So, to answer your
question, no, subclassing would not be a solution - I'd almost need a
subclass per controller function, which would reintroduce the
boilerplate I'm trying to get rid of.

BTW, there may be other use case for per-instance descriptors... Python
is so dynamic that you can almost use it like a prototype-based language.


(snip code)

 

[bruno at modulix]

Bruno Desthuilliers wrote:
(snip)


Agreed. Per-instance descriptors could be interesting (that's why the
subject line caught my attention).
But your solution involves a custom __getattribute__ in the class, which
I would always avoid if possible (and I gather you're uneasy about it too).

I'm not uneasy about overriding __getattribute__, just about overriding
it that way -

Here, I don't see why that's better than having a descriptor in the
class and, if it needs per-instance behavior, then make it dependent on
something provided by the instance.

Each instance will need it's own set of descriptors. The class is used
as a decorator for many controller functions. The descriptors are used
to encapsulate some gory implementation details about how to get such or
such object from the framework (they of course depend on instance
specific data, but that's not the point).

e.g.,
class DummyDescriptor1(object):
def __get__(self, obj, objtype=None):
if isinstance(obj, objtype):
return obj.foo.__get__(obj)
else:
return self

class MyClass4(object):
baaz = DummyDescriptor1()
def __init__(self, foo, bar = None):
self.foo = foo
self.bar = bar

This would imply a decorator subclass and a descriptor subclass for each
and every controller function - which is what I'm trying to avoid.

(snip)

 

[bruno at modulix]

(snip)



Don't know if this matters, but if you override __getattribute__, you'll
slow down all attribute accesses to this object.

Yes, I know, but this shouldn't be a major annoyance here.

If this matters, you
could write something like:

class MyClass(object):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
def __getattr__(self, name):
if name == 'baaz':
return self.bar
elif name == 'bar':
return 'no bar'

Don't focus on the dummy example I gave - the real descriptors are doing
something a bit less stupid !-)

Could you explain again why you don't want baaz to be a class-level
attribute?

Because the class is a decorator for many controller functions, and each
controller function will need it's own set of descriptors, so I don't
want to mess with the class.

Think of the decorator as a prototype, each controller function
customizing it according to it's need - this customisation including
the decorator instance attaching descriptors and methods to itself
according to parameters passed at __init__ time. The decorator instance
also passes itself as first arg to the controller function - which then
practically become an instance method too.

Don't tell me, I know this is a somewhat weird architecture, and it
could mostly be done with more conventional subclassing. BTW, this was
how a first implementation worked, and it required almost twice more
code than the new one I'm experimenting, without being half as flexible.

As I said, it's mostly syntactic sugar, but what, I'm lazy enough to
spend time on writing code that will allow me to write less code in the
end !-)

 

[Steven Bethard]

Because the class is a decorator for many controller functions, and each
controller function will need it's own set of descriptors, so I don't
want to mess with the class.

So you're trying to add property-like attributes to functions? That is,
you want something like:

@my_decorator
def f(...):
...

f.foo # calls f._get_foo()

> This would imply a decorator subclass and a descriptor subclass for
> each and every controller function - which is what I'm trying to
> avoid.

So you only want one decorator? Doesn't that mean that all functions
will have the same attributes? But if that were true you would only
need one descriptor for all controller functions, so I must not be
understanding that right.

Can you give a little more realistic code sample? I'm still not sure
what it is you really want to do. Don't worry about showing the
implementation you're thinking of. Just show me how you want to use
these things and what it ought to look like.


STeVe

 

[bruno at modulix]

Steven Bethard wrote:
(some smart questions)

Steven , I owe you a *big* thank.

I knew they must have been something wrong, but couldn't point what. Now
I see, and it's of course totally obvious. Using a class as a
decorator, I have of course only one instance of it per function - and
for some attributes, I need an instance per function call.

Duh

Well, at least I will have learn some new things...

 

[Steven Bethard]

Using a class as a
decorator, I have of course only one instance of it per function - and
for some attributes, I need an instance per function call.

Per function call? And you want the attributes on the function, not the
result of calling the function? If so, that'd be pretty difficult...

I guess you could do something like:
.... def __init__(self, func):
.... self.func = func
.... self.call_no = 0
.... self.get_foo = lambda: 'one thing'
.... def __call__(self, *args, **kwargs):
.... self.call_no += 1
.... if self.call_no == 1:
.... del self.get_foo
.... self.get_bar = lambda: 'another thing'
.... else:
.... del self.get_bar
.... self.get_baz = lambda: 'a third thing'
.... return self.func(*args, **kwargs)
........ def f(*args, **kwargs):
.... print args, kwargs
....Traceback (most recent call last):
Traceback (most recent call last):
'a third thing'

But that looks pretty nasty to me. It sounds like your architecture
could use some redesigning -- having different attributes on a function
for each function call seems like a bad idea. Can't you have the
returned objects carry the different attributes?

STeVe

 

[bruno at modulix]

Steven Bethard wrote:

(snip code)

But that looks pretty nasty to me.

It sounds like your architecture
could use some redesigning

Done - in much more sane way. Got rid of some more boilerplate and of
the whole problem of per-instance descriptors BTW !-)

I should probably sleep more and program less...

Thanks again, Steven. You've been of great help - even if indirectly.
Remind me to buy you a beer if we ever meet !-)