Descriptors and side effects

[mrkafk]

Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors.

For example, let's take a simple example where you set an attribute s
to a string and have another attribute l set automatically to its
length.
def __init__(self,val):
self.s=val
self.l=len(val)
print "creating value: ", self.s
print "id(self.l)", id(self.l)
def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l
def __get__(self, obj, type=None):
print "getting value:", self.s, "length:", self.l
return self.l

m=Desc('abc')
l=m.l


creating value: abc
id(self.l) 10049688setting value: test string length: 11

However, the attribute ta.l didn't get updated:
3

This is so much weirder that object id of ta.l is the same as id of
instance of descriptor:
10049688

A setter function should have updated self.l just like it updated
self.s:

def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l

Yet it didn't happen.

From my POV, the main benefit of a descriptor lies in its side effect:

on dot access (getting/setting) I can get other attributes updated
automatically: say, in class of Squares I get area automatically
updated on updating side, etc.

Yet, I'm struggling with getting it done in Python. Descriptors are a
great idea, but I would like to see them implemented in Python in a
way that makes it easier to get desireable side effects.

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors.

For example, let's take a simple example where you set an attribute s
to a string and have another attribute l set automatically to its
length.

def __init__(self,val):
self.s=val
self.l=len(val)
print "creating value: ", self.s
print "id(self.l)", id(self.l)
def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l
def __get__(self, obj, type=None):
print "getting value:", self.s, "length:", self.l
return self.l


m=Desc('abc')
l=m.l

First point : I don't get why Desc and some derive from str. Second
point: I don't get why you're storing the value and it's length in the
descriptor itself - obviously, you can't expect one descriptor instance
to be mapped to 2 distinct attributes. Third point: you understand that,
the way you wrote it, your descriptor will behave as a class (ie:shared)
attribute, don't you. Fourth point: if you hope some.l to be rebound
when m.l is, then you should learn Python basics before trying to jump
into descriptors.

The obvious, simple way to some your problem is to use a couple of
properties:

class Some(object):
@apply
def m():
def fget(self):
return self._m
def fset(self, val):
self._m = val
self._l = len(val)
return property(**locals())
@apply
def l():
def fget(self):
return self._l
def fset(self):
raise AttributeError("%s.l is readonly" % self)
def __init__(self, m):
self.m = m

Now if you absolutely insist on using custom descriptors, you'll need
two of them: one to manage access to s, and the second to manage access
to l (which btw is a very bad name):

class DescS(object):
def __init__(self, val):
self._default = val

def __set___(self, obj, val):
obj._s = val
obj._l = len(val)

def __get__(self, obj, cls):
if obj is None:
return self # or self._default, or whatever
try:
return obj._s
except AttributeError:
return self._default


class DescL(object):
def __init__(self, descS):
self._descS = descS
def __get__(self, obj, cls):
if obj is None:
return self # or self._default, or whatever
try:
return obj._l
except AttributeError:
return len(self._descS._default)


class Test(object):
m = DescS('abc')
l = DescL(m)

(nb : not tested)

 

[Rich Harkins]

Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors.
[snip]

A setter function should have updated self.l just like it updated
self.s:

def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l

Yet it didn't happen.

[snip]

I noticed that Python will block all attribute overrides (either via
__dict__ through setattr) if the property has a __set__ method. The
standard property has this method and there is no way that I can find to
defeat it. So, here is what I use:

class ConstProperty(object):
"""
Provides a property that keeps its return value. The function will
only be called on the first access. After that the same value can
be used over and over again with no function call penalty. If the
cached value needs to be cleared, simply del the attribute.
... def __init__(self, x):
... self.x = x
... @ConstProperty
... def y(self):
... print "HERE"
... return self.x ** 2
... HERE
25 25
"""

def __init__(self, fn):
self.fn = fn

def __get__(self, target, cls=None):
if target is None:
return self.fn # Helps pydoc
else:
obj = self.fn(target)
setattr(target, self.fn.__name__, obj)
return obj

This is a little different than what you originally posted, but
hopefully it is close enough to be helpful.

Cheers!
Rich

 

[Bruno Desthuilliers]

Rich Harkins a écrit :

Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors.
[snip]

A setter function should have updated self.l just like it updated
self.s:

def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l

Yet it didn't happen.

[snip]

I noticed that Python will block all attribute overrides (either via
__dict__ through setattr) if the property has a __set__ method.

It doesn't "block", it controls access to... Of course, if the __set__
method is a no-op, then nothing will happen.

The
standard property has this method and there is no way that I can find to
defeat it.

"defeat" ? Why don't you just pass the appropriate fset function to
property ?

So, here is what I use:

class ConstProperty(object):
"""
Provides a property that keeps its return value. The function will
only be called on the first access. After that the same value can
be used over and over again with no function call penalty. If the
cached value needs to be cleared, simply del the attribute.

... def __init__(self, x):
... self.x = x
... @ConstProperty
... def y(self):
... print "HERE"
... return self.x ** 2
...
25
"""

def __init__(self, fn):
self.fn = fn

def __get__(self, target, cls=None):
if target is None:
return self.fn # Helps pydoc
else:
obj = self.fn(target)
setattr(target, self.fn.__name__, obj)
return obj

HERE
25

I'm sorry, but this looks like a very complicated way to do a simple thing:

class MySimpleClass(object):
def __init__(self, x):
self.x = x
self.y = x ** 2

 

[Rich Harkins]

Bruno Desthuilliers wrote:
[snip]

I'm sorry, but this looks like a very complicated way to do a simple thing:

class MySimpleClass(object):
def __init__(self, x):
self.x = x
self.y = x ** 2

Sure, for the absurdly simplified case I posed as an example.

Here's another:

class Path(tuple):
@ConstProperty
def pathstr(self):
print "DEBUG: Generating string"
return '/'.join(self)

def __add__(self, other):
if isinstance(other, tuple):
return Path(tuple.__add__(self, other))
else:
return Path(tuple.__add__(self, (other,)))
DEBUG: Generating string
/x/y/z/x/y/z

Basically, you can use ConstProperty above for items you don't want to
calculate automatically, but only when someone actually WANTS it. After
it is applied, then the penalties for function call of the property and
the computation are wiped out once the second access is requested.

Now, in the original example, len() might be considered too little for
this use and should be just generated in the constructor "for free".
OTOH, that assumes that __len__ hasn't been overridden to do something
more complicated and time consuming. If the antecedent object is
static, and the derivative consequent is also static, then ConstProperty
works very well and shouldn't cost more on the first access than any
other built-in property function.

BTW, another use is to avoid creating lots of unnecessary objects for
free unless they are accessed. Another quickie example:

class Node(object):
hasChildList = False
hasAttributesDict = False

@ConstProperty
def children(self):
self.hasChildList = True
return []

@ConstProperty
def attributes(self):
self.hasAttributesDict = True
return {}

The extra class/object attributes can be used to test for whether the
associated objects were created. When used in a large tree, not
creating a lot of extra lists and dictionaries can save a lot of memory
and CPU as the children and attributes are not created or explored
unless they were manipulated.

Rich

 

[Rich Harkins]

Rich Harkins a écrit :

Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors. [snip]

A setter function should have updated self.l just like it updated
self.s:

def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l

Yet it didn't happen.

[snip]

I noticed that Python will block all attribute overrides (either via
__dict__ through setattr) if the property has a __set__ method.

It doesn't "block", it controls access to... Of course, if the __set__
method is a no-op, then nothing will happen.

The
standard property has this method and there is no way that I can find to
defeat it.

"defeat" ? Why don't you just pass the appropriate fset function to
property ?

So, here is what I use:

class ConstProperty(object):
"""
Provides a property that keeps its return value. The function will
only be called on the first access. After that the same value can
be used over and over again with no function call penalty. If the
cached value needs to be cleared, simply del the attribute.

... def __init__(self, x):
... self.x = x
... @ConstProperty
... def y(self):
... print "HERE"
... return self.x ** 2
...
25
"""

def __init__(self, fn):
self.fn = fn

def __get__(self, target, cls=None):
if target is None:
return self.fn # Helps pydoc
else:
obj = self.fn(target)
setattr(target, self.fn.__name__, obj)
return obj

HERE
25

I'm sorry, but this looks like a very complicated way to do a simple thing:

class MySimpleClass(object):
def __init__(self, x):
self.x = x
self.y = x ** 2

 

[Bruno Desthuilliers]

Rich Harkins a écrit :

Bruno Desthuilliers wrote:
[snip]

I'm sorry, but this looks like a very complicated way to do a simple thing:

class MySimpleClass(object):
def __init__(self, x):
self.x = x
self.y = x ** 2

Sure, for the absurdly simplified case I posed as an example.

Here's another:

class Path(tuple):

Ok, for an immutable type, it might eventually work.

@ConstProperty
def pathstr(self):
print "DEBUG: Generating string"
return '/'.join(self)

import os.path
help(os.path)

def __add__(self, other):
if isinstance(other, tuple):
return Path(tuple.__add__(self, other))
else:
return Path(tuple.__add__(self, (other,)))
>

DEBUG: Generating string
/x/y/z

path.pathstr

/x/y/z

>>> p = Path(('home', 'bruno'))
>>> p += ['toto', 'tata']
>>> p.pathstr

DEBUG: Generating string
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/tmp/python-8690chu", line 31, in __get__
File "/usr/tmp/python-8690chu", line 40, in pathstr
TypeError: sequence item 2: expected string, list found

Basically, you can use ConstProperty above for items you don't want to
calculate automatically, but only when someone actually WANTS it.

Which is easy to do with properties too.

After
it is applied, then the penalties for function call of the property and
the computation are wiped out once the second access is requested.

Agreed. But I wouldn't use such a scheme for mutable types - which are
still the common case.

Now, in the original example, len() might be considered too little for
this use and should be just generated in the constructor "for free".
OTOH, that assumes that __len__ hasn't been overridden to do something
more complicated and time consuming. If the antecedent object is
static, and the derivative consequent is also static,

You mean 'immutable', I assume...

then ConstProperty
works very well and shouldn't cost more on the first access than any
other built-in property function.

BTW, another use is to avoid creating lots of unnecessary objects for
free unless they are accessed. Another quickie example:

class Node(object):
hasChildList = False
hasAttributesDict = False

@ConstProperty
def children(self):
self.hasChildList = True
return []

@ConstProperty
def attributes(self):
self.hasAttributesDict = True
return {}

Hmm... Perhaps not such a bad idea after all !-)

 

[Rich Harkins]

Which is easy to do with properties too.

True enough. It's the caching of the return value that's the value add
of course.

Agreed. But I wouldn't use such a scheme for mutable types - which are
still the common case.

In many cases, yeah. Though I use a lot of immutable stuff in some of
my pet projects and such. ConstProperty is definitely not meant as a
replacement for property, only when something constant can be derived
from something else constant, especially when the derivation is expensive.

You mean 'immutable', I assume...

Yeah, that's probably the better term.

[snip]

Again, I've used it quite a bit for various things and it's worked well
for the sort of thing the OP was requesting. Of course, your mileage
may vary.

Cheers!
Rich

PS: Sorry about the weird reposts. Thunderbird chaos.