Why property works only for objects?

M

Michal Kwiatkowski

Hi,

Code below shows that property() works only if you use it within a class.

------------------------------------------------
class A(object):
pass

a = A()
a.y = 7

def method_get(self):
return self.y

a.x = property(method_get)
print a.x # => <property object at 0xb7dc1c84>

A.x = property(method_get)
print a.x # => 7
------------------------------------------------

Is there any method of making descriptors on per-object basis? I would
like to customize x access in different objects of class A. Is this
possible and how?

mk
 
S

Steven Bethard

Michal said:
Code below shows that property() works only if you use it within a class.

Yes, descriptors are only applied at the class level (that is, only
class objects call the __get__ methods).
Is there any method of making descriptors on per-object basis?

I'm still not convinced that you actually want to, but you can write
your own descriptor to dispatch to the instance object (instead of the
type):
.... def __init__(self, func_name):
.... self.func_name = func_name
.... def __get__(self, obj, type=None):
.... if obj is None:
.... return self
.... return getattr(obj, self.func_name)(obj)
........ x = InstanceProperty('_x')
....Traceback (most recent call last):
File "<interactive input>", line 1, in ?
.... return 42
....42


STeVe
 
M

Michal Kwiatkowski

Steven Bethard napisa³(a):
I'm still not convinced that you actually want to, but you can write
your own descriptor to dispatch to the instance object (instead of the
type):

Ok, this works for attributes I know a name of at class definition. What
about other situations? I could possibly want to have different sets of
attributes for instances of a class. Is it still possible to do? I don't
also like current solution, as it wraps things around, creating another
attribute ('_x') in the process. Aren't there any cleaner solutions?

The problem is I have an instance of a given class (say BaseClass) and I
want it to implement some attribute accesses as method calls. I'm not a
creator of this object, so changing definition of BaseClass or
subclassing it is not an option. Code below doesn't work, but shows my
intention:

# obj is instance of BaseClass
def get_x(self):
# ...
def set_x(self, value):
# ...
obj.x = property(get_x, set_x)

Changing __setattr__/__getattr__ of an object also doesn't work for the
same reason: dictionary lookup is made only in class attributes,
ommiting object attributes. It's confusing, because writting obj.x and
obj.__getattribute__('x') gives a different results. There also seems
not to be any clear way to define methods for objects. Something like:

class C(object):
pass

def method(self):
return self.x

c = c.x
c.method = method
c.method() # => AttributeError

So another question arise. Is it possible to make function a method (so
it will receive calling object as first argument)?

mk
 
A

Alex Martelli

Michal Kwiatkowski said:
The problem is I have an instance of a given class (say BaseClass) and I
want it to implement some attribute accesses as method calls. I'm not a
creator of this object, so changing definition of BaseClass or
subclassing it is not an option.

Wrong! Of _course_ it's an option -- why do you think it matters at all
whether you're the creator of this object?!
Code below doesn't work, but shows my
intention:

# obj is instance of BaseClass
def get_x(self):
# ...
def set_x(self, value):
# ...
obj.x = property(get_x, set_x)

def insert_property(obj, name, getter, setter):
class sub(obj.__class__): pass
setattr(sub, name, property(getter, setter))
obj.__class__ = sub

See? Of COURSE you can subclass -- not hard at all, really.


Alex
 
A

Alex Martelli

Michal Kwiatkowski said:
So another question arise. Is it possible to make function a method (so
it will receive calling object as first argument)?

Sure, impor types then call types.MethodType:

f = types.MethodType(f, obj, someclass)

(f.__get__ is also fine for Python-coded functions) -- make sure that
someclass is obj's class or some ancestor of it, of course.


Alex
 
M

Michal Kwiatkowski

Alex Martelli napisa³(a):
Wrong! Of _course_ it's an option -- why do you think it matters at all
whether you're the creator of this object?!

Statically typed languages background. Sorry. ;)
def insert_property(obj, name, getter, setter):
class sub(obj.__class__): pass
setattr(sub, name, property(getter, setter))
obj.__class__ = sub

See? Of COURSE you can subclass -- not hard at all, really.

Let me understand it clearly. If I change __class__ of an object,
existing attributes (so methods as well) of an object are still
accessible the same way and don't change its values. Only resolution of
attributes/methods not found in object is changed, as it uses new
version of __class__ to lookup names. Is this right?

mk
 
M

Michal Kwiatkowski

Alex Martelli napisa³(a):
Sure, impor types then call types.MethodType:

f = types.MethodType(f, obj, someclass)

(f.__get__ is also fine for Python-coded functions) -- make sure that
someclass is obj's class or some ancestor of it, of course.

I wasn't aware of types module. Thanks for your reply.

mk
 
M

Michal Kwiatkowski

Bruno Desthuilliers napisa³(a):
Attributes, yes. Not methods. Methods are looked up in the class.

My experience shows exactly the opposite. Any attribute/method you try
to access is first looked up in object dictionary, then inside class
definition.

import types

class C(object):
def f(self):
print "old method f()"

obj = C()

def f(self):
print "new method f()"

obj.f = types.MethodType(f, C)

obj.f() # => "new method f()"

Since that works, intuitively for me would be to assign object's
descriptors like that:

obj.x = property(types.MethodType(lambda self: 42, C))

But I just get a property object. So, it seems descriptors have little
bit of magic, as they don't work identically for classes and objects.

The same goes for special methods and attributes (written as __*__). So
I cannot change __getattr__/__setattr__/__metaclass__ or any other
attribute that starts with __ for a single object. It's not so bad
except for situations were class of an object defines its own
__getattribute__ method, which takes control of an object from us. To
get/set any attribute of an object we must use object type methods:

class C(object):
def __getattribute__(self, name):
return 42

obj = C()

obj.a = 5

print obj.a # => 42
print object.__getattribute__(obj, 'a') # => 5

I gets even more strange when you try to modify, say __len__ or
__repr__. Docstring for object.__repr__ says:
"x.__repr__() <==> repr(x)" which doesn't seem to be always true:

class C(object):
def __repr__(self):
return "class repr"

obj = C()
obj.__repr__ = types.MethodType(lambda self: "instance repr", C)

print repr(obj) # => class repr
print obj.__repr__() # => instance repr

Maybe the manual should say "x.__class__.__repr__() <==> repr(x)" instead?

I'm trying to understand attributes lookups made by Python, having
properties and special methods in mind. So far I've come up with kind of
reasoning I've coded below. I would appreciate any comments and
suggestions.

def lookup_name(obj, name):
get = lambda obj, name: object.__getattribute__(obj, name)
has = lambda obj, name: name in get(obj, '__dict__')

# assume C is a new style class
C = get(obj, '__class__')

# 1) use class' __getattribute__ method
try:
if has(C, '__getattribute__'):
return get(C, '__getattribute__')(obj, name)
except AttributeError: pass

# 2) lookup in object's dictionary
try:
if has(obj, name):
return get(obj, name)
except AttributeError: pass

# 3) lookup in classes
for c in obj.__class__.mro():
try:
if has(c, name):
desc = get(c, name)
# 3a) handle descriptors
try:
return get(desc, '__get__')(obj)
except: pass
# 3b) no descriptors -> use value
return desc
except AttributeError: pass

raise AttributeError, "Not found!"

mk
 
A

Alex Martelli

Michal Kwiatkowski said:
Let me understand it clearly. If I change __class__ of an object,
existing attributes (so methods as well) of an object are still
accessible the same way and don't change its values. Only resolution of
attributes/methods not found in object is changed, as it uses new
version of __class__ to lookup names. Is this right?

Attributes of the _instance_ are unchanged. Anything that would
normally be looked up in the OLD __class__, such as methods, is gone:
which is why you normally set as the NEW __class__ some subclass of the
old one, so that nothing is really gone;-).


Alex
 
B

Bruno Desthuilliers

Michal Kwiatkowski a écrit :
Steven Bethard napisa³(a):



Ok, this works for attributes I know a name of at class definition. What
about other situations? I could possibly want to have different sets of
attributes for instances of a class.

Then you want a different class for each different set of attributes.
Is it still possible to do? I don't
also like current solution, as it wraps things around, creating another
attribute ('_x') in the process. Aren't there any cleaner solutions?

The problem is I have an instance of a given class (say BaseClass) and I
want it to implement some attribute accesses as method calls. I'm not a
creator of this object, so changing definition of BaseClass or
subclassing it is not an option.

What you want is delegation. Which is pretty easy with Python, look at
__getattr__ and __setattr__.
Code below doesn't work, but shows my
intention:

# obj is instance of BaseClass
def get_x(self):
# ...
def set_x(self, value):
# ...
obj.x = property(get_x, set_x)

class ObjWrapper(object):
def __init__(self, obj):
self.obj = obj

# special case x
def _get_x(self):
return self.obj.x

def _set_x(self, value):
self.obj.x = value

x = property(fget=_get_x, fset=_set_x)

# default lookup, directly delegate to obj
def __getattr__(self, name):
return getattr(self.obj, name)

def __setattr__(self, name, value):
# this one is a bit more tricky
# and I don't remember it right now,
# but it's in the fine manual anyway

obj = ObjWrapper(obj)
obj.x # calls ObjWrapper._get_x()

Changing __setattr__/__getattr__ of an object also doesn't work for the
same reason: dictionary lookup is made only in class attributes,
ommiting object attributes. It's confusing, because writting obj.x and
obj.__getattribute__('x') gives a different results. There also seems
not to be any clear way to define methods for objects. Something like:

class C(object):
pass

def method(self):
return self.x

c = c.x
c.method = method
c.method() # => AttributeError

import types
c.method = types.MethodType(method, c, c.__class__)
So another question arise. Is it possible to make function a method (so
it will receive calling object as first argument)?

Yeps, just wrap it in a Method object (cf above)
 
B

Bruno Desthuilliers

Michal Kwiatkowski a écrit :
Alex Martelli napisa³(a):



Let me understand it clearly. If I change __class__ of an object,
existing attributes (so methods as well)
of an object are still
accessible the same way and don't change its values. Only resolution of
attributes/methods not found in object is changed, as it uses new
version of __class__ to lookup names. Is this right?

Attributes, yes. Not methods. Methods are looked up in the class. But in
the example above (brillant, as usual), Alex dynamically creates a
subclass of obj.__class__, so inheritence takes care of methods lookup.
 
A

Alex Martelli

Michal Kwiatkowski said:
I'm trying to understand attributes lookups made by Python, having
properties and special methods in mind. So far I've come up with kind of
reasoning I've coded below. I would appreciate any comments and
suggestions.

First, let's forget legacy-style classes, existing only for backwards
compatibility, and focus on new-style ones exclusively -- never use
legacy classes if you can avoid that.

Descriptors come in two varieties: overriding and non-overriding. Both
kinds have a __get__ method (that's what makes them descriptors).
Overriding ones also have a __set__, non-overriding ones don't. (Until
pretty recently, the two kinds were known as data and non-data, but the
new terminology of overriding and non-overriding is much clearer).

I covered descriptors in one of my talks at Pycon '05 and elsewhere; you
can find just about all of my talks by starting at www.aleax.it (mostly
just PDF forms of the slides in my presentations). I cover them in quite
a central role in the forthcoming 2nd edition of Python in a Nutshell,
too. But, summarizing:

overriding descriptors are FIRST looked up in the class (class overrides
instance); non-overriding descriptors, first in the instance (class does
not override). Functions are NON-overriding descriptors.

Special methods IMPLICITLY looked up by Python ALWAYS go to the class
ONLY -- adding an x.__getitem__ per-class attribute that happens to be
callable doesn't mean that attribute is gonna handle z=x[y] and the like
(in legacy-style classes the rule was different, but that led to a host
of complications that we're much better off without!).


Alex
 
B

Bruno Desthuilliers

Michal Kwiatkowski a écrit :
Bruno Desthuilliers napisa³(a):



My experience shows exactly the opposite. Any attribute/method you try
to access is first looked up in object dictionary, then inside class
definition.

Yes, that's what I said.

You wrote:
"""
existing attributes (so methods as well) of an object are still
accessible the same way and don't change its values
"""

Attributes (I mean instance attributes) living in the object's dict,
they aren't impacted by the runtime class change. Methods being in the
most common case (I'd say > 99.9 %) defined *in the class*, if you
change the class of an object, this is very likely to impact resolution
of methods lookup.

Now I agree that the statement "methods are looked up in the class" is
wrong. Methods are of course first looked up in the object, then in the
class. But the case of a method living in the object's dict is not that
common...
 
M

Michal Kwiatkowski

Alex Martelli napisa³(a):
First, let's forget legacy-style classes, existing only for backwards
compatibility, and focus on new-style ones exclusively -- never use
legacy classes if you can avoid that.

Ok, let's cover only new-style classes in our discussion.

I've read your comments and am on a way of reading your articles. Still,
with my current knowledge I'm trying to write pure python attributes
lookup function. I've failed for example given below:

class C(object):
__dict__ = {}

obj = C()
obj.a = 7
obj.__dict__ = {}
print object.__getattribute__(obj, '__dict__')
print object.__getattribute__(C, '__dict__')
print obj.a # => 7 !!!

First print returns "{}" and the second returns

{'__dict__': {},
'__module__': '__main__',
'__weakref__': <attribute '__weakref__' of 'C' objects>,
'__doc__': None}

Neither of them have "a" attribute. How come obj.a doesn't raise an
exception? Where obj.a is kept?

mk
 
A

Alex Martelli

Michal Kwiatkowski said:
class C(object):
__dict__ = {}

obj = C()
obj.a = 7
obj.__dict__ = {}
print object.__getattribute__(obj, '__dict__')
print object.__getattribute__(C, '__dict__')
print obj.a # => 7 !!!

First print returns "{}" and the second returns

{'__dict__': {},
'__module__': '__main__',
'__weakref__': <attribute '__weakref__' of 'C' objects>,
'__doc__': None}

Neither of them have "a" attribute. How come obj.a doesn't raise an
exception? Where obj.a is kept?

It's easier to trace if you use a unique value rather than 7 ...:
.... __dict__ = {}
.... [{'a': <object object at 0x3d438>}]

so, at this point, you know that obj.a is kept in a dictionary where
it's the only value. That's the dictionary you would USUALLY be able to
get to as obj.__dict__, but...:
{}

....the presence of '__dict__' as an entry in C is confusing the issue,
because that's what you get in this case as obj.__dict__.

C.__dict__ gives you a dictproxy, not a real dict, by the way:
obj.__dict__ is C.__dict__['__dict__']
True

The funny thing is that builtins like var, which should know better,
also get fooled...:
vars(obj) {}
vars(obj) is C.__dict__['__dict__']
True

....and so does the assignment to obj.__dict__...:
[{'a': <object object at 0x3d438>, '__dict__': {}}]

Now, both obj.a and obj.__dict__ are entries in a dictionary where
they're the only two entries -- exactly the dictionary that would
NORMALLY be obj.__dict__.

I think a fair case can be made that you've found a bug in Python here:
the existence of that __dict__ in C's class body is clearly causing
unintended anomalies. Fortunately, getattr and friends don't in fact
get confused, but vars does, as does assignment to obj.__dict__...


Alex
 
M

Michal Kwiatkowski

Alex Martelli napisa³(a):
{}

...the presence of '__dict__' as an entry in C is confusing the issue,
because that's what you get in this case as obj.__dict__.

It still bugs me. What's the actual procedure when doing attribute
assignment? I understand it like this:

obj.attr = value
* if instance class has __setattr__, call it
* else: if class has an attribute with name "attr" check if it's a
descriptor; when it's overriding descriptor, call its __set__
method, otherwise raise AttributeError
* else: bind "attr" as object attribute

I've initialized C.__dict__ as plain dictionary, so it's not a
descriptor. C also doesn't have __setattr__ method. I'm probably wrong,
as __dict__ is considered somewhat special, but I can't really
understand to which degree. Any other attribute I try to bind to object
will be stored inside object dictionary, without looking up class
dictionary. __dict__ is special, because it's *the* dictionary that
stores all references to object's attributes. So, when assigning to
__dict__ I would expect old dictionary to be simply replaced with this
one. I don't understand why class attributes has any factor in this.
C.__dict__ gives you a dictproxy, not a real dict, by the way:
obj.__dict__ is C.__dict__['__dict__']
True

Why is that? I've initialized it as dictionary. When the type was
changed and why? Is this used anywhere?
The funny thing is that builtins like var, which should know better,
also get fooled...:
vars(obj) {}
vars(obj) is C.__dict__['__dict__']
True

Docstring for vars says:
With an argument, equivalent to object.__dict__.

So this behavior simply conforms to the doc.
I think a fair case can be made that you've found a bug in Python here:
the existence of that __dict__ in C's class body is clearly causing
unintended anomalies. Fortunately, getattr and friends don't in fact
get confused, but vars does, as does assignment to obj.__dict__...

What solution do you propose?

mk
 
A

Alex Martelli

Michal Kwiatkowski said:
Alex Martelli napisa?(a):

It still bugs me. What's the actual procedure when doing attribute
assignment? I understand it like this:

obj.attr = value
* if instance class has __setattr__, call it
* else: if class has an attribute with name "attr" check if it's a
descriptor; when it's overriding descriptor, call its __set__
method, otherwise raise AttributeError
* else: bind "attr" as object attribute

Yes, this is correct.

I've initialized C.__dict__ as plain dictionary, so it's not a
descriptor. C also doesn't have __setattr__ method. I'm probably wrong,
as __dict__ is considered somewhat special, but I can't really
understand to which degree. Any other attribute I try to bind to object
will be stored inside object dictionary, without looking up class
dictionary. __dict__ is special, because it's *the* dictionary that
stores all references to object's attributes. So, when assigning to
__dict__ I would expect old dictionary to be simply replaced with this
one. I don't understand why class attributes has any factor in this.

As I said, it looks to me like you've found a bug here. __dict__ (like
all names starting and ending in double underscores) is special, in the
sense that Python can do with it whatever it wishes, but there is
neither any documentation nor any reason explaining the behavior you
have observed.
C.__dict__ gives you a dictproxy, not a real dict, by the way:
obj.__dict__ is C.__dict__['__dict__']
True

Why is that? I've initialized it as dictionary. When the type was
changed and why? Is this used anywhere?

That's type(C)'s doing, and as such, perfectly correct: C.__dict__ for
any type C always returns a dictproxy (so that Python, internally,
doesn't _have_ to use a real dict, but rather a structure of slots that
may be much more compact and fast to access).

The funny thing is that builtins like var, which should know better,
also get fooled...:
vars(obj) {}
vars(obj) is C.__dict__['__dict__']
True

Docstring for vars says:
With an argument, equivalent to object.__dict__.

So this behavior simply conforms to the doc.

But the behavior of obj.__dict__={} doesn't.

What solution do you propose?

Opening a bug report on the Python bugtracker would maximize the
likelihood that something gets done about this bug.


Alex
 
S

Shalabh Chaturvedi

Michal said:
Alex Martelli napisa³(a):

It still bugs me. What's the actual procedure when doing attribute
assignment? I understand it like this:

obj.attr = value
* if instance class has __setattr__, call it
* else: if class has an attribute with name "attr" check if it's a
descriptor; when it's overriding descriptor, call its __set__
method, otherwise raise AttributeError
* else: bind "attr" as object attribute

Here is a step-by-step description of what happens when you set or get
an attribute on an object:

http://cafepy.com/article/python_attributes_and_methods/ch01s05.html

Cheers,
Shalabh
 
M

Michal Kwiatkowski

Alex Martelli napisa³(a):
Yes, this is correct.

Can you also check my reasoning for getting attributes?

value = obj.attr
* if instance class has __getattribute__, call it
* else: lookup "attr" in all parent classes using class __mro__;
if it's a descriptor call its __get__ method, return its value
otherwise (when descriptor doesn't have __get__, it's unreadable
and AttributeError is raised)
* else: check instance __dict__ for "attr", return it when found
* else: lookup __getattr__ in instance class and call it when found
* else: raise AttributeError
Opening a bug report on the Python bugtracker would maximize the
likelihood that something gets done about this bug.

Bug submitted:

http://sourceforge.net/tracker/index.php?func=detail&aid=1448042&group_id=5470&atid=105470

mk
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,183
Messages
2,570,965
Members
47,511
Latest member
svareza

Latest Threads

Top