cannot create my own dict

[A.T.Hofkamp]

Hello all,

This morning I tried to create my own read-only dictionary, and failed
miserably.
I don't understand why, can somebody enlighten me?

Below is a brute-force experiment that cannot deal with "x in obj", plz read
the explanation below the code:
--------------------------------
class myowndict(object):
def __init__(self, mydict):
self.mydict = mydict

# Below is produced with
# print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
# for v in dir(dict)])
# commented-out functions done by hand
#
#self.__class__ = self.mydict.__class__
self.__cmp__ = self.mydict.__cmp__
self.__contains__ = self.mydict.__contains__
self.__delattr__ = self.mydict.__delattr__
self.__delitem__ = self.mydict.__delitem__
#self.__doc__ = self.mydict.__doc__
self.__eq__ = self.mydict.__eq__
self.__ge__ = self.mydict.__ge__
self.__getattribute__ = self.mydict.__getattribute__
self.__getitem__ = self.mydict.__getitem__
self.__gt__ = self.mydict.__gt__
self.__hash__ = self.mydict.__hash__
#self.__init__ = self.mydict.__init__
self.__iter__ = self.mydict.__iter__
self.__le__ = self.mydict.__le__
self.__len__ = self.mydict.__len__
self.__lt__ = self.mydict.__lt__
self.__ne__ = self.mydict.__ne__
#self.__new__ = self.mydict.__new__
self.__reduce__ = self.mydict.__reduce__
self.__reduce_ex__ = self.mydict.__reduce_ex__
self.__repr__ = self.mydict.__repr__
self.__setattr__ = self.mydict.__setattr__
self.__setitem__ = self.mydict.__setitem__
self.__str__ = self.mydict.__str__
self.clear = self.mydict.clear
self.copy = self.mydict.copy
self.fromkeys = self.mydict.fromkeys
self.get = self.mydict.get
self.has_key = self.mydict.has_key
self.items = self.mydict.items
self.iteritems = self.mydict.iteritems
self.iterkeys = self.mydict.iterkeys
self.itervalues = self.mydict.itervalues
self.keys = self.mydict.keys
self.pop = self.mydict.pop
self.popitem = self.mydict.popitem
self.setdefault = self.mydict.setdefault
self.update = self.mydict.update
self.values = self.mydict.values

# end of __init__

if __name__ == '__main__':
fd = myowndict({1:10})
print 1 in fd # FAILS! (with "TypeError: iterable argument required")
--------------------------------

I wanted to make my own dictionary. However, a simple element test failed
(after implementing various __*__ functions), and I cannot figure out why.

The above code is a brute force attempt, where I forward all methods (except
__class__, __doc__, __init__, and __new__) to my local 'mydict' object.

IT STILL FAILS.

So if copying all methods of a native dictionary is not enough, what should I
do to make my class work as a dictionary WITHOUT deriving from dict (which will
obviously work).



Sincerely,
Albert

 

[Steve Holden]

Hello all,

This morning I tried to create my own read-only dictionary, and failed
miserably.
I don't understand why, can somebody enlighten me?

Below is a brute-force experiment that cannot deal with "x in obj", plz read
the explanation below the code:
--------------------------------
class myowndict(object):
def __init__(self, mydict):
self.mydict = mydict

# Below is produced with
# print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
# for v in dir(dict)])
# commented-out functions done by hand
#
#self.__class__ = self.mydict.__class__
self.__cmp__ = self.mydict.__cmp__
self.__contains__ = self.mydict.__contains__
self.__delattr__ = self.mydict.__delattr__
self.__delitem__ = self.mydict.__delitem__
#self.__doc__ = self.mydict.__doc__
self.__eq__ = self.mydict.__eq__
self.__ge__ = self.mydict.__ge__
self.__getattribute__ = self.mydict.__getattribute__
self.__getitem__ = self.mydict.__getitem__
self.__gt__ = self.mydict.__gt__
self.__hash__ = self.mydict.__hash__
#self.__init__ = self.mydict.__init__
self.__iter__ = self.mydict.__iter__
self.__le__ = self.mydict.__le__
self.__len__ = self.mydict.__len__
self.__lt__ = self.mydict.__lt__
self.__ne__ = self.mydict.__ne__
#self.__new__ = self.mydict.__new__
self.__reduce__ = self.mydict.__reduce__
self.__reduce_ex__ = self.mydict.__reduce_ex__
self.__repr__ = self.mydict.__repr__
self.__setattr__ = self.mydict.__setattr__
self.__setitem__ = self.mydict.__setitem__
self.__str__ = self.mydict.__str__
self.clear = self.mydict.clear
self.copy = self.mydict.copy
self.fromkeys = self.mydict.fromkeys
self.get = self.mydict.get
self.has_key = self.mydict.has_key
self.items = self.mydict.items
self.iteritems = self.mydict.iteritems
self.iterkeys = self.mydict.iterkeys
self.itervalues = self.mydict.itervalues
self.keys = self.mydict.keys
self.pop = self.mydict.pop
self.popitem = self.mydict.popitem
self.setdefault = self.mydict.setdefault
self.update = self.mydict.update
self.values = self.mydict.values

# end of __init__

if __name__ == '__main__':
fd = myowndict({1:10})
print 1 in fd # FAILS! (with "TypeError: iterable argument required")
--------------------------------

I wanted to make my own dictionary. However, a simple element test failed
(after implementing various __*__ functions), and I cannot figure out why.

The above code is a brute force attempt, where I forward all methods (except
__class__, __doc__, __init__, and __new__) to my local 'mydict' object.

IT STILL FAILS.

So if copying all methods of a native dictionary is not enough, what should I
do to make my class work as a dictionary WITHOUT deriving from dict (which will
obviously work).

You have to overwrite the "__new__" method to return an object of your
new type.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline

 

[Peter Otten]

This morning I tried to create my own read-only dictionary, and failed
miserably.
I don't understand why, can somebody enlighten me?

Below is a brute-force experiment that cannot deal with "x in obj", plz read
the explanation below the code:
--------------------------------
class myowndict(object):
def __init__(self, mydict):
self.mydict = mydict

# Below is produced with
# print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
# for v in dir(dict)])
# commented-out functions done by hand
#
#self.__class__ = self.mydict.__class__
self.__cmp__ = self.mydict.__cmp__
self.__contains__ = self.mydict.__contains__
self.__delattr__ = self.mydict.__delattr__
self.__delitem__ = self.mydict.__delitem__
#self.__doc__ = self.mydict.__doc__
self.__eq__ = self.mydict.__eq__
self.__ge__ = self.mydict.__ge__
self.__getattribute__ = self.mydict.__getattribute__
self.__getitem__ = self.mydict.__getitem__
self.__gt__ = self.mydict.__gt__
self.__hash__ = self.mydict.__hash__
#self.__init__ = self.mydict.__init__
self.__iter__ = self.mydict.__iter__
self.__le__ = self.mydict.__le__
self.__len__ = self.mydict.__len__
self.__lt__ = self.mydict.__lt__
self.__ne__ = self.mydict.__ne__
#self.__new__ = self.mydict.__new__
self.__reduce__ = self.mydict.__reduce__
self.__reduce_ex__ = self.mydict.__reduce_ex__
self.__repr__ = self.mydict.__repr__
self.__setattr__ = self.mydict.__setattr__
self.__setitem__ = self.mydict.__setitem__
self.__str__ = self.mydict.__str__
self.clear = self.mydict.clear
self.copy = self.mydict.copy
self.fromkeys = self.mydict.fromkeys
self.get = self.mydict.get
self.has_key = self.mydict.has_key
self.items = self.mydict.items
self.iteritems = self.mydict.iteritems
self.iterkeys = self.mydict.iterkeys
self.itervalues = self.mydict.itervalues
self.keys = self.mydict.keys
self.pop = self.mydict.pop
self.popitem = self.mydict.popitem
self.setdefault = self.mydict.setdefault
self.update = self.mydict.update
self.values = self.mydict.values

# end of __init__

if __name__ == '__main__':
fd = myowndict({1:10})
print 1 in fd # FAILS! (with "TypeError: iterable argument required")
--------------------------------

I wanted to make my own dictionary. However, a simple element test failed
(after implementing various __*__ functions), and I cannot figure out why.

The above code is a brute force attempt, where I forward all methods (except
__class__, __doc__, __init__, and __new__) to my local 'mydict' object.

IT STILL FAILS.

__special__ methods are looked up in the class, never in the instance for
newstyle classes:
.... __contains__ = class_method
.... copy = class_method
.... def __init__(self):
.... self.__contains__ = inst_method
.... self.copy = inst_method
....

class
False
instance

So if copying all methods of a native dictionary is not enough, what should I
do to make my class work as a dictionary WITHOUT deriving from dict (which will
obviously work).

Write wrappers like

def __contains__(self, value):
return value in self.mydict

Peter

 

[Bruno Desthuilliers]

A.T.Hofkamp a écrit :

Hello all,

This morning I tried to create my own read-only dictionary, and failed
miserably.
I don't understand why, can somebody enlighten me?

Below is a brute-force experiment that cannot deal with "x in obj", plz read
the explanation below the code:

Why don't you just subclass dict and override the needed method(s) ?

class ReadOnlyDict(dict):
@classmethod
def _raise(cls):
raise AttributeError("%s is read-only" % cls.__name__)

def __setitem__(self, name, val):
self._raise()

def setdefault(self, name, value=None):
self._raise()

# TODO : check if I missed some other setter ?

def test(d):
print d
try:
d['c'] = 3
except AttributeError, e:
print "__setitem__ ok, got : %s" % e
else:
assert('failed on __setitem__')
try:
d.setdefault('c', 3)
except AttributeError, e:
print "setdefault ok, got : %s" % e
else:
assert('failed on setdefault')

if __name__ == '__main__':
d1 = ReadOnlyDict(a=1, b=2)
test(d1)

d2 = ReadOnlyDict({'a':1, 'b':2})
test(d2)

d3 = ReadOnlyDict((c, i+1) for i, c in enumerate('ab'))
test(d3)

d4 = ReadOnlyDict(**d1)
test(d4)

NB : works fine so far, but choke on fromkeys, which obviously uses
__setitem__

# Below is produced with
# print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
# for v in dir(dict)])
# commented-out functions done by hand
#
#self.__class__ = self.mydict.__class__
self.__cmp__ = self.mydict.__cmp__
self.__contains__ = self.mydict.__contains__

(snip lots of useless code)

May I suggest that you learn to use __getattr__ and __setattr__ ?-)

HTH

 

[Bruno Desthuilliers]

Steve Holden a écrit :

A.T.Hofkamp wrote:
(snip)

You have to overwrite the "__new__" method to return an object of your
new type.

???

This applies to subclasses of immutable types, but I don't see the point
here. The problem is - as Peter pointed out - that __magic__ methods are
not looked up in instances when using new-style classes.

 

[Bruno Desthuilliers]

Bruno Desthuilliers a écrit :

A.T.Hofkamp a écrit :
(snip)

# Below is produced with
# print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
# for v in dir(dict)])
# commented-out functions done by hand
#
#self.__class__ = self.mydict.__class__
self.__cmp__ = self.mydict.__cmp__
self.__contains__ = self.mydict.__contains__

(snip lots of useless code)

May I suggest that you learn to use __getattr__ and __setattr__ ?-)

Mmmm... Should have tested before posting this. Please ignore this
remark, looks like __getattr__ is not called for __magic__ methods. Doh

 

[Bruno Desthuilliers]

A.T.Hofkamp a écrit :

Hello all,

This morning I tried to create my own read-only dictionary, and failed
miserably.
I don't understand why, can somebody enlighten me?
(snip)

So if copying all methods of a native dictionary is not enough, what should I
do to make my class work as a dictionary WITHOUT deriving from dict (which will
obviously work).

Sorry, I missed this last requirement. BTW, why don't you want to
subclass dict ?

Anyway, something like the following should do the trick
(NB : Q&D, needs at least some testing):

class ReadOnlyDict(object):
def __init__(self, *args, **kw):
self._dict = dict(*args, **kw)

def __getattr__(self, name):
if name in ('__setitem__', 'setdefault'):
raise AttributeError("%s is read-only" % \
self.__class__.__name__)
return getattr(self._dict, name)

for name in dir(dict):
if name.startswith('__') \
and name not in ('__new__', '__init__',
'__setitem__', '__class__', '__dict__'):
exec("%s = dict.%s" % (name, name))

 

[Steven D'Aprano]

Sorry, I missed this last requirement. BTW, why don't you want to
subclass dict ?

Possibly because it is a learning exercise?

 

[A.T.Hofkamp]

A.T.Hofkamp a écrit :

Hello all,

Thanks for all the suggestions, the cause of the problem seems to be that I
assumed that I can export magic methods from a member. Apparently, that is not
true :-(

Sorry, I missed this last requirement. BTW, why don't you want to
subclass dict ?

The reason for this problem is that we are busy re-structuring a large Python
program, and we want to ensure nobody is messing up our data structures (at
least not until we are finished).
Most data types have a nice frozen variant (set -> frozenset, list -> tuple,
__setattr__ override), but dictionaries do not.

As a result I wanted to have a read-only dictionary.
There is one in the Cook book, but it uses property's that I don't understand
and they are also not in de Python language reference (at least I couldn't find
them in the index and not in the table of contents).
I don't like code that I don't understand (in particular when bugs in that code
will be nasty to debug), so I decided to write my own, not in the last place,
because I expected it to be simple in Python.
I can derive from dict, but the problem with that is that I start with a
read/write dictionary, and I can only hope to plug all holes to prevent my data
from leaking out.
By starting from 'object', I certainly don't have that problem, I start with a
closed bucket and punch holes in it in a controlled way.
(I rather have the program drop dead due to not having enough access than
have it continue with too much access causing havoc 500 statements later in a
totally unrelated area.)

Rather than write a bunch of code like

def __contains__(self, val):
return val in self.mydict

I thought I'd simply do

self.__contains__ == self.d.__contains__

which is exactly the same but less work (or so I thought), and possibly
slightly faster.

Well, no such luck thus :-(


Tnx for clearing up the problem,
Albert

 

[John J. Lee]

I can derive from dict, but the problem with that is that I start with a
read/write dictionary, and I can only hope to plug all holes to prevent my data
from leaking out.
By starting from 'object', I certainly don't have that problem, I start with a
closed bucket and punch holes in it in a controlled way.
(I rather have the program drop dead due to not having enough access than
have it continue with too much access causing havoc 500 statements later in a
totally unrelated area.)

Rather than write a bunch of code like

def __contains__(self, val):
return val in self.mydict

I thought I'd simply do

self.__contains__ == self.d.__contains__

which is exactly the same but less work (or so I thought), and possibly
slightly faster.

Well, no such luck thus :-(

UserDict.DictMixin

http://docs.python.org/lib/module-UserDict.html


John

 

[David]

The reason for this problem is that we are busy re-structuring a large Python

program, and we want to ensure nobody is messing up our data structures (at
least not until we are finished).

python-egenix-mxproxy (Debian package name) may be useful.

http://www.egenix.com/products/python/mxBase/