magical expanding hash

[braver]

I need a magical expanding hash with the following properties:

* it creates all intermediate keys

meh['foo']['bar] = 1

-- works even if meh['foo'] didn't exist before

* allows pushing new elements to leaves which are arrays

meh['foo']['list] << elem1
meh['foo']['list] << elem2

* allows incrementing numeric leaves

meh['foo']['count'] += 7

* serializable

I have such a class in ruby. Can python do that?

 

[James Stroud]

I need a magical expanding hash with the following properties:

* it creates all intermediate keys

meh['foo']['bar] = 1

-- works even if meh['foo'] didn't exist before

* allows pushing new elements to leaves which are arrays

meh['foo']['list] << elem1
meh['foo']['list] << elem2

* allows incrementing numeric leaves

meh['foo']['count'] += 7

* serializable

I have such a class in ruby. Can python do that?

Is this too magical?


class meh(dict):
def __getitem__(self, item):
if self.has_key(item):
return dict.__getitem__(self, item)
else:
anitem = meh()
dict.__setitem__(self, item, anitem)
return anitem


m = meh()

m['bob']['carol']['ted'] = 2

print m['bob']['carol']['ted']

 

[Paul Rubin]

I need a magical expanding hash with the following properties: ...
I have such a class in ruby. Can python do that?

Python's built-in dict objects don't do that but you could write such
a class pretty straightforwardly.

 

[Giovanni Bajo]

I need a magical expanding hash with the following properties:

* it creates all intermediate keys

meh['foo']['bar] = 1

-- works even if meh['foo'] didn't exist before

* allows pushing new elements to leaves which are arrays

meh['foo']['list] << elem1
meh['foo']['list] << elem2

* allows incrementing numeric leaves

meh['foo']['count'] += 7

* serializable

I have such a class in ruby. Can python do that?

Is this too magical?


class meh(dict):
def __getitem__(self, item):
if self.has_key(item):
return dict.__getitem__(self, item)
else:
anitem = meh()
dict.__setitem__(self, item, anitem)
return anitem

Actually what the OP wants is already a method of dict, it's called
setdefault(). It's not overloaded by "[]" because it's believed to be better to
be able to say "I want auto-generation" explicitally rather than implicitly: it
gives the user more power to control, and to enforce stricter rules.
.... def __getitem__(self, item):
.... return dict.setdefault(self, item, meh())
....

a = meh()
a["foo"]["bar"] = 2
a["foo"]["dup"] = 3
print a["foo"]["bar"] 2
print a

{'foo': {'dup': 3, 'bar': 2}}


So I advise using this class, and suggest the OP to try using setdefault()
explicitally to better understand Python's philosophy.

BTW: remember that setdefault() is written "setdefault()" but it's read
"getorset()".

 

[Diez B. Roggisch]

BTW: remember that setdefault() is written "setdefault()" but it's read
"getorset()".

I can only second that. The misleading name has - well, mislead me

Regards,

Diez

 

[Paul Rubin]

I can only second that. The misleading name has - well, mislead me

Hmm,

x[a][c][d] = e # x is a "magic" dict

becomes

x.setdefault(a,{}).setdefault(b,{}).setdefault(c,{})[d] = e

if I understand correctly. Ugh.

 

[Steven Bethard]

Hmm,

x[a][c][d] = e # x is a "magic" dict

becomes

x.setdefault(a,{}).setdefault(b,{}).setdefault(c,{})[d] = e

if I understand correctly. Ugh.

Agreed. I really hope that Python 3.0 applies Raymond Hettinger's
suggestion "Improved default value logic for Dictionaries" from
http://wiki.python.org/moin/Python3.0Suggestions

This would allow you to make the setdefault() call only once, instead of
on every lookup:

class meh(dict):
def __init__(self, *args, **kwargs):
super(meh, self).__init__(*args, **kwargs)
self.setdefault(function=meh)

STeVe

 

[braver]

Nice. What about pushing to leaves which are arrays, or incrementing
leaves which are numbers? If the array leaf didn't exist, or a number
wasn't set yet, << must create an empty array and push the element from
the RHS into it, and += must init the leaf to 0 and add the RHS to it.
Here's the corresponding ruby:

# ruby!

class MagicalExpandingHash < Hash
def initialize(*params)
if params.first.is_a? MagicalExpandingHash
@parentObj, @parentKey = params[0..1]
params = params[2..-1]
end
super(*params) { |h,k|
h[k] = MagicalExpandingHash.new(self,k)
}
end
def <<(elem)
if @parentObj[@parentKey].empty?
@parentObj[@parentKey] = [ elem ]
else
raise ArgumentError, "Can't push onto populated index", caller
end
end
def +(elem)
unless elem.is_a? Numeric
raise ArgumentError, "Can't add a non-Numeric value", caller
end
if @parentObj[@parentKey].empty?
@parentObj[@parentKey] = elem
else
raise ArgumentError, "Can't add to populated index", caller
end
end

def to_hash
h = Hash.new
self.each_pair {|k,v| h[k]=(v.class==self.class)? v.to_hash : v }
return h
end

def from_hash(h)
h.each_pair {|k,v| self[k]=(v.is_a? Hash) ?
self.class.new.from_hash(v) : v}
end

def marshal_dump
self.to_hash
end
def marshal_load(h)
from_hash(h)
end
end

# examples
if $0 == __FILE__
meh = MagicalExpandingHash.new

meh['usa']['france'] << 'tocqueville'
meh['usa']['france'] << 'freedom fries'
meh['life']['meaning'] += 42

puts meh.inspect
# => {"usa"=>{"france"=>["tocqueville", "freedom fries"]},
"life"=>{"meaning"=>42}}
end

 

[Paul Rubin]

Nice. What about pushing to leaves which are arrays, or incrementing
leaves which are numbers? If the array leaf didn't exist, or a number
wasn't set yet, << must create an empty array and push the element from
the RHS into it, and += must init the leaf to 0 and add the RHS to it.

Are you trying to simulate Ruby syntax or just implement those functions?
Implementing the functions is easy enough. If you want Ruby syntax,
use Ruby.

 

[braver]

Actually, the behavior is important to translate perl into ruby. Can
it be implemented in python looking similarly?

 

[Paul Rubin]

Actually, the behavior is important to translate perl into ruby. Can
it be implemented in python looking similarly?

It's kind of bizarre in Python to use << as a mutation operator, but I
guess you could do it. Sort of like 'cout << "hello world"' in C++.

 

[Steve Holden]

Hmm,

x[a][c][d] = e # x is a "magic" dict

becomes

x.setdefault(a,{}).setdefault(b,{}).setdefault(c,{})[d] = e

if I understand correctly. Ugh.

Agreed. I really hope that Python 3.0 applies Raymond Hettinger's
suggestion "Improved default value logic for Dictionaries" from
http://wiki.python.org/moin/Python3.0Suggestions

This would allow you to make the setdefault() call only once, instead of
on every lookup:

class meh(dict):
def __init__(self, *args, **kwargs):
super(meh, self).__init__(*args, **kwargs)
self.setdefault(function=meh)

STeVe

In fact, why not go one better and also add a "default" keyword
parameter to dict()?

regards
Steve

 

[braver]

Exactly, << as in C++/ruby streams. But notice the extra checks needed
to see whether we want a new leaf which is an array or a number, or we
create an intermediate hash level. Would the checks look the same in
python?

 

[Fredrik Lundh]

Exactly, << as in C++/ruby streams. But notice the extra checks needed
to see whether we want a new leaf which is an array or a number, or we
create an intermediate hash level. Would the checks look the same in
python?

we?

trust me, the number of people who think it's a good idea to write perl

</F>

 

[Fredrik Lundh]

Exactly, << as in C++/ruby streams. But notice the extra checks needed
to see whether we want a new leaf which is an array or a number, or we
create an intermediate hash level. Would the checks look the same in
python?

we?

trust me, the number of people who think it's a good idea to write perl-
inspired ruby and run that code in a python interpreter is very limited.

and even if you succeed in persuading someone else to write the code
for you, don't you think your users will find out pretty quickly that python's
not ruby ?

</F>

 

[Paul Rubin]

Exactly, << as in C++/ruby streams. But notice the extra checks needed
to see whether we want a new leaf which is an array or a number, or we
create an intermediate hash level. Would the checks look the same in
python?

You could check what is being shifted and make a new leaf of the
appropriate type. If you put a number there though, you wouldn't be
able to then add more nodes beneath that number.

 

[braver]

The point of this exercise is to compare how either ruby or python can
implement perl's default behavior when dealing with hashes. Since
these are bread and butter of scripting, having a MEH class handy can
enable fast semantically equivalent translation. This can be
beneficial for demonstrating feasibility of migrating to python.
Instead of debating philosophical justifications, I rather wonder
what's the most appropriate pythonic way to solve the problem as stated.

 

[Fredrik Lundh]

The point of this exercise is to compare how either ruby or python can
implement perl's default behavior when dealing with hashes. Since
these are bread and butter of scripting, having a MEH class handy can
enable fast semantically equivalent translation. This can be
beneficial for demonstrating feasibility of migrating to python.

if you want to write perl code, why migrate to some other language ?

Instead of debating philosophical justifications, I rather wonder
what's the most appropriate pythonic way to solve the problem
as stated.

write python code.

</F>

 

[braver]

Can assigning to hash without intermediate levels, possibly adding to a
numeric leaf or adding an element to a leaf array, be python code?

h['a']['b']['c'] += 42

If it can, I'd like to have a class which supports it.

Is keeping a list at the leaf of a hash python code?

h['a']['b']['c'].push(7) # or override push as an operator of your
choosing

Hashes with accumulating lists or counters at the leaves are universal
data structures used in python as much as anywhere else. Python is
used for scripting purposes at least as much as for some abstract ones.
Having a useful data structure is handy.

The multi-level hashes with default or accumulation come up naturally
in text parsing. Designing a dedicated class structure may or may not
be a better choice, depending on expediency.

 

[Paul Rubin]

Can assigning to hash without intermediate levels, possibly adding to a
numeric leaf or adding an element to a leaf array, be python code?

h['a']['b']['c'] += 42

If it can, I'd like to have a class which supports it.

Yes, it's simple enough to write a class like that. What is your
purpose in asking someone else to write it for you? If you're going
to write Python applications that use that class, you're going to have
to learn enough Python to easily write the class yourself.

The multi-level hashes with default or accumulation come up naturally
in text parsing. Designing a dedicated class structure may or may not
be a better choice, depending on expediency.

I understand that, I've written things like that in the past (not in
Python as it happens) and they were useful.