Pre-PEP: Dictionary accumulator methods

[Steven Bethard]

I am surprised nobody suggested we put those two methods into a
separate module (say dictutils or even UserDict) as functions:

from dictutils import tally, listappend

tally(mydict, key)
listappend(mydict, key, value)

Sorry to join the discussion so late (I've been away from my email for a
week) but this was exactly my reaction too. In fact, I have a
'dicttools' module with similar methods in it:

# like "tally" but without ability to set increment
def counts(iterable, key=None):
result = {}
for item in iterable:
# apply key function if necessary
if key is None:
k = item
else:
k = key(item)
# increment key's count
try:
result[k] += 1
except KeyError:
result[k] = 1
return result

# like "listappend" but with the option to use key and value funcs
def groupby(iterable, key=None, value=None):
result = {}
for item in iterable:
# apply key function if necessary
if key is None:
k = item
else:
k = key(item)
# apply value function if necessary
if value is None:
v = item
else:
v = value(item)
# append value to key's list
try:
result[k].append(v)
except KeyError:
result[k] = [v]
return result

These two functions have covered all my use cases for "tally" and
"listappend" -- I always want to perform the increments or list appends
over a sequence of values, so having functions that operate on sequences
covers all my needs.

STeVe

 

[Steven Bethard]

FWIW, here is my take on the defaultdict approach:

def defaultdict(defaultfactory, dictclass=dict):
class defdict(dictclass):
def __getitem__(self, key):
try:
return super(defdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, defaultfactory())
return defdict

d = defaultdict(int)()
d["x"] += 1
d["x"] += 1
d["y"] += 1
print d

d = defaultdict(list)()
d["x"].append(1)
d["x"].append(2)
d["y"].append(1)
print d

Michele Simionato

Very pretty! =)

It does mean, however, that if the defaultfactory function takes any
arguments, you have to wrap the function to make this work. I'd
probably prefer something like:

py> def defaultdict(*args, **kwargs):
.... defaultfactory, args = args[0], args[1:]
.... class defdict(dict):
.... def __getitem__(self, key):
.... try:
.... return super(defdict, self).__getitem__(key)
.... except KeyError:
.... return self.setdefault(key, defaultfactory(
.... *args, **kwargs))
.... return defdict
....
py> d = defaultdict(int)()
py> d['x'] += 1
py> d['x'] += 1
py> d['y'] += 1
py> d
{'y': 1, 'x': 2}
py> d = defaultdict(list, [0])()
py> d['x'].append(1)
py> d['x'].append(2)
py> d['y'].append(1)
py> d
{'y': [0, 1], 'x': [0, 1, 2]}

That said, I still think a dictools module is a better solution to this
problem.

STeVe

 

[Jack Diederich]

Sorry to join the discussion so late (I've been away from my email for a
week) but this was exactly my reaction too. In fact, I have a
'dicttools' module with similar methods in it:

<snipped>

I like this approach, it will give us a chance to test & tweak the signature
before hanging it off dict proper. It feels similar to the strings module
to str transition, sets module to set builtin, and itertools module to iter
transition.

itertools to iter transition, huh? I slipped that one in, I mentioned it
to Raymond at PyCon and he didn't flinch. It would be nice not to have to
sprinkle 'import itertools as it' in code. iter could also become a type
wrapper instead of a function, so an iter instance could be a wrapper that
figures out whether to call .next or __getitem__ depending on it's argument.
for item in iter(mylist).imap:
print item
or
for item in iter.imap(mylist):
print item

I haven't digested that too much, just a thought.

-jackdied

 

[Michael Spencer]

<snipped>

I like this approach, it will give us a chance to test & tweak the signature
before hanging it off dict proper. It feels similar to the strings module
to str transition, sets module to set builtin, and itertools module to iter
transition.

itertools to iter transition, huh? I slipped that one in, I mentioned it
to Raymond at PyCon and he didn't flinch. It would be nice not to have to
sprinkle 'import itertools as it' in code.

Not that that is such a pain, but accessing itertools functions from an
"outlying" module seems somewhat incompatible with putting iterative approaches
center stage.

iter could also become a type

wrapper instead of a function, so an iter instance could be a wrapper that
figures out whether to call .next or __getitem__ depending on it's argument.
for item in iter(mylist).imap:
print item

Also opening the door for iter to be subclassed. For example, could listiter
become a specialization of iter - one that uses __getitem__ and which could
allow reverse iteration?

or
for item in iter.imap(mylist):
print item

I haven't digested that too much, just a thought.

-jackdied

A very good thought IMO.

Michael

 

[Greg Ewing]

py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]

which can be written more succinctly as

def defaultdict(defaultfactory, *args, **kwargs):
...

 

[Steven Bethard]

py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]

which can be written more succinctly as

def defaultdict(defaultfactory, *args, **kwargs):
...

Not if you want to allow the defaultfactory to be called with a keyword
argument 'defaultfactory'. Compare my code:

py> def defaultdict(*args, **kwargs):
.... defaultfactory, args = args[0], args[1:]
.... print defaultfactory, args, kwargs
....
py> defaultdict(dict, defaultfactory=True)
<type 'dict'> () {'defaultfactory': True}

with the code you suggested:

py> def defaultdict(defaultfactory, *args, **kwargs):
.... print defaultfactory, args, kwargs
....
py> defaultdict(dict, defaultfactory=True)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: defaultdict() got multiple values for keyword argument
'defaultfactory'

Uncommon, sure, but I'd rather not rule it out if there's no need to.

STeVe