yet another recipe on string interpolation

[Michele Simionato]

I was playing with string.Template in Python 2.4 and I came out with the
following recipe:

import sys
from string import Template

def merge(*dictionaries):
"""Merge from right (i.e. the rightmost dictionary has the precedence)."""
merg = {}
for d in dictionaries:
merg.update(d)
return merg

def interp(s, dic = None):
if dic is None: dic = {}
caller = sys._getframe(1)
return Template(s) % merge(caller.f_globals, caller.f_locals, dic)

language="Python"
print interp("My favorite language is $language.")

Do you have any comments? Suggestions for improvements?

Michele Simionato

 

[Peter Hansen]

I was playing with string.Template in Python 2.4 and I came out with the
following recipe:

def merge(*dictionaries):
"""Merge from right (i.e. the rightmost dictionary has the precedence)."""

Do you have any comments? Suggestions for improvements?

My only comment is about *your* comment, above. I would
have called the behaviour described "merge from left".
Maybe best just to cut out that part of the comment, and
leave it as "merge where rightmost dictionary has precedence".

Cute recipe though.

-Peter

 

[Peter Otten]

I was playing with string.Template in Python 2.4 and I came out with the
following recipe:

import sys
from string import Template

def merge(*dictionaries):
"""Merge from right (i.e. the rightmost dictionary has the
precedence).""" merg = {}
for d in dictionaries:
merg.update(d)
return merg

def interp(s, dic = None):
if dic is None: dic = {}
caller = sys._getframe(1)
return Template(s) % merge(caller.f_globals, caller.f_locals, dic)

language="Python"
print interp("My favorite language is $language.")

Do you have any comments? Suggestions for improvements?

Nice. recipe. I think the perlish format looks better than our current
"%(name)s" thingy. While interp() does some magic it should still be
acceptable for its similarity with eval(). Here are some ideas:
You could virtualize the merger of dictionaries:

class merge(object):
def __init__(self, *dicts):
self.dicts = dicts
def __getitem__(self, key):
for d in reversed(self.dicts):
try:
return d[key]
except KeyError:
pass
raise KeyError(key)

I would use either implicitly or explicitly specified dictionaries, not
both:

def interp(s, *dicts, **kw):
if not dicts:
caller = sys._getframe(1)
dicts = (caller.f_globals, caller.f_locals)
return Template(s).substitute(merge(*dicts), **kw)

As of 2.4b2, the modulo operator is not (not yet, or no longer?) defined, so
I had to use substitute() instead.

I really like your option to provide multiple dictionaries - maybe you
should suggest changing the signature of the substitute() method to the
author of the Template class.

Peter

 

[Michele Simionato]

My only comment is about *your* comment, above. I would
have called the behaviour described "merge from left".
Maybe best just to cut out that part of the comment, and
leave it as "merge where rightmost dictionary has precedence".
Ok.

Cute recipe though.

Thanks ;-)

I was asking for comments about the performance of this approach
and/or design choices: for instance, in the case an explicit
dictionary is passed, it could make sense to use it for interpolation,
without merging it with the locals and globals. I could add a default
argument to set this behavior, but I am not sure if this is worthwhile ...

Michele Simionato

 

[Dave Benjamin]

language="Python"
print interp("My favorite language is $language.")

Do you have any comments? Suggestions for improvements?

Well, for what it attempts to do, I'd say your solution is ideal.
However, I think that Ka-Ping Yee's implementation was more full-featured:

http://lfw.org/python/Itpl20.py

Of course, this goes above and beyond the functionality of Template, so
your code example is orthogonal to the reasons I'd prefer his solution.

He did the same sort of stack trick that you're doing, also. This code
was the first that I'd ever seen do it, and this was before
sys._getframe() was available.

Dave

 

[Raymond Hettinger]

"Michele Simionato":

I was playing with string.Template in Python 2.4 and I came out with the
following recipe:

import sys
from string import Template

def merge(*dictionaries):
"""Merge from right (i.e. the rightmost dictionary has the precedence)."""
merg = {}
for d in dictionaries:
merg.update(d)
return merg

def interp(s, dic = None):
if dic is None: dic = {}
caller = sys._getframe(1)
return Template(s) % merge(caller.f_globals, caller.f_locals, dic)

The ASPN chainmap() recipe is an alternative to merge(). It spares the initial
effort of combining all the dictionaries. Instead, it does the lookups only
when a specific key is needed. In your example, it is likely that most of
globals will never be looked-up by the template, so the just-in-time approach
will save time and space. Also, chainmap() is a bit more general and will work
with any object defining __getitem__.

The interp() part of the recipe is nice.


Raymond Hettinger


http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305268 :
--------- chainmap() ---------


import UserDict

class Chainmap(UserDict.DictMixin):
"""Combine multiple mappings for sequential lookup.

For example, to emulate Python's normal lookup sequence:

import __builtin__
pylookup = Chainmap(locals(), globals(), vars(__builtin__))
"""

def __init__(self, *maps):
self._maps = maps

def __getitem__(self, key):
for mapping in self._maps:
try:
return mapping[key]
except KeyError:
pass
raise KeyError(key)


Raymond Hettinger

 

[Michele Simionato]

As of 2.4b2, the modulo operator is not (not yet, or no longer?) defined, so
I had to use substitute() instead.

I have downloaded Python 2.4b2 today and looked at the source code for
the Template class. There is no __mod__ method, in contraddiction with
PEP 292. Is there somebody who knows what happened and the rationale
for the change? Should PEP 292 be updated?

Michele Simionato

 

[Raymond Hettinger]

Peter Otten <[email protected]> wrote in message

I have downloaded Python 2.4b2 today and looked at the source code for
the Template class. There is no __mod__ method, in contraddiction with
PEP 292. Is there somebody who knows what happened and the rationale
for the change? Should PEP 292 be updated?

It was updated a good while ago:
http://www.python.org/peps/pep-0292.html

The PEP STATES that the API is through the substitute() and safe_substitute()
methods. Also, it notes that __mod__ is easily aliased to one of the those via
a Template subclass.

By using named methods instead of the % operator:
* we avoid the precedence issues associated with %
* we avoid confusion as to whether a tuple is an acceptable second argument
* we avoid confusion with "%s" style inputs.
* we enable keyword arguments: mytempl.substitute(weather="rainy")
* we get something more documenting than punctuation.


Raymond Hettinger

 

[Raymond Hettinger]

By using named methods instead of the % operator:

* we avoid the precedence issues associated with %
* we avoid confusion as to whether a tuple is an acceptable second argument
* we avoid confusion with "%s" style inputs.
* we enable keyword arguments: mytempl.substitute(weather="rainy")
* we get something more documenting than punctuation.

s/documenting/self-documenting


One other thought. In both the original and current design, it is possible and
perhaps likely that the template instantiation and method application will be
remote from one another. Unfortunately, the original design forced the
safe/non-safe decision to be made at the time of instantiation.

When you think about it, that was a mis-assignment of responsibilities. The
choice of safe vs. non-safe has nothing to do with the contents of the template
and everything to do with how the template is used. The code that uses the
template (applies the method) needs to know whether it should handle exceptions
or not. Hence, it should be the one to make the safe / non-safe choice.

Consider whether the following fragment is correct:

mytemp % values

Should it have been wrapped in a try/except? Can values be a tuple? Do we have
a reasonable expectation of whether mytemp contains $placeholder ro
%(placeholder)s? The approach taken by the revised pep makes all of the issues
clear.

There were a couple of downsides to switching the API. Formerly, if you knew
that client code used the % operator with a dictionary, then you could pass in
an instance of the Template class and have it do the right thing (polymorphism
on a good day). Also, most folks already know what % means, so there was less
of a learning curve. IOW, the switch was an improvement, but not a pure win.
C'est le vie.


Raymond Hettinger