String changing on the fly

[Eli Daniel]

Hi,

I'm new to Python. Can you please tell me if the following is
possible.

My users are using other scripting launguage to write scripts. They
are used to write somthing like (keeping it simple)
T = 100
do_action ('It cost $T dollars')

where the above syntax ('It cost $T dollars') is interperted as ('It
cost 100 dollars').
I saw on way of doing it by using 'it cost '+ str(T) + ' dollars'.
But trying to keep it backward compatible, is there a way for the
function do_action, which resides in a class in a seperate file, to
take the original string (with the '$T') and replace it with the value
of T?

cheers

 

[John Hunter]

Eli> Hi, I'm new to Python. Can you please tell me if the
Eli> following is possible.

Eli> My users are using other scripting launguage to write
Eli> scripts. They are used to write somthing like (keeping it
Eli> simple) T = 100 do_action ('It cost $T dollars')

Eli> where the above syntax ('It cost $T dollars') is interperted
Eli> as ('It cost 100 dollars'). I saw on way of doing it by
Eli> using 'it cost '+ str(T) + ' dollars'. But trying to keep it
Eli> backward compatible, is there a way for the function
Eli> do_action, which resides in a class in a seperate file, to
Eli> take the original string (with the '$T') and replace it with
Eli> the value of T?

Python supports string interpolation with dictionaries, like
John paid 2 dollars

You can use the locals dictionary to work with locally defined variables
Bill paid 5 dollars

You can use regular expressions to convert $ syntax to python %(name)s
syntax


import re
def dollar_replace(matchobj):
return '%(' + matchobj.group(1) + ')s'

def like_perl(s,d):
rgx = re.compile('\$(\w+)')
fmt = re.sub('\$(\w+)', dollar_replace, s)
return fmt%d

T = 2
first = 'John'
print like_perl('$first paid cost $T dollars', locals())


You will probably want to tweak the regular expression so that any
valid python variable name is valid.

Cheers,
JDH

 

[John Hunter]

Continuing the fun, you can define a new python class that inherits
from string, but overrides the "convert me to a string representation"
method __str__ to evaluate itself in the context of the global
variables

import re


class PerlString(str):
def __str__(self):

rgx = re.compile('\$(\w+)')
fmt = re.sub('\$(\w+)', self.dollar_replace, self)
return fmt%globals()

def dollar_replace(self, matchobj):
return '%(' + matchobj.group(1) + ')s'

s = PerlString('$first paid cost $T dollars')


T = 2
first = 'John'
print s

T = 5
first = 'Bill'
print s

 

[Paul McGuire]

Continuing the fun, you can define a new python class that inherits
from string, but overrides the "convert me to a string representation"
method __str__ to evaluate itself in the context of the global
variables

import re


class PerlString(str):

<snip>

How do you know the language was Perl? Could be Tcl, which also uses the
'$' substitution form. (Although "T=100" would be written "set T 100" in
that case.)

-- Paul

 

[Hans Nowak]

class PerlString(str):
def __str__(self):

rgx = re.compile('\$(\w+)')
fmt = re.sub('\$(\w+)', self.dollar_replace, self)
return fmt%globals()

def dollar_replace(self, matchobj):
return '%(' + matchobj.group(1) + ')s'

s = PerlString('$first paid cost $T dollars')


T = 2
first = 'John'
print s

T = 5
first = 'Bill'
print s

Mooi hoor, but this only works for global variables. It won't work with
variables inside a function, unless you extend the function to pass in
locals.

Maybe something like this would be a reasonable alternative:
.... args = map(str, args)
.... return ''.join(args)
....'My name is Bill and I paid $2 dollars'

--Hans

 

[Alex Martelli]

Hi,

I'm new to Python. Can you please tell me if the following is
possible.

My users are using other scripting launguage to write scripts. They
are used to write somthing like (keeping it simple)
T = 100
do_action ('It cost $T dollars')

where the above syntax ('It cost $T dollars') is interperted as ('It
cost 100 dollars').
I saw on way of doing it by using 'it cost '+ str(T) + ' dollars'.
But trying to keep it backward compatible, is there a way for the
function do_action, which resides in a class in a seperate file, to
take the original string (with the '$T') and replace it with the value
of T?

As others have said, you need to pass do_action the dictionary of the
local variables of the function (or module's body) from which you're
calling it: do_action('it cost $T dollars', vars()). If you don't pass
this argument explicitly it IS possible for do_action to recover it by
black magic, but that would be very bad practice.

Apart from this, your best bet might be to upgrade to 2.4 (the current
beta appears quite good), which offers exactly this functionality in the
string.Template class. Otherwise, see how string.py does it (in the 2.4
standard library) and copy the solution. But reusing the solution
directly is much preferable, and for that you need 2.4.


Alex

 

[Eli Daniel]

As others have said, you need to pass do_action the dictionary of the
local variables of the function (or module's body) from which you're
calling it: do_action('it cost $T dollars', vars()). If you don't pass
this argument explicitly it IS possible for do_action to recover it by
black magic, but that would be very bad practice.

Apart from this, your best bet might be to upgrade to 2.4 (the current
beta appears quite good), which offers exactly this functionality in the
string.Template class. Otherwise, see how string.py does it (in the 2.4
standard library) and copy the solution. But reusing the solution
directly is much preferable, and for that you need 2.4.


Alex

Thank you all.
I've written a solution which sends that vars on every function call
as suggested (do_action('it cost $T dollars', vars()) which works
great.
I've spent some time trying to avoid sending the vars (before I saw
the previous post) with no much success (I've tried all kind of
imports which resulted in some weird issues for a new guy like me).
I'll try the beta, if it solves the problem that's great and if not
the above solution is good as well.

p.s.
The previous language was TCL, not perl, although I've written T=100
(but also wrote 'keeping it simple').

thanks,
cheers
Eli

 

[Alex Martelli]

...
Thank you all.

You're welcome.

I've written a solution which sends that vars on every function call
as suggested (do_action('it cost $T dollars', vars()) which works
great.
Good!

I've spent some time trying to avoid sending the vars (before I saw
the previous post) with no much success (I've tried all kind of
imports which resulted in some weird issues for a new guy like me).
I'll try the beta, if it solves the problem that's great and if not
the above solution is good as well.

It doesn't, per se, allow you to avoid passing vars() -- it just supples
in class string.Template the functionality for $-interpolation.

To avoid passing vars() you need black magic inside function do_action:
it must root around in the call-stack and find out the local variables
of its caller. That's not impossible, but it's not considered Python
style. "Explicit is better than implicit": when you pass vars() it's
obvious that the function is going to use the value of some of your
local variables, and you also gain some flexibility by being able to
pass another dictionary; when you pass nothing, it's pretty deep and
dark and mysterious what IS going to happen.

If you feel you must allow calling do_action without vars(), while not
recommended and not good Python style, you can. Python shares 4.5 out
of 5 principles that define the Spirit of C according to the intro of
C's official standard, after all, and the first two are:
1. trust the programmer
2. don't prevent the programmer from doing what needs to be done.

so, here's the 2.4 code one might use for that (NOT recommended...):

import string, inspect

def do_action(message, thevars=None)
if thevars is None:
thevars = inspect.currentframe(1).f_locals
print string.Template(message) % thevars


The black magic is the call to inspect.currentframe, which is very
clearly documented (at an interactive prompt, do import inspect and then
help(inspect.currentframe)) with:
'''
This function should be used for internal and specialized purposes only.
'''

Well, maybe not THAT clear, but hopefully scary enough. The concept is:
if you know what you're doing, we trust you, and we don't prevent you
from doing what you judge NEEDS to be done, even though it requires
black magic that reaches behind the scenes of the execution model,
plucking a frame out of the call-stack and accessing its implementation
detail 'f_locals' attribute -- "on your head be it";-).

Just be sure it NEEDS to be done -- be sure it's a PROBLEM to do things
cleanly, explicitly, simply, and in the obvious way -- Python strongly
urges you to rethink your framing of the issue, but doesn't force you.


Alex

 

[Eli Daniel]

You're welcome.


It doesn't, per se, allow you to avoid passing vars() -- it just supples
in class string.Template the functionality for $-interpolation.

To avoid passing vars() you need black magic inside function do_action:
it must root around in the call-stack and find out the local variables
of its caller. That's not impossible, but it's not considered Python
style. "Explicit is better than implicit": when you pass vars() it's
obvious that the function is going to use the value of some of your
local variables, and you also gain some flexibility by being able to
pass another dictionary; when you pass nothing, it's pretty deep and
dark and mysterious what IS going to happen.

If you feel you must allow calling do_action without vars(), while not
recommended and not good Python style, you can. Python shares 4.5 out
of 5 principles that define the Spirit of C according to the intro of
C's official standard, after all, and the first two are:
1. trust the programmer
2. don't prevent the programmer from doing what needs to be done.

so, here's the 2.4 code one might use for that (NOT recommended...):

import string, inspect

def do_action(message, thevars=None)
if thevars is None:
thevars = inspect.currentframe(1).f_locals
print string.Template(message) % thevars


The black magic is the call to inspect.currentframe, which is very
clearly documented (at an interactive prompt, do import inspect and then
help(inspect.currentframe)) with:
'''
This function should be used for internal and specialized purposes only.
'''

Well, maybe not THAT clear, but hopefully scary enough. The concept is:
if you know what you're doing, we trust you, and we don't prevent you
from doing what you judge NEEDS to be done, even though it requires
black magic that reaches behind the scenes of the execution model,
plucking a frame out of the call-stack and accessing its implementation
detail 'f_locals' attribute -- "on your head be it";-).

Just be sure it NEEDS to be done -- be sure it's a PROBLEM to do things
cleanly, explicitly, simply, and in the obvious way -- Python strongly
urges you to rethink your framing of the issue, but doesn't force you.


Alex

Hi,
My first script was is and running - with the version which accepts
the vars().
That's ok, although if it could have easily be removed - I would have.
I think that for now with my little knowledge of Python I should not
try the black magic, as you said- I should know what I'm doing, and
I'm not really.

Anyway when migerating from TCL, it's easy to teach somone to add at
the end of the function call an additional parameter, "vars()" that
telling him to sabtitue every parameter with '+ str(thevar)'. The
reason is not that the later is harder to understand or write, it is
simply less readable especialy when you write a large text with some
variables as a parameter.

cheers,
Eli

 

[Alex Martelli]

Anyway when migerating from TCL, it's easy to teach somone to add at
the end of the function call an additional parameter, "vars()" that
telling him to sabtitue every parameter with '+ str(thevar)'. The
reason is not that the later is harder to understand or write, it is
simply less readable especialy when you write a large text with some
variables as a parameter.

Sure, I agree. string.Template in 2.4 chose $ for interpolation in part
exactly because it's popular in several other languages (perl, tcl,
bash). Besides, +'ing a bunch of str(...) wouldn't be idiomatical
Python anyway; before string.Template, we tended to use more often the
C-derived concept of formatting --

'hello Mr %s' % surname

rather than

'hello Mr %(surname)s' % vars()


....

Alex

 

[Dave Benjamin]

Well, maybe not THAT clear, but hopefully scary enough. The concept is:
if you know what you're doing, we trust you, and we don't prevent you
from doing what you judge NEEDS to be done, even though it requires
black magic that reaches behind the scenes of the execution model,
plucking a frame out of the call-stack and accessing its implementation
detail 'f_locals' attribute -- "on your head be it";-).

This type of stack manipulation is much more common in Tcl, where it is used
to accomplish tasks as simple as passing an array to a function! The "black
magic" procedures in Tcl are "upvar" and "uplevel". Their use is discouraged
in most cases, as one might expect in any language, but they are useful in
implementing control structures.

One particularly nice implementation of string interpolation, done by
Ka-Ping Yee, is available here:

http://lfw.org/python/
(search for "string interpolation for Python")

It "hacks the stack" to find the calling environment's variables, and allows
many neat tricks like the following:

from Itpl import printpl
printpl("Here is a $string.")
printpl("Here is a $module.member.")
printpl("Here is an $object.member.")
printpl("Here is a $functioncall(with, arguments).")
printpl("Here is an ${arbitrary + expression}.")
printpl("Here is an $array[3] member.")
printpl("Here is a $dictionary['member'].")

I have used this successfully in a homebrew templating system for several
websites, and found that it offers just the right compromise between
convenience and flexibility. YMMV.

Another, similar use of this "black magic" that I've done is to write a
function that works like PHP's "compact":

http://php.net/compact

If you've got a lot of dictionary construction and deconstruction to do,
this simple trick can save a lot of {'redundant': redundant,
'mapping': mapping} between locals and dictionaries.

(but of course it still feels dirty)

 

[Alex Martelli]

If you've got a lot of dictionary construction and deconstruction to do,
this simple trick can save a lot of {'redundant': redundant,
'mapping': mapping} between locals and dictionaries.

Nah -- just pass in locals() as an argument!

(but of course it still feels dirty)

So don't do it!-)


Alex

 

[Dave Benjamin]

Nah -- just pass in locals() as an argument!

Eight characters, Alex! Ten, if you consider the comma and space. =)

So don't do it!-)

These were my younger days, of course...