decimal by default

[Daniel]

I'm writing an application that (among other things) evaluates
mathematical expressions. The user enters strings containing literals
and names that later get evaluated using the Python interpreter. Here's
a short (very simplified) example:
Traceback (most recent call last):
...
TypeError: You can interact Decimal only with int, long or Decimal data
types.

I understand why I got the error, so there's no need to explain that.
It is a requirement that the 'names' dict contains Decimal values. And
of course it's unacceptable to expect my users to enter Decimal('...')
every time they enter a non-integer number. My initial solutioin is to
use a regular expression to wrap each float value with Decimal('...')
before the expression is evaluated. But I don't like that solution for
two reasons:

1. It seems error prone and inelegant. Paraphrase: if you've got a
problem and you think "Ahh, I'll use regular expressions..." now you've
got two problems.

2. Error reporting is not as intuitive (I'm using the Python
interpreter and therefore my users see Python exceptions when their
expressions don't evaluate). After the expressions have been shot up
with all the extra Decimal junk to make them evaluate correctly they
are not nearly as recognizable (or easy to read) and the user is likely
to think "but I didn't even write that expression...where is that
Decimal('...') stuff coming from?"

Ideally I'd like to have a way to tell the interpreter to use Decimal
by default instead of float (but only in the eval() calls). I
understand the performance implications and they are of no concern. I'm
also willing to define a single global Decimal context for the
expressions (not sure if that matters or not). Is there a way to do
what I want without rolling my own parser and/or interpreter? Is there
some other alternative that would solve my problem?

Thanks,
~ Daniel

 

[Alex Martelli]

Ideally I'd like to have a way to tell the interpreter to use Decimal
by default instead of float (but only in the eval() calls). I
understand the performance implications and they are of no concern. I'm
also willing to define a single global Decimal context for the
expressions (not sure if that matters or not). Is there a way to do
what I want without rolling my own parser and/or interpreter? Is there
some other alternative that would solve my problem?

What about:

c = compile(thestring, thestring, '<eval>')

cc = new.code( ...all args from c's attributes, except the 5th
one, constants, which should instead be:
decimalize(c.co_consts)...)

i.e.

cc = new.code(c.co_argcount, c.co_nlocals, c.co_stacksize, c.co_flags,
c.co_code, decimalize(c.co_consts), c.co_names,
c.co_varnames, c.co_filename, c.co_name,
c.co_firstlineno, c.co_lnotab)

where

def decimalize(tuple_of_consts):
return tuple( maydec(c) for c in tuple_of_consts )

and

def maydec(c):
if isinstance(c, float): c = decimal.Decimal(str(c))
return c


Yeah, the new.code call IS very verbose, just because it needs to
tediously and boilerplatedly repeat every attribute as an argument, but
you can easily wrap the boilerplate in an auxiliary function and forget
about it. (If you often want to change one or two tiny thing in a code
object you probably already have a suitable auxiliary function around).

Now, you can eval(cc) [[I suggest you also explicitly pass dictionaries
for locals and globals, but, hey, that's just a good idea with eval in
general!-)]] and, ta-da!


Alex

 

[Daniel]

What about:

c = compile(thestring, thestring, '<eval>')

cc = new.code( ...all args from c's attributes, except the 5th
one, constants, which should instead be:
decimalize(c.co_consts)...)

Wow, what an elegant solution! I had no hope that it would be this
simple. I always wondered what compile() was useful for and now I know
at least one thing. I'll try it out tomorrow. Thanks a lot Alex!

~ Daniel

 

[Alex Martelli]

Wow, what an elegant solution! I had no hope that it would be this

Heh, funny, I was originally parsing your response as ironic, because
_I_ don't think of this as elegant -- too boilerplatey (as the expansion
I showed right after dispays!)... took me a sec to see you really mean
it!-)

simple. I always wondered what compile() was useful for and now I know
at least one thing. I'll try it out tomorrow. Thanks a lot Alex!

You're welcome! And, compile is also useful for many other things, such
as any situation where you may need to run eval multiple times on the
same string of source code (typically on multiple distinct
dicts/namespaces): compile the source once, then in the loop eval the
code object (bytecode) rather than the source -- that saves time.

Also, you may do introspection on the code object -- for example, I show
in the Nutshell's 2nd ed how this lets you perform a "safe eval" -- a
way to let the user specify any Python "literal" without risking a
malicious user running arbitrary code (essentially, you refuse to eval
the code object if its co_names isn't empty -- or, you might let said
co_names possibly contain just a few names you deem "safe", such as,
say, 'sin', 'cos', 'tan', which you can get into your namespace from the
math module). Such introspection on names may also allow some further
optimization, particularly in the repeated-execution case, if there are
"well-known names" that you're able to compute "just in time" (nowadays
you can also use a special mapping to "only compute at need" the values
for the names that are actually needed).

Beyond which, we get into the realm of byecode hacks...!-)


Alex