Decimal, __radd__, and custom numeric types...

[Blake T. Garretson]

I'm having some issues with decimal.Decimal objects playing nice with
custom data types. I have my own matrix and rational classes which
implement __add__ and __radd__. They know what to do with Decimal
objects and react appropriately.

The problem is that they only work with Decimals if the custom type is
on the left (and therefore __add__ gets called), but NOT if the Decimal
is on the left. The Decimal immediately throws the usual "TypeError:
You can interact Decimal only with int, long or Decimal data types."
without even trying my __radd__ method to see if my custom type can
handle Decimals.

From the Python docs (specifically sections 3.3.7 and 3.3.8), I thought

that the left object should try its own __add__, and if it doesn't know
what to do, THEN try the right object's __radd__ method. I guess
Decimal objects don't do this? Is there a way to change this behavior?
If Decimal objects prematurely throw a TypeError before trying the
__rop__, is Decimal broken, or was it designed this way? I think I'm
missing something...

Thanks,
Blake

 

[Nick Craig-Wood]

I'm having some issues with decimal.Decimal objects playing nice with
custom data types. I have my own matrix and rational classes which
implement __add__ and __radd__. They know what to do with Decimal
objects and react appropriately.

The problem is that they only work with Decimals if the custom type is
on the left (and therefore __add__ gets called), but NOT if the Decimal
is on the left. The Decimal immediately throws the usual "TypeError:
You can interact Decimal only with int, long or Decimal data types."
without even trying my __radd__ method to see if my custom type can
handle Decimals.

From the Python docs (specifically sections 3.3.7 and 3.3.8), I thought
that the left object should try its own __add__, and if it doesn't know
what to do, THEN try the right object's __radd__ method. I guess
Decimal objects don't do this? Is there a way to change this behavior?
If Decimal objects prematurely throw a TypeError before trying the
__rop__, is Decimal broken, or was it designed this way? I think I'm
missing something...

It looks like from reading 3.3.8 if decimal raised a NotImplemented
exception instead of a TypeError then it would work.

For objects x and y, first x.__op__(y) is tried. If this is not
implemented or returns NotImplemented, y.__rop__(x) is tried. If
this is also not implemented or returns NotImplemented, a
TypeError exception is raised. But see the following exception:

Exception to the previous item: if the left operand is an instance
of a built-in type or a new-style class, and the right operand is
an instance of a proper subclass of that type or class, the right
operand's __rop__() method is tried before the left operand's
__op__() method. This is done so that a subclass can completely
override binary operators. Otherwise, the left operand's __op__
method would always accept the right operand: when an instance of
a given class is expected, an instance of a subclass of that class
is always acceptable.

You could try this with a local copy of decimal.py since it is
written in Python.

 

[Kanenas]

On 28 Feb 2005 12:11:33 -0800, "Blake T. Garretson"

[...]

From the Python docs (specifically sections 3.3.7 and 3.3.8), I thought
that the left object should try its own __add__, and if it doesn't know
what to do, THEN try the right object's __radd__ method.

To me it reads more like the interpreter is responsible for picking
which method to call. Specifically, section 3.3.8 of the reference
manual states that y.__rop__() will be called if x.__op__() is not
implemented or returns NotImplemented. The decision to call
y.__rop__() is made outside the x.__op__() method, implying that
x.__op__() doesn't handle a call to y.__rop__() in this case. The
other rules dealing with whether x.__op__() or y.__rop__() is called
don't apply to your situation (but they could, keep reading).

I guess
Decimal objects don't do this? Is there a way to change this behavior?
If Decimal objects prematurely throw a TypeError before trying the
__rop__, is Decimal broken, or was it designed this way? I think I'm
missing something...

You could change the behavior of Decimal.__add__ by patching it or you
could use subclassing. If your classes are subclasses of Decimal,
their __rop__ methods will be called before Decimal.__op__ is tried.
I'm guessing your matrix and rational classes don't use decimal
representation, which makes this an OOP style-breaking kludge.

 

[Nick Coghlan]

If Decimal objects prematurely throw a TypeError before trying the
__rop__, is Decimal broken, or was it designed this way?

I suspect the former, since I can't recall this subject coming up at any point
during the PEP approval or implementation process. And I was one of the people
who worked on it before 2.4 was released

So I'd suggest:

a) Checking that replacing the relevant "raise TypeError" calls in
Lib/Decimal.py with "return NotImplemented" gives you friendlier behaviour.

b) Filing a bug report on SF

I'll be bringing the question up on py-dev as well.

Cheers,
Nick.

 

[Nick Coghlan]

a) Checking that replacing the relevant "raise TypeError" calls in
Lib/Decimal.py with "return NotImplemented" gives you friendlier behaviour.

It turns out this isn't really practical - there's too much code in the module
relying on those TypeErrors being raised.

So this may change for Python 2.5 (I think it's a genuine bug), but you're
probably stuck with it for 2.4 (since changing it is a big enough semantic
change to break code in the same module, so who knows what it would do to user
code).

Bugger

Cheers,
Nick.

 

[Nick Coghlan]

It turns out this isn't really practical - there's too much code in the
module relying on those TypeErrors being raised.

Then again, maybe it's not so bad:

Py> class C:
.... def __add__(self, other):
.... print "OK!"
.... __radd__ = __add__
....
Py> x = decimal.Decimal()
Py> C() + x
OK!
Py> x + C()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "C:\Python24\lib\decimal.py", line 906, in __add__
other = _convert_other(other)
File "C:\Python24\lib\decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long or Decimal da
ta types."
TypeError: You can interact Decimal only with int, long or Decimal data types.

Py> x = friendly_decimal.Decimal()
Py> C() + x
OK!
Py> x + C()
OK!

Cheers,
Nick.
http://boredomandlaziness.skystorm.net/misc/friendly_decimal.py

 

[Blake T. Garretson]

Thanks for the suggestions and modified module. I will probably just
use this "fixed" module to solve my immediate problem. I appreciate
your post to python-dev as well; it looks like this may be addressed in
a future release.

Thanks,
Blake

 

[Nick Coghlan]

Thanks for the suggestions and modified module. I will probably just
use this "fixed" module to solve my immediate problem.

Expect its performance to be worse than the standard version - it does the right
thing, but it sure ain't pretty. . .

Cheers,
Nick.