int/long unification hides bugs

[Cliff Wells]

The problem with int/long unification is that there is no simple
pure-Python alternative for those of us who need a bounded integer
type. If our code depended on that raised OverflowError in order to
ensure that our computations were bounded, we're left high and dry
with ints and longs unified. We must either drop down to C and write
a bounded integer type, or we're stuck with code that no longer works.

Color me ignorant (I still maintain that numbers were a bad idea to
start with), but can you give an example of something that would
*require* this? It seems to me that whether an OverflowError occurs is
only one way of determining whether a computation is bounded, and
further, it's a rather arbitrary marker (it is disassociated from the
capabilities of the computer hardware it's run on) . I'd think
computational resources are the real concern (time and memory). These
can be handled other ways (e.g. running the computation in a separate
process and killing it if it exceeds certain time/memory constraints).

I suppose my position is this: if I have a computer that can perform a
calculation in a reasonable time without exhausting the resources of the
machine then I would think that Python shouldn't try to stop me from
doing it, at least not based on some arbitrary number of bits.

Regards,
Cliff

 

[Jeremy Fincher]

Color me ignorant (I still maintain that numbers were a bad idea to
start with), but can you give an example of something that would
*require* this?

Accepting integer equations from untrusted users and evaluating them.
That's my exact use case, in fact.

It seems to me that whether an OverflowError occurs is
only one way of determining whether a computation is bounded,

It's not really a way of *determining* whether a computation is
bounded, but *guaranteeing* that it is bounded. What I eventually did
was simply convert all numbers to floats, and do my calculations with
those; but this results in some ugliness, for instance, when a user
asks for the value of "10**24".

These
can be handled other ways (e.g. running the computation in a separate
process and killing it if it exceeds certain time/memory constraints).

They're also significantly more complex and significantly less
portable.

I suppose my position is this: if I have a computer that can perform a
calculation in a reasonable time without exhausting the resources of the
machine then I would think that Python shouldn't try to stop me from
doing it, at least not based on some arbitrary number of bits.

That's exactly it. With bounded integers, I know for sure that my
program can perform a calculation in a reasonable time without
exhausting the resources of the machine. Without them, I have to go
to great lengths to receive that assurance.

Jeremy

 

[Josiah Carlson]

Accepting integer equations from untrusted users and evaluating them.
That's my exact use case, in fact.

So seemingly you have a problem with:
val = RangedInteger(int(user_provided), mylow, myhigh)

Ok, so you've got a problem with that. *shrug*

It's not really a way of *determining* whether a computation is
bounded, but *guaranteeing* that it is bounded. What I eventually did
was simply convert all numbers to floats, and do my calculations with
those; but this results in some ugliness, for instance, when a user
asks for the value of "10**24".

Or any one of the other infinite values that binary floating point does
not represent exactly. What you have done is to take your problem of
limiting range, and attempt to fold it into a type with a larger dynamic
range, but with limited precision. Replacing an "OverflowError", with a
possible future infinite value returned by float and known precision
issues with non-integer math.

What gets me is that you seem to be saying that BEFORE 2.4 has
officially come out, you have switched to using floats because 2.4 will
remove the OverflowError when integers overflow.

That's exactly it. With bounded integers, I know for sure that my
program can perform a calculation in a reasonable time without
exhausting the resources of the machine. Without them, I have to go
to great lengths to receive that assurance.

Ah, but you, or anyone else, only needs to implement it once for some
class of things, and it is done. People who want more or different
functionality are free to derive. With a little bit of futzing, you
could take the base code given by Andrew Dalke, add in custom handlers
for binary exponentiation with __pow__, and you're basically done.

Is there anything stopping you from doing this other than laziness and a
desire to complain "Python 2.4 doesn't have functionality X that it
choose_one_or_more(used_to_have, that_i_want)"?


- Josiah

 

[Andrew Dalke]

How long with this take to run?

It wouldn't -- I didn't implement __pow__.

I think our inability to write a RangedNumber that piggybacks on
Python's integers should be obvious.

It sounds like you don't like that the simple
__pow__ implementation will raise a MemoryError after
a long time when you would rather have it raise a
bounds violation error early.

If you want that, you could have an implementation that
uses a few logs to check for that possibility.

Though I'm not sure how to do the bounds checking for
the 3 arg form. Given a as above, I know that

pow(a, a, 2**50)

but without doing the computation my bounds checker
can't (or at least shouldn't) be that clever. Should
it require that the 3rd arg always be in the allowed
range?

What about

a = RangedNumber(100, 100, 400)
pow(a, 25424134, 321)

? It turns out the result is 321 which is in the right
range, but I can't know that without basically doing
the calculation.


Andrew
(e-mail address removed)