'long int too large to convert to int' in 'garbage collection' ignored

[Peter Kwan]

Hi,

I believe I have discovered a bug in Python 2.3. Could anyone suggest a get around?

When I tested my existing Python code with the newly released Python 2.3, I get the following warning:

FutureWarning: hex/oct constants > sys.maxint will return positive values in Python 2.4 and up

It is because I used some constants such as 0x80ff3366, so I change it to 0x80ff3366L, hoping to get rid of the warning.

But then I get the following fatal error:

Exception exceptions.OverflowError: 'long int too large to convert to int' in 'garbage collection' ignored
Fatal Python error: unexpected exception during garbage collection

abnormal program termination

I have tried testing the same code in Python 1.5.2 to Python 2.2. In all these versions, there is no error, no matter I use 0x80ff3366 or 0x80ff3366L. But in Python 2.3, I get warning or fatal error in these cases.

The details are as follows:

The code is running on Windows 2000. Python 2.3.

The Python code calls a C library and pass 0x80ff3366 or 0x80ff3366L to the C library. In the C library, it calls

//convert the given argument to Long in case it is not already Long
PyObject *longObj = PyNumber_Long(argObj);

//get the value of the Python Long variable
long ret = PyLong_AsLong(longObj);

I have done some testing and confirm that the error does not happen if the second line "PyLong_AsLong" is commented out. Also, the erorr does not happen immediately when executing the above statement. It occurs slightly afterwards (during garbage collection?).

Could someone suggest a word around without touching the C library, and without making the Python code too ugly?

Regards
Peter Kwan

 

[John Machin]

Hi,

I believe I have discovered a bug in Python 2.3.

This is consistent with most developer's belief systems, which include
code similar to:

blame_list = ["God", "Bill Gates", "Intel", "software supplier", "team
mate", "self"]

and need application of the .reverse() method.

Could anyone suggest a
get around?

When I tested my existing Python code with the newly released Python
2.3, I get the following warning:

FutureWarning: hex/oct constants > sys.maxint will return positive
values in Python 2.4 and up

It is because I used some constants such as 0x80ff3366, so I change it
to 0x80ff3366L, hoping to get rid of the warning.

But then I get the following fatal error:

Exception exceptions.OverflowError: 'long int too large to convert to
int' in 'garbage collection' ignored
Fatal Python error: unexpected exception during garbage collection

abnormal program termination

I have tried testing the same code in Python 1.5.2 to Python 2.2. In all
these versions, there is no error, no matter I use 0x80ff3366 or
0x80ff3366L. But in Python 2.3, I get warning or fatal error in these
cases.

The details are as follows:

The code is running on Windows 2000. Python 2.3.

The Python code calls a C library and pass 0x80ff3366 or 0x80ff3366L to
the C library. In the C library, it calls

//convert the given argument to Long in case it is not already Long
PyObject *longObj = PyNumber Long(argObj);

Danger; you are not testing for an error here. Some clueless caller
could supply an argument that can't be converted to Python Long. You
should do something like this:

PyObject *longObj;

longObj = PyNumber Long(argObj);
if (longObj == NULL) something_nasty_happened(....);

//get the value of the Python Long variable
long ret = PyLong AsLong(longObj);

I have done some testing and confirm that the error does not happen if
the second line "PyLong AsLong" is commented out. Also, the erorr does
not happen immediately when executing the above statement. It occurs
slightly afterwards (during garbage collection?).

You need to distinguish between the happening of the error and the
reporting of it. The error *happens* when you try to convert a long
int value to int type and it won't fit. You are not testing for the
error and clearing it, so it is *reported* later.

Any behavioural difference between versions is probably due to (a)
extra checking in the garbage collection phase in Python 2.3 (b) you
do a one-shot test and immediately exit Python or somehow trigger a
garbage collection before executing some other code -- if you do more
Python execution I would expect the delayed exception to be triggered
whatever the version of Python.

Here's what the doco says:
"""For C programmers, however, error checking always has to be
explicit. All functions in the Python/C API can raise exceptions,
unless an explicit claim is made otherwise in a function's
documentation. In general, when a function encounters an error, it
sets an exception, discards any object references that it owns, and
returns an error indicator -- usually NULL or -1. A few functions
return a Boolean true/false result, with false indicating an error.
Very few functions return no explicit error indicator or have an
ambiguous return value, and require explicit testing for errors with
PyErr_Occurred()"""

The last sentence applies to PyLong_AsLong().

It is of course very remotely possible that the behaviour of
PyLong_AsLong() changed in 2.3 but your posting is very remote from
evidence of that.

Could someone suggest a word around without touching the C library, and
without making the Python code too ugly?

Your C library needs (IMHO) touching so that, with any Python version,
it (a) permits either int or long int arguments (b) detects and
reports bad argument types (c) detects argument values that are not in
your permissible range (d) checks all API function calls for errors.

By the way, what is your permissible range for this argument? Is it a
32-bit vector, or is it whatever what (used to) fit in a Python int?
What should happen on a 64-bit machine?

As a design principle, you are the expert who is providing the library
to non-experts, so the Python code that has to be written to call a
function in your library should be no more ugly than having to have
"L" at the end of constants, if necessary. The work-around should be
done in your C code.

Have you considered using Pyrex? Even if you don't use it to develop
whole modules, looking at the C code that's generated can be very
useful if you need help on how to do reference counting or error
handling (including how to clean up before you return from your
function, both in normal circumstances and when an error has been
detected).

Hope this helps,
John

 

[Aahz]

I believe I have discovered a bug in Python 2.3.

This is consistent with most developer's belief systems, which include
code similar to:

blame_list = ["God", "Bill Gates", "Intel", "software supplier", "team
mate", "self"]

and need application of the .reverse() method.

Possibly. OTOH, I have in the past discovered a real bug in MS SQL
Server.