2.3 -> 2.4: long int too large to convert to int

[Grant Edwards]

I give up, how do I make this not fail under 2.4?

fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))

I get an OverflowError: long int too large to convert to int

ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
the high-order bit set. I'm assuming Python thinks it's a
signed value. How do I tell Python that 0xc0047a80 is an
unsigned 32-bit value?

 

[Terry Reedy]

I give up, how do I make this not fail under 2.4?


fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))

I get an OverflowError: long int too large to convert to int

ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
the high-order bit set. I'm assuming Python thinks it's a
signed value. How do I tell Python that 0xc0047a80 is an
unsigned 32-bit value?

In 2.3 and before, you get this:-1073448320

In 2.4, positive hex literals are treated as positive numbers, and that is
your problem: your literal is greater than the largest int and hence gets
stored as long int. I would try -1073448320 as the arg.

Terry J. Reedy

 

[chrisperkins99]

I give up, how do I make this not fail under 2.4?

fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))

I get an OverflowError: long int too large to convert to int

ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
the high-order bit set. I'm assuming Python thinks it's a
signed value. How do I tell Python that 0xc0047a80 is an
unsigned 32-bit value?

You could sort-of fake it like this,

def unsigned(val):
return struct.unpack('i', struct.pack('I', val))[0]

fcntl.ioctl(self.dev.fileno(), unsigned(0xc0047a80), ...)

but good luck writing a docstring explaining why a function called
"unsigned" takes a positive long and returns a negative int...


Chris Perkins

 

[Grant Edwards]

In 2.3 and before, you get this:
-1073448320

I don't particular care how Python prints the value -- I just
want that value passed to the function I'm calling.

In 2.4, positive hex literals are treated as positive numbers, and that is
your problem: your literal is greater than the largest int and hence gets
stored as long int.

I knew that, I just couldn't come up with a good way to fix it.

I would try -1073448320 as the arg.

That should work, but it's kind of lame (no offense).

ioctl values are always, always written in hex. A block of
ioctl values is generally assigned to a particular driver such
that the high order N (is it 4 oe 5?) hex digits are unique to
that driver. Writing the value in decimal is going to
completely confuse anybody looking at the code.

I rather like the other suggestion of writing a function that
accepts 0x<whatever> and returns the appropriate integer value.

Another poster suggested a solution using struct. Here's my
solution (which assume python integers are represented in 2's
compliment binary):

def ioctlValue(i):
if i & 0x80000000:
i = -((i^0xffffffff)+1)
return i

 

[Grant Edwards]

You could sort-of fake it like this,

def unsigned(val):
return struct.unpack('i', struct.pack('I', val))[0]

fcntl.ioctl(self.dev.fileno(), unsigned(0xc0047a80), ...)

I rather like this

if i & 0x8000000:
i = -((i^0xffffffff)+1)

As long as I'm obfscating the code, who can resist some bitwise
operations. Of course it'll break on machines that don't use
2's compliment, but that's just iceing on the cake.

 

[Bengt Richter]

I don't particular care how Python prints the value -- I just
want that value passed to the function I'm calling.

I do care, dang it. IMIFO (in my increasingly frustrated opinion ;-)
one ought to be able to write literals for negative integers.
A simple variation on 0x... coul be 0h... where what follows 0h is
base-16-complement, which turns out to be 0hfc0047a80 for the negative
number you want, and would be 0h0c0047a80 if you wanted the positive number
with the same least significant bits.

I knew that, I just couldn't come up with a good way to fix it.

IMO you shouldn't have to fight it.

That should work, but it's kind of lame (no offense).

Yes, it is lame ;-)

see more on the notation (of which hex is only the particular base-16 case)

http://groups.google.co.uk/group/comp.lang.python/msg/2c411ca9251774dc

(It doesn't show in the examples, but unfortunately the code has a bug that I fixed
in a later post,

http://groups.google.co.uk/group/comp.lang.python/msg/359927a23eb15b3e


the encoding of -1073448320 would be
'0hfc0047a80'

'0b10.98926551680'
-1073448320 note the correspondence to previous line for base 10 ;-)
-3

repeating the "sign digit" doesn't change the decoded value:
5

irrespective of the base:
-1073448320L

ioctl values are always, always written in hex. A block of
ioctl values is generally assigned to a particular driver such
that the high order N (is it 4 oe 5?) hex digits are unique to
that driver. Writing the value in decimal is going to
completely confuse anybody looking at the code.

I rather like the other suggestion of writing a function that
accepts 0x<whatever> and returns the appropriate integer value.

Sure, but there's no reason we shouldn't be allowed to specify a constant
as a literal IMO.

Another poster suggested a solution using struct. Here's my
solution (which assume python integers are represented in 2's
compliment binary):

def ioctlValue(i):
if i & 0x80000000:
i = -((i^0xffffffff)+1)
return i

Do you think it's PEP-able, or should I quit being obnoxious ;-)

I think str.mod format like %x except %<width>.<base>b would make it
easy to write '0h%08b.16' % a_signed_integer and get something both
readable and inputtable as a constant. (0h.<the rest> would be short
for 0b16.<the rest>) BTW, %b (or %B for uppercase) could default to base 16.
The ouput would only be as wide as necessary, with the leading digit
guaranteed 0 or f (which is 0 or <base-1> in the general case).
</rant>

Regards,
Bengt Richter

 

[Raymond L. Buvel]

I give up, how do I make this not fail under 2.4?

fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))

I get an OverflowError: long int too large to convert to int

ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
the high-order bit set. I'm assuming Python thinks it's a
signed value. How do I tell Python that 0xc0047a80 is an
unsigned 32-bit value?

Everyone seems to be suggesting that the fix for the problem is to
somehow cobble together some way of forcing an unsigned integer into a
signed integer (what you would do with a cast in C). However, if I
understand the long<->int consolidation this is not consistent with that
effort.

As far as I can tell, the underlying problem is that the C routine
fcntl.ioctl is expecting a signed integer. These are the kinds of
problems that need to be fixed. The function should be asking for an
unsigned integer. This is possible with the C API at least since Python
2.3. Without these fixes, the long<->int consolidation is going to
continue to produce frustration. There are many functions in the
standard library that you would expect to take unsigned integers but
actually specify signed integers.

 

[Grant Edwards]

Everyone seems to be suggesting that the fix for the problem
is to somehow cobble together some way of forcing an unsigned
integer into a signed integer (what you would do with a cast
in C). However, if I understand the long<->int consolidation
this is not consistent with that effort.

As far as I can tell, the underlying problem is that the C
routine fcntl.ioctl is expecting a signed integer.

Well, that's what the man page says.

In practice it's just expecting an int-sized chunk of bits: it
wants a unique bit pattern to feed to a 'case' statement rather
than an "integer" in the number-line, arithmetic operations
sense of the word. C's implicit coercion rules make it a moot
point, but Python's coercion rules have been changed to be
incompatible with C's. Hilarity ensues.

These are the kinds of problems that need to be fixed. The
function should be asking for an unsigned integer. This is
possible with the C API at least since Python 2.3. Without
these fixes, the long<->int consolidation is going to continue
to produce frustration. There are many functions in the
standard library that you would expect to take unsigned
integers but actually specify signed integers.

Unfortunately the C API is cast in stone (at least in
comparison to Python standards). I guess somebody could go
through the C-Python code and "lie" to Python about it in order
to fix some of the issues.

What I would really, really like are fixed length integer types
so that I can manipulate 8, 16, 32 and maybe 64 bit, 2's
compliment values. I've seen some pretty good "user-space"
pure-python implimentations, but haven't gotten around to using
them in production yet.

One of the nasty bits in a pure-python approach is that there's
no way to write a literal with a fixed length. For example,
instead of writing 0xf7 to get an 8-bit value and 0x12345789 to
get a 32-bit value, you have to instantiate a class like
Word8(0xf7) and Word32(0x12345678).

That starts to make things pretty hard to read.

 

[Terry Reedy]

One of the nasty bits in a pure-python approach is that there's
no way to write a literal with a fixed length. For example,
instead of writing 0xf7 to get an 8-bit value and 0x12345789 to
get a 32-bit value, you have to instantiate a class like
Word8(0xf7) and Word32(0x12345678).

That starts to make things pretty hard to read.

This is no worse than having to write decimal(.53489384) or whatever to get
a decimal float rather than a binary float, or indeed, than writing
cname(init_data) to get an instance of all types/classes. There are many
more possible classes than sensible literal formats. A few basic and
general types have been blessed with literals that translate into inplicit
type constructor calls. Indeed, some literals seem necessary to start the
object construction process. However, most types and classes, including
your particular special-use classes, do not have corresponding literals and
never will in the general release.

If PyPy is successful in both being more flexible than CPython and at least
about as fast, then you might be able to customize an interpreter with more
builtin int classes and more careful parsing of int literals to initialize
them.a

Terry J. Reedy

 

[Bengt Richter]

What I would really, really like are fixed length integer types
so that I can manipulate 8, 16, 32 and maybe 64 bit, 2's
compliment values. I've seen some pretty good "user-space"
pure-python implimentations, but haven't gotten around to using
them in production yet.

One of the nasty bits in a pure-python approach is that there's
no way to write a literal with a fixed length. For example,
instead of writing 0xf7 to get an 8-bit value and 0x12345789 to
get a 32-bit value, you have to instantiate a class like
Word8(0xf7) and Word32(0x12345678).

That starts to make things pretty hard to read.

I'm not sure at what point you actually need "fixed width" other
than passing to some non-python interface, in which case an
interface object could have a suitable property to do the final
trimming or padding of bits. Or do you want to define some kind
of mathematical space? For specifying bits in literals see my
other post in this thread (I think ;-)

Regards,
Bengt Richter

 

[Grant Edwards]

This is no worse than having to write decimal(.53489384) or
whatever to get a decimal float rather than a binary float, or
indeed, than writing cname(init_data) to get an instance of
all types/classes. There are many more possible classes than
sensible literal formats. A few basic and general types have
been blessed with literals that translate into inplicit type
constructor calls. Indeed, some literals seem necessary to
start the object construction process. However, most types
and classes, including your particular special-use classes, do
not have corresponding literals and never will in the general
release.

Oh, I realize that. I was just fantasizing about features that
would make it easier to write the narrow set of applications
that I tend to write. I didn't mean that I thought the need
was widespread enough to seriously consider adding it to the
language.

 

[Reinhold Birkenfeld]

I give up, how do I make this not fail under 2.4?

fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))

I get an OverflowError: long int too large to convert to int

ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
the high-order bit set. I'm assuming Python thinks it's a
signed value. How do I tell Python that 0xc0047a80 is an
unsigned 32-bit value?

This is fixed in the 2.5 CVS branch, where the ioctl() Python C wrapper
expects an unsigned integer instead of a signed one.

Reinhold