Extracting 3-byte integers

[Bob Greschke]

I have some binary data read from a file that is arranged like

<3-byte int> <3-byte int> <3-byte int> etc.

The "ints" are big-endian and there are 169 of them. Is there any clever
way to convert these to regular Python ints other than (struct) unpack'ing
them one at a time and doing the math?

Thanks!

Bob

 

[John Machin]

I have some binary data read from a file that is arranged like

<3-byte int> <3-byte int> <3-byte int> etc.

The "ints" are big-endian and there are 169 of them. Is there any clever
way to convert these to regular Python ints other than (struct) unpack'ing
them one at a time and doing the math?

I'd call that "arithmetic", not "math"

Here's another way, not touted as "clever":

|>> guff = "\x00\x00\x01\x00\x02\x01\x03\x00\x00\x00\x00\xFF"
|>> expected = [1, 513, 3 * 65536, 255]
|>> expected
[1, 513, 196608, 255]

|>> import array
|>> b = array.array('B', guff)
|>> b
array('B', [0, 0, 1, 0, 2, 1, 3, 0, 0, 0, 0, 255])
|>> actual = [b[x]*65536 + b[x+1]*256 + b[x+2] for x in range(0, len(b), 3)]
|>> actual
[1, 513, 196608, 255]
|>> actual == expected
True

Cheers,
John

 

[Scott David Daniels]

I have some binary data read from a file that is arranged like
<3-byte int> <3-byte int> <3-byte int> etc.
The "ints" are big-endian and there are 169 of them. Is there any clever
way to convert these to regular Python ints other than (struct) unpack'ing
them one at a time and doing the math?

Best way is with scipy (or Numeric or numarray), but for vanilla Python:

import array, itertools
ubytes = array.array('B')
sbytes = array.array('b')
ubytes.fromfile(binarysource, 169 * 3)
sbytes.fromstring(ubytes[::3].tostring())

result = array.array('i', (msb * 256 + mid) * 256 + lsb
for msb, mid, lsb
in itertools.izip(sbytes,
ubytes[1::3], ubytes[2::3])))

If you want to get fancy, you can replace the last statement with:

del ubytes[::3]
uhalf = array.array('H')
uhalf.fromstring(ubytes.tostring())
if array.array('H', [1]) != '\x00\x01': uhalf.byteswap()
result = array.array('i', (msb * 65536 + low for msb, low
in itertools.izip(sbytes, uhalf)))


That was fun.

--Scott David Daniels
(e-mail address removed)

 

[John Machin]

I have some binary data read from a file that is arranged like

<3-byte int> <3-byte int> <3-byte int> etc.

The "ints" are big-endian and there are 169 of them. Is there any
clever way to convert these to regular Python ints other than (struct)
unpack'ing them one at a time and doing the math?

I'd call that "arithmetic", not "math"

Here's another way, not touted as "clever":

|>> guff = "\x00\x00\x01\x00\x02\x01\x03\x00\x00\x00\x00\xFF"
|>> import array
|>> b = array.array('B', guff)
|>> actual = [b[x]*65536 + b[x+1]*256 + b[x+2] for x in range(0, len(b),
3)]

Two further points:

(1) If using struct.unpack, it's not necessary to unpack 3 bytes at a
time; one could substitute b = struct.unpack('507B', guff) in the above.

(2) The OP didn't specify whether the ints were signed or unsigned. The
above is for unsigned. To convert an unsigned X to signed, here's the
"math":

if X >= 0x800000: X -= 0x1000000

Cheers,
John

 

[Bob Greschke]

I have some binary data read from a file that is arranged like

<3-byte int> <3-byte int> <3-byte int> etc.

The "ints" are big-endian and there are 169 of them. Is there any clever
way to convert these to regular Python ints other than (struct) unpack'ing
them one at a time and doing the math?

Thanks guys! I was looking for a 'one liner' of some kind, but I guess
there isn't one (too bad you can't tell array or struct.unpack how many
bytes you want an integer to be). Yeah, these are signed ints so I'll have
to convert them.

To me balancing my checkbook involves higher "math".

Bob