uint32_t 0x80000000 unpacked as 0x7FFFFFFF

[A. Farber]

Hello,

I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
to unpack a C-structure received over a unix pipe
(where event_type is enum and SID_LEN is 32):

typedef struct request_header {
char sid[SID_LEN + 1];
event_type event;
time_t modified;
uint32_t arg;
} request_header;

I try to unpack it this way:

($self->{SID}, $self->{EVENT},
$self->{MOD}, $self->{ARG}) =
unpack 'A32x4L3', $self->{REQUEST};

It mostly works well, but for the ARG 0x80000000
I wrongly get 0x7FFFFFFF (which is same but negated?)

Could someone please advise me?

This signed/unsigned binary stuff is so confusing :-(
I tried to read perlpacktut several times, never got it 100%.

Thank you
Alex

 

[John W. Krahn]

I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
to unpack a C-structure received over a unix pipe
(where event_type is enum and SID_LEN is 32):

typedef struct request_header {
char sid[SID_LEN + 1];
event_type event;
time_t modified;
uint32_t arg;
} request_header;

char is a built-in C type that is defined by the standard but
event_type, time_t and uint32_t may be a macro or typedef somewhere in
your program and you have to determine what C type they represent in
order to unpack them.

I try to unpack it this way:

($self->{SID}, $self->{EVENT},
$self->{MOD}, $self->{ARG}) =
unpack 'A32x4L3', $self->{REQUEST};

char sid[SID_LEN + 1]; implies, but does not guarantee, a C "string" so
you probably need 'Z33' to unpack it. Also, why are you skipping 4
bytes between $self->{SID} and $self->{EVENT}?

It mostly works well, but for the ARG 0x80000000
I wrongly get 0x7FFFFFFF (which is same but negated?)

They are two different numbers:

$ perl -le 'print for 0x80000000, 0x7FFFFFFF'
2147483648
2147483647

Could someone please advise me?

This signed/unsigned binary stuff is so confusing :-(

I think that char depends on the compiler? Some define it as signed and
some as unsigned? I have no idea whether event_type is signed or
unsigned or what size it is. time_t, I believe, is usually a signed
type but the size depends on how it is defined. And the 'u' at the
front of uint32_t implies that it is an unsigned type that is 32 bits wide.

I tried to read perlpacktut several times, never got it 100%.

John

 

[Jens Thoms Toerring]

I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
to unpack a C-structure received over a unix pipe
(where event_type is enum and SID_LEN is 32):

typedef struct request_header {
char sid[SID_LEN + 1];
event_type event;
time_t modified;
uint32_t arg;
} request_header;

I try to unpack it this way:

($self->{SID}, $self->{EVENT},
$self->{MOD}, $self->{ARG}) =
unpack 'A32x4L3', $self->{REQUEST};

It mostly works well, but for the ARG 0x80000000
I wrongly get 0x7FFFFFFF (which is same but negated?)

I hope you realize that this is extremely system dependend.
The sizes of at least 'event_type' and 'time_t' depend on
the system you're using as well as the number of padding
bytes that may exist between the members of the structure
(that may even be depend on the C compiler you're using!).

And 0x80000000 and 0x7FFFFFFF are different uint32_t numbers,
the latter being 1 less than the former.

As John Kramer already pointed out, for the 33 byte long
string you need "Z33" to unpack it. Due to padding bytes
between the 'sid' and 'event' member the "x3" is needed
(as long as there are three bytes of padding).

Things ain't getting simpler from here. If I am not
mistaken, an enum value must be an int. So to unpack
it using "i" seems to be the best choice since it
should pick the right size automatically.

Next you've got to check if there are padding bytes following
the 'event_type' member and, if necessary skip them with enough
"x" - on my machine there are none.

'time_t' is even worse since al the C standard says about that
type is that it's "arithmetic type capable of representing times".
So it could be a signed or unsigned int or long but as well a
floating point type. All you can do is try to determine from
your header files what it is on your system. On mine (which
is x64) it seems to be a signed 64-bit integer value. Thus
on my machine I have to use "q" for unpack() - but that may
be different on a 32-bit system!

Now again padding bytes may follow... None so on my machine.

The final 'uint32_t' is simple since here only "L" is to be
considered.

Taking it all together I need on my machine

unpack "Z33x3iqL", $self->{ REQUEST };

to pick the data apart. It may need something else on yours.

That's why it is normally not a good idea to send structures
in binary form around since it's layout and the endian-ness
and sizes of the members might differ from machine to machine
(and even compiler to compiler).

Regards, Jens

 

[A. Farber]

Hello,

I've prepared a simple test case and
found the real reason for my problem -
the argument for atol() was out of range.

I'll probably switch to strtol()
in the C part of my C+Perl program.

Yes, I was using A32x4 because the terminating
null takes 4 bytes (because of padding).

$ cat test-case.c
#include <stdio.h>
#include <unistd.h>

#define SID "01234567890123456789012345678901"
#define SID_LEN 32

typedef enum event_type {
ALIVE = 1 << 0,
CHAT = 1 << 1,
JOIN = 1 << 2,
LOBBY = 1 << 3,
INDEX = 1 << 4,
END = 1 << 5,
EXIT = 1 << 6
} event_type;

typedef struct request_header {
char sid[SID_LEN + 1];
event_type event;
time_t modified;
uint32_t arg;
} request_header;


int main() {
request_header req;

strcpy(req.sid, SID);
req.event = INDEX;
req.modified = 0;
/* req.arg = 0x80000000; */
req.arg = atol("2147483648");

return sizeof(req) != write(STDOUT_FILENO, &req, sizeof(req));
}

$ cat test-case.pl
#!/usr/bin/perl -w

use strict;
use Data:umper;

my ($req, $href);

$req = qx(./test-case);

($href->{SID}, $href->{EVENT}, $href->{MOD}, $href->{ARG}) =
unpack 'A32x4L3', $req;

print Dumper($href);
printf "%x\n", $href->{ARG};

$ ./test-case.pl
$VAR1 = {
'SID' => '01234567890123456789012345678901',
'MOD' => 0,
'EVENT' => 16,
'ARG' => 2147483647
};
7fffffff

$ perl -v
This is perl, v5.8.8 built for i386-openbsd

$ gcc -v
gcc version 3.3.5 (propolice)


Thanks for comments
Alex

 

[A. Farber]

I hope you realize that this is extremely system dependend.

This isn't too bad in my case, because it is
an Apache-module in C talking over a Unix-pipe to
a daemon in Perl, i.e. both run at the same machine...

Regards
Alex

 

[Jens Thoms Toerring]

This isn't too bad in my case, because it is
an Apache-module in C talking over a Unix-pipe to
a daemon in Perl, i.e. both run at the same machine...

Ok. It's just that the same program wouldn't work on my 64-bit
machine. And another point: for the enumerated type you should
not unpack() for an unsigned value, they're definitely signed.
So better use either "l" or "i" instead of "L" for that one.

Regards, Jens

 

[Jens Thoms Toerring]

Hello,

I've prepared a simple test case and
found the real reason for my problem -
the argument for atol() was out of range.

I'll probably switch to strtol()
in the C part of my C+Perl program.

Yes, I was using A32x4 because the terminating
null takes 4 bytes (because of padding).

$ cat test-case.c
#include <stdio.h>
#include <unistd.h>

#define SID "01234567890123456789012345678901"
#define SID_LEN 32

typedef enum event_type {
ALIVE = 1 << 0,
CHAT = 1 << 1,
JOIN = 1 << 2,
LOBBY = 1 << 3,
INDEX = 1 << 4,
END = 1 << 5,
EXIT = 1 << 6
} event_type;

typedef struct request_header {
char sid[SID_LEN + 1];
event_type event;
time_t modified;
uint32_t arg;
} request_header;

int main() {
request_header req;

strcpy(req.sid, SID);
req.event = INDEX;
req.modified = 0;
/* req.arg = 0x80000000; */
req.arg = atol("2147483648");

Here you try to convert to a signed long (that's what the
return type of atol() is). But that number (2^31 or, in
hex, 0x80000000) doesn't fit into a 32-bit long (which
seems to be what you have on your machine), and thus
atol() returns LONG_MAX, which is 2147483647 (0x7FFFFFFFF)
on your machine.

Using strtol() wouldn't change that since it also will
return LONG_MAX on too large a number. You better use
the strtoul() function instead (uint32_t and unsigned
long being obviously the same on your machine).

Regards, Jens

 

[A. Farber]

Using strtol() wouldn't change that since it also will
return LONG_MAX on too large a number. You better use
the strtoul() function instead (uint32_t and unsigned
long being obviously the same on your machine).

gcc says:

module/mod_pref.c:74: warning: implicit declaration of function
`strtoul_is_not_a_portable_function_use_strtol_instead'

Regards
Alex

 

[Jens Thoms Toerring]

gcc says:

module/mod_pref.c:74: warning: implicit declaration of function
`strtoul_is_not_a_portable_function_use_strtol_instead'

That's rather surprising since strtoul() is required by
the C standard to exist, and that since 20 years now.
Did you include <stdlib.h>, where it's declared? And
then my version of gcc (4.3.2) doesn't utter any com-
plaints about the use of strtoul() even when invoked
with

gcc -std=c89 -W -Wall -ansi -pedantic

A web search for the warning seems to indicate that
some Apache header file may result in this nonsense
getting spit out if strtoul() isn't declared (i.e.
if you forgot to include <stdlib.h>). I am at a loss
to understand what that is supposed to be good for
(except that it's, of course, a bad idea to try to
use the function without a declaration in scope).

Regards, Jebs

 

[John W. Krahn]

As John Kramer already pointed out,

John Kramer?


John

 

[Tad J McClellan]

John Kramer?

Maybe he meant Cosmo Kramer?

 

[Jens Thoms Toerring]

John Kramer?

Sorry, I meant you, of course. I guess I have to
get my glasses checked...
Regards, Jens