Mapping a struct to a memory block....

[barcaroller]

I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

Example:

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

// memblock contains exactly 6 bytes
st* var = (st*) memblock;

 

[Ian Collins]

I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

Example:

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

// memblock contains exactly 6 bytes
st* var = (st*) memblock;

On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.

 

[Ben Pfaff]

I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

Compilers are allowed to insert padding between structure members
or after the end of a structure. The C language doesn't provide
a way to prevent or control this (although your implementation
might).

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

The standard type names for these types would be int16_t and
int32_t, declared in <stdint.h> in C99 implementations that
support them.

 

[Keith Thompson]

I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

[...]

You have to worry about the compiler padding the struct.

See question 2.12 in the comp.lang.c FAQ, <http://www.c-faq.com/>.

 

[barcaroller]

On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.

Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.

 

[Ian Collins]

Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.

They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.

 

[barcaroller]

They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner. Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).

 

[Ian Collins]

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner. Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).

Probably. I've never seen a case where this isn't true, but you never know!

 

[Keith Thompson]

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner.

Strictly speaking, I think that's only required within a single
execution of a program, but practically speaking you can pretty much
depend on binary files being compatible as long as the same compiler
was used to compile the code that writes them and the code that reads
them. (But some compilers may provide command-line options that
affect layout.)

It's likely, but by no means guaranteed, that layouts will be
consistent across different compilers on the same system. Compiler
vendors are motivated to make things easy for their users.

Across different platforms, though, all bets are off unless you use
text-based formats.

Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).

Yes.

 

[Mark McIntyre]

Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.

Its safe provided both reads and writes are being performed by the
same version of the same software on the same hardware, compiled by
the same compiler...

People make good money writing tools to convert binary data of this
sort between versions, platforms and applications.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[Mark McIntyre]

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner.

Provided you don't change any of the compile or link flags.
Or upgrade to a new version or patchlevel of the compiler.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[Francine.Neary]

Compilers are allowed to insert padding between structure members
or after the end of a structure. The C language doesn't provide
a way to prevent or control this (although your implementation
might).

That's true, though you can work out how much padding there is using
the famous offsetof macro I was asking about a while ago. This seems
like a pretty good use of offsetof to me - lets you unpack a struct to
write a portable binary representation to file.

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

The standard type names for these types would be int16_t and
int32_t, declared in <stdint.h> in C99 implementations that
support them.
--
int main(void){char p[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz.\
\n",*q="kl BIcNBFr.NKEzjwCIxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+=strchr(p,*q++)-p;if(i>=(int)sizeof p)i-=sizeof p-1;putchar(p\
);}return 0;}

 

[Ian Collins]

That's true, though you can work out how much padding there is using
the famous offsetof macro I was asking about a while ago. This seems
like a pretty good use of offsetof to me - lets you unpack a struct to
write a portable binary representation to file.

No it doesn't, you are assuming both the reader and writer use the same
padding, endian representation and data model or the information is in
the file.

Your sig is broken, the separator should be "-- "

 

[Ben Pfaff]

Your sig is broken, the separator should be "-- "

There's nothing wrong with my sig. Someone else misquoted it.

 

[Ian Collins]

There's nothing wrong with my sig. Someone else misquoted it.

Sorry, I didn't spot it was quoted.