alligned memory allocation

K

Kutty Banerjee

Hi,
I ve got the following piece of code which does the role of allocating
aligned memory (not sure what aligned memory allocation is either).

void *
_align_calloc(size_t bytes, unsigned long alignment)
{
unsigned long ptr, buf;

ASSERT( alignment > 0 );

ptr = (unsigned long) calloc(bytes + alignment + sizeof(void *));
if (!ptr)
return NULL;

buf = (ptr + alignment + sizeof(void *)) & ~(unsigned long)(alignment -
1);
*(unsigned long *)(buf - sizeof(void *)) = ptr;

#ifdef DEBUG
/* mark the non-aligned area */
while ( ptr < buf - sizeof(void *) ) {
*(unsigned long *)ptr = 0xcdcdcdcd;
ptr += sizeof(unsigned long);
}
#endif

return (void *)buf;
}

Can somebody please explain what this code is doing? I m a little confused
with the memory being allocated to a non pointer data namely unsigned long
ptr ?


kutty
 
C

Christian Bau

"Kutty Banerjee said:
Hi,
I ve got the following piece of code which does the role of allocating
aligned memory (not sure what aligned memory allocation is either).

void *
_align_calloc(size_t bytes, unsigned long alignment)
{
unsigned long ptr, buf;

ASSERT( alignment > 0 );

ptr = (unsigned long) calloc(bytes + alignment + sizeof(void *));
if (!ptr)
return NULL;

buf = (ptr + alignment + sizeof(void *)) & ~(unsigned long)(alignment -
1);
*(unsigned long *)(buf - sizeof(void *)) = ptr;

#ifdef DEBUG
/* mark the non-aligned area */
while ( ptr < buf - sizeof(void *) ) {
*(unsigned long *)ptr = 0xcdcdcdcd;
ptr += sizeof(unsigned long);
}
#endif

return (void *)buf;
}

Can somebody please explain what this code is doing? I m a little confused
with the memory being allocated to a non pointer data namely unsigned long
ptr ?

It is a case of a clueless programmer trying to return a pointer that is
aligned in a certain way. I'll give you an overview what he is trying to
do and where he gets it wrong:

malloc and calloc return pointers that are properly aligned for _any_ C
type. Now there might be reasons why you would want a pointer that is
aligned to multiples of 128 bytes, for example, and malloc won't do
that.

So what he does is allocate a pointer that is bigger, lets say a pointer
"char* p", and in the case of 128 byte alignment you would add some
value from 0 to 127 to make it aligned on a multiple of 128 bytes. But
if you did just that and returned the result pointer, then you wouldn't
be able to free the memory later: To be able to call free, you must have
the original pointer. So you store the original pointer just in front of
the pointer that you return.

Now your original author does a few remarkably stupid things. He makes
assumptions that are completely unwarranted in portable C and actually
wrong on more and more computers that are in actual use. No wonder you
are confused :-(

First, he assumes that "alignment" is not only greater than zero, but
also a power of two, AND at least as large as the alignment requirement
of a void* value. Why doesn't he write this in the ASSERT? To test
whether x is a power of two: If you write x and x-1 in binary, then if x
is a power of two no bit will be set both in x and x-1. If x is nonzero
and not a power of two, then the highest bit of x is also set in x-1. So
you could write

ASSERT(align >= sizeof (void *) && (align & (align-1)) != 0);

Next, he assumes that you can cast a void* to unsigned long and perform
pointer arithmetic on the unsigned long. Bullshit. Take an Athlon 64 and
you may be surprised to see that void* = 64 bit, unsigned long = 32 bit,
and casting void* to unsigned long produces a whole lot of garbage. Take
16 bit DOS with "huge" memory model, and adding to an unsigned long to
do pointer arithmetic will produce garbage. Try it on a Cray and heaven
knows what you get. But in fact it is very easy to do what he wants in a
much more portable way:

char* p = calloc (bytes + alignment + sizeof (void *));
char* q = p + sizeof (void *) + alignment;
q -= ((unsigned long) q) % alignment;

// If you want your code unreadable like the original author,
// then change the last line to
// q -= ((unsigned long) q) & (alignment - 1);

Now it doesn't matter what the relative sizes of pointers and unsigned
long are.

The next mistake that he makes is that you can store an unsigned long if
you have space for a void*. Bad mistake. Just as you have systems where
sizeof (void *) > sizeof (unsigned long), you have systems where it is
the other way round. Buy a new Macintosh and you will get a machine that
uses 64 bit integers and 32 bit pointers. So he has this pointer "buf",
subtracts four byte = sizeof (void *), and stores an eight byte unsigned
long. Congratulations. The last four bytes of that unsigned long will be
overwritten by the caller as soon as they use the memory. Crash when you
try to free the memory later on.

Have fun with this guy's code. Expect the worst. People who think they
can program and can't are the worst kind.
 
C

CBFalconer

Christian said:
It is a case of a clueless programmer trying to return a pointer
that is aligned in a certain way. I'll give you an overview what
he is trying to do and where he gets it wrong:
.... snip ...

Have fun with this guy's code. Expect the worst. People who
think they can program and can't are the worst kind.

The code is obviously not portable, and is also obviously intended
for system use (the _align... name is not in the user name
space). This is a problem that is simply not soluble with
portable code, so the only requirement is that it work, somehow or
other, with the original implementation. Your caustic criticisms
are probably undeserved.

To the OP: The internal workings of a particular system are often
wierd and wonderful, and highly non-standard. You are not
expected to understand them on the basis of the language.
 
C

Christian Bau

CBFalconer said:
The code is obviously not portable, and is also obviously intended
for system use (the _align... name is not in the user name
space). This is a problem that is simply not soluble with
portable code, so the only requirement is that it work, somehow or
other, with the original implementation. Your caustic criticisms
are probably undeserved.

The point is that this code is unportable without any good reason. And
assuming that unsigned long and void* have the same size, today, is just
utterly stupid. Not the "it works everywhere except on the DeathStation
9000" kind of stupid, but the "I bought this brand new computer and my
code crashes all over the place" kind of stupid.
 
C

CBFalconer

Christian said:
The point is that this code is unportable without any good reason.
And assuming that unsigned long and void* have the same size, today,
is just utterly stupid. Not the "it works everywhere except on the
DeathStation 9000" kind of stupid, but the "I bought this brand new
computer and my code crashes all over the place" kind of stupid.

You yourself pointed out a reason for it - to provide alignment to
larger values than does malloc. Unsigned long and void* may well
have the same size on that system, they do under DJGPP. I believe
gcc requires such system routines (although I don't approve).
This is probably not the forum to ask questions about it, although
I am hard pressed to think of a suitable newsgroup.
 
M

Mark F. Haigh

CBFalconer said:
Christian Bau wrote:

The code is obviously not portable, and is also obviously intended
for system use (the _align... name is not in the user name
space). This is a problem that is simply not soluble with
portable code, so the only requirement is that it work, somehow or
other, with the original implementation. Your caustic criticisms
are probably undeserved.

I strongly disagree. As Christian pointed out, the original code
chooses unportability even though choosing (more) portability is
essentially free.

In fact, to me it's more unmaintainable. If you're doing something
like this, I immediately wonder why the author did not simply call
memalign then memset. Or simply wrote a small function to round the
alignment to the nearest power of two and then do a memalign and
memset. Now I just spent time that I shouldn't have had to spend.

Bad code is bad code, whether it works on the original implementation
or not. The mark of a bad programmer is that they write portable code
when there's a reason to. The mark of a good programmer is that they
write portable code when there is no reason not to.

Mark F. Haigh
(e-mail address removed)
 
C

Christian Bau

CBFalconer said:
You yourself pointed out a reason for it - to provide alignment to
larger values than does malloc. Unsigned long and void* may well
have the same size on that system, they do under DJGPP.
I believe
gcc requires such system routines (although I don't approve).

And I hope sincerely that they have been written better.
This is probably not the forum to ask questions about it, although
I am hard pressed to think of a suitable newsgroup.

char* p;
unsigned long x;

* (unsigned long *) (p - sizeof (void *)) = x;

is not non-portable, it is just plain stupid. The original poster was
confused _by the stupidity_ of the code. There were other errors in the
code that had nothing to do with portability, but just with stupidity.
 
M

Mark F. Haigh

Mark said:
I strongly disagree. As Christian pointed out, the original code
chooses unportability even though choosing (more) portability is
essentially free.

In fact, to me it's more unmaintainable. If you're doing something
like this, I immediately wonder why the author did not simply call
memalign then memset. Or simply wrote a small function to round the
alignment to the nearest power of two and then do a memalign and
memset. Now I just spent time that I shouldn't have had to spend.

Bad code is bad code, whether it works on the original implementation
or not. The mark of a bad programmer is that they write portable code
when there's a reason to. The mark of a good programmer is that they
write portable code when there is no reason not to.

yeah, yeah, memalign not in C standard, or POSIX for that matter, yeah
yeah. My point remains essentially the same. If memalign, snprintf,
etc, are not implemented on your system, then implement 'em and use 'em.
Even at a glance, the entire _align_calloc thing the OP quoted should
jump out at you and say 'probably a bad idea'.

Some of these supposed systems programming guys come up with this kind
of thing and then throw casts around until the compiler shuts up. This
is neither weird nor wonderful, just annoying to those of us that have
to clean up afterwards.

Or maybe I'm just cynical, you decide.



Mark F. Haigh
(e-mail address removed)
 
D

Dan Pop

In said:
You yourself pointed out a reason for it - to provide alignment to
larger values than does malloc.

malloc provides maximally aligned blocks, *by definition*:

The pointer returned if the allocation succeeds
is suitably aligned so that it may be assigned to a pointer to
any type of object and then used to access such an object or an
^^^^^^^^^^^^^^^^^^
array of such objects in the space allocated (until the space
is explicitly deallocated).

Dan
 
C

CBFalconer

Dan said:
malloc provides maximally aligned blocks, *by definition*:

The pointer returned if the allocation succeeds
is suitably aligned so that it may be assigned to a pointer to
any type of object and then used to access such an object or an
^^^^^^^^^^^^^^^^^^
array of such objects in the space allocated (until the space
is explicitly deallocated).

For the C language. There are entities in the world other than C
language objects, and sometimes C code is used to organize and
manipulate them.
 
D

Dan Pop

In said:
For the C language. There are entities in the world other than C
language objects, and sometimes C code is used to organize and
manipulate them.

How is this possible, without treating them as C objects?

Dan
 
X

xarax

Dan Pop said:
How is this possible, without treating them as C objects?

Depends on the hardware on which the C program is
running, which means that the C code is not intended
to be portable to other platforms, yet still have
defined behavior and not use implementation-dependent
compiler features.

The IBM Mainframe has hardware structures of various
alignments, like 64-byte, 256-byte, 4096-byte,
1MB alignment, and larger. There is no C compiler
"object" that conforms to those alignments, yet
it's necessary to represent such requirements in
C code. That requires pointer<->integer conversions
and arithmetic that yield defined behavior. In such
cases, malloc() will not yield a properly aligned
"object". The usual approach is to "over allocate"
with malloc() and massage the pointer upward to an
appropriate alignment boundary, saving somewhere the
original pointer for a subsequent free().

A smart implementation of calloc() and malloc()
would examine the length to determine a plausible
maximum alignment and attempt to return a pointer
that conforms to the plausible max alignment, but
that is not a requirement. They only need to conform
to maximum alignment of a primitive C object (like
a "long long" or a "double", whatever is largest in
the implementation).


--
----------------------------
Jeffrey D. Smith
Farsight Systems Corporation
24 BURLINGTON DRIVE
LONGMONT, CO 80501-6906
http://www.farsight-systems.com
z/Debug debugs your Systems/C programs running on IBM z/OS!
Are ISV upgrade fees too high? Check our custom product development!
 
C

CBFalconer

Dan said:
How is this possible, without treating them as C objects?

By taking advantage of system specific knowledge, and accepting
the non-portability of the result. Your system implementation of
malloc, free, and realloc is almost certainly a case in point. My
implementation of the same for DJGPP is certainly such. You can
find it on my site if curious.
 
C

Chris Torek

CBFalconer said:
[snippage]

In said:
For the C language. There are entities in the world other than C
language objects, and sometimes C code is used to organize and
manipulate them.

How is this possible, without treating them as C objects?

By faking it, of course.

Typically one might build a struct or array that (by chance *and*
purpose both) corresponds to the hardware's requirements, then use
non-portable trickery to cause the existing mechanisms to produce
values that work with this.

The result is not guaranteed by the language, certainly, and often
even the compiler-vendor may not guarantee it (other than internally
if the vendor is also in the hardware or hardware-support business).
But if it works, and is more maintainable than (e.g.) writing the
entire thing in assembly, that may be enough to cause one to use it.

The fact that a "struct page_table" (for instance) can only be
allocated by the special "page table allocator", and:

struct page_table tmp;
... use &tmp ...

fails miserably, is relevant to someone who thinks the system is
written in ANSI/ISO C, but not so important one who knows it is
much more restricted than that. In other words, these non-C-objects
*sometimes* work like real C objects, and sometimes do not, and we
(I can speak for more than just myself here :) ) pay this price
willingly for the value returned -- namely, some measure of
convenience. When it stops working well, we change the code
(and/or the compiler, in those cases where we control it).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,961
Messages
2,570,130
Members
46,689
Latest member
liammiller

Latest Threads

Top