substitute for string 0 termination

[CBFalconer]

However, most news servers I know of these days don't accept it,
it's too easily abused by attackers (cancels similarly). And once
it's got to a client it's unlikely that supersedes will do anything
apart from look like a new message...

However, an adequate substitute is to gaze admiringly at your own
superb and intelligent reply, and generally drink in the subtleties
involved, before hitting that final 'send' key.

If you read and post offline you can even postpone this idolatrous
self-admiration session until just before going back on line. That
may give you a more encompassing view of the positive effects you
are bringing to the universe.

 

[CBFalconer]

Which one is that, how do you that whether it is properly aligned for
*any* C type?

The DS9 requires that all the log2(sizeof(type)) bits of the
address be 1 bits. This allows it to discard those bits in pointer
representations.

 

[Michael Mair]

Maybe there is an error in my train of thought, but I thought
about:
MALLOC_WRAPPER(size)
checking whether size is large enough for "small header+safety distance"
I assume it is, otherwise this goes with n*size/(n-1)*size:
[...]

C89:
We run into problems as soon as the user is not honest with us because
(s)he allocates "too much". Example: struct hack.
Our memory alignment requirements may be violated.

That isn't dishonest, it's the only way to get a variable amount of
data. Indeed, you're doing a form of it yourself.

Mmmh... The struct hack is non-conforming and works in the realm of
UB even though it seems to do the same thing everywhere.
However, nobody can disallow me to malloc(size + 23) whenever
I malloc something -- there goes the method.
What I am doing is mainly hoping that whatever is to be allocated
can be put into an array.
As I mentioned: If you make a special wrapper for struct hack/flexible
array member, you still can use this method.

For exactly the same reasons, it won't work, and this use is now blessed
by the Standard.

Right. And by allocating too much memory (as we are already doing
so why bother with more waste?) we can still use the above approach.

So, if used correctly and within these restrictions (and with an
overallocating solution for struct hack/f.a.m. working essentially
along the above lines but in the other direction), these should work.

If your restrictions are sufficient, yes, a non-general solution can be
made to work. Another is to use:

union AlignStuff
{
int i;
long l;
long long ll;
void *vp;
} AS[2];

and whatever other types you know your system uses, and use

ptrdiff_t align = (char*)AS[1] - (char*)AS[0];

(because that is guaranteed to give the distance between two objects
aligned for the worst case, so allocate space for your overhead in
n*align bytes). You can include in that all of the types you know your
program uses, but make sure that if other types are added they are
included in the union as well (for instance, a pointer to long or to a
function might be a different alignment from pointer to void).

Hmmm, but this is still no "one size fits all" approach.
I do not like the waste of the other method but within its
restrictions, you can program it in a portable way.

As I said a few weeks back here, it is possible to make a general
allocator which isn't portable (because it knows about system details),
or a portable one which isn't general (because it doesn't know those
details), but not one which is both portable and general within the C
language (if you can run a program to get information from the system or
interactively when installing, for instance from the GNU autoconf
configure script, then you can make it portable and general -- as long
as your program or script is itself portable!).

Yep. But I was not looking for a general allocator.

BTW: You always can store the addresses allocated via malloc()
together with other information as requested size at the last
malloc()/realloc(). This of course gives you a non-constant
access time. The above approach is more or less just playing
around to find out inhowfar you can realise the "I store the
information just at a negative offset w.r.t beginning of
the storage area"

Question:

In my suggestion above, is (char*)AS[1] - (char*)AS[0] guaranteed to
be the same as sizeof(AS[0])? Where?

I am not sure about using ptrdiff_t at all.

You probably look for

"6.5.3.4 The sizeof operator
[...]

3 [...]
When applied to an operand that has structure or union type, the result
is the total number of bytes in such an object, including internal and
trailing padding.
"

"6.7.2.1 Structure and union specifiers
[...]

14 The size of a union is sufficient to contain the largest of its
members. The value of at most one of the members can be stored in a
union object at any time. A pointer to a union object, suitably
converted, points to each of its members (or if a member is a bit-
field, then to the unit in which it resides), and vice versa.

15 There may be unnamed padding at the end of a structure or union.
"


Cheers
Michael

 

[Bart C]

However, an adequate substitute is to gaze admiringly at your own
superb and intelligent reply, and generally drink in the subtleties
involved, before hitting that final 'send' key.

I suppose actually *testing* the code first is out of the question?

Any code I haven't tried I'd mark as 'untested', which is OK if it's just to
convey an idea.

Bart

 

[Richard Bos]

Er, not so: You could allocate the double, triple, ... amount of the
memory and pass only the address of the last "element" (i.e. the one
with the right size towards the end of allocated memory).

Take the following snippet:

int *ip;
char *cp;

ip=extra_malloc(sizeof *ip + sizeof char);
cp=ip + sizeof *ip;
*cp='a';
*ip=1438;

Now suppose sizeof(int) is 4, and ints must be properly aligned on
4-byte addresses. Had I used the normal, implementation-provided
malloc(), the above code would have been perfectly conforming. A little
odd, perhaps, but perfectly OK. malloc() returns a good, omni-aligned
pointer; ip is a valid, well-aligned int *; and cp, being a char *,
cannot be unaligned.
Ok, but now with the extra header. What I do is call extra_malloc(5).
extra_malloc, presumably, calls malloc(5). _This_ pointer is aligned for
everything, including an int. Now it adds 5 to that pointer. The result
is not properly int-aligned any more. Yet it gets passed out as if it's
a perfectly good pointer, and I assign it to an int *...

Richard

 

[pete]

Bart C wrote:

Any code I haven't tried I'd mark as 'untested',
which is OK if it's just to convey an idea.

I don't see any point in marking code "untested".
All posted code is subject to review and
code marked "tested" has no real guarantee.

 

[Michael Mair]

Take the following snippet:

int *ip;
char *cp;

ip=extra_malloc(sizeof *ip + sizeof char);
cp=ip + sizeof *ip;
*cp='a';
*ip=1438;

Now suppose sizeof(int) is 4, and ints must be properly aligned on
4-byte addresses. Had I used the normal, implementation-provided
malloc(), the above code would have been perfectly conforming. A little
odd, perhaps, but perfectly OK. malloc() returns a good, omni-aligned
pointer; ip is a valid, well-aligned int *; and cp, being a char *,
cannot be unaligned.
Ok, but now with the extra header. What I do is call extra_malloc(5).
extra_malloc, presumably, calls malloc(5). _This_ pointer is aligned for
everything, including an int. Now it adds 5 to that pointer. The result
is not properly int-aligned any more. Yet it gets passed out as if it's
a perfectly good pointer, and I assign it to an int *...

Which is the issue I already addressed in the other subthread.
It works only if the user gives me the size of an object which
can be an array member. However, with allocating more memory for
things like the struct hack (or flexible array members) by telling
the size of the original "array-able" type plus the required extra
memory, you could again "heal" it

Just for the record: I think this is really a stupid way of doing
it but it is feasible and can be implemented in standard C.


Cheers
Michael

 

[Ben Pfaff]

I don't see any point in marking code "untested".
All posted code is subject to review and
code marked "tested" has no real guarantee.

I have certain standards for code that I post. Part of that
includes testing. At the very least, it will uncover typos,
omissions of headers, and so on. If I don't take the time that I
normally would to test code, I often mark it "untested" to warn
people. I don't see why this is a bad thing.

 

[Michael Wojcik]

I understand this of course, but still I'd like to know why a function like
memsize can't be introduced in the std library.

Here are some good reasons:

- There may be extant implementations where the *implementation* does
not, in fact, keep track of the size of the allocated area. The
implementation may be running in a hosted environment which itself
handles allocation, and malloc, calloc, realloc, and free might just
be wrappers. Your "memsize" could require a substantial and
infelicitous change to such an implementation.

- Either memsize must have defined behavior (whether explicit in the
standard or implementation-defined) for arguments which are not
pointers returned by malloc/calloc/realloc, or it must have undefined
behavior for them. The former would require implementations to be
able to identify pointers to allocated memory and distinguish them
from arbitrary values, which is unreasonable. The latter would
introduce a whole new class of nasty bugs. (Essentially these would
be the same bugs that already exist for free and realloc, but the
mooted memsize would be useful in many contexts where those two are
not, so it would likely be used incorrectly far more often.)

- If memsize always returns the requested size, the implementation now
has to keep track of two sizes: requested and actual.

- If memsize always returns the actual size, you've created potential
skew in the program logic. The allocating function may be written to
treat the maximum amount of data in the allocated area as the
requested size, while another function may use memsize (because,
presumably, the use of memsize is to avoid keeping track of the
requested size) and fill the area to its actual size. Return to the
allocating function and you have silent data truncation. It's easy
to see how this might happen, particularly in maintenance.

What it really comes down to for me, though, is that there few good
purposes served by "memsize" - far fewer than many people seem to
think. A program should keep track of the requested sizes of the
areas it allocates, and treat them as being no larger. Some have
suggested that memsize would be useful for a library which wants to
verify that a receiving area is sufficiently large, but that still
would require a valid pointer from the caller - and what if the
caller wants to use an auto or static array, or a pointer into the
middle of an allocated area, as the destination instead?

--
Michael Wojcik (e-mail address removed)

Thanks for your prompt reply and thanks for your invitatin to your
paradise. Based on Buddihism transmigration, I realize you, European,
might be a philanthropist in previous life! -- supplied by Stacy Vickers

 

[Joe Wright]

I have certain standards for code that I post. Part of that
includes testing. At the very least, it will uncover typos,
omissions of headers, and so on. If I don't take the time that I
normally would to test code, I often mark it "untested" to warn
people. I don't see why this is a bad thing.

There are several degrees of testing. When I post code here I do expect
comment. "This crap won't even compile!" is not what I want to hear.

Wrap your code example in a real program that demonstrates the point of
the code example. Test it to the point that it compiles cleanly and
makes your point. Post the whole program.

 

[Michael Mair]

Which is the issue I already addressed in the other subthread.
It works only if the user gives me the size of an object which
can be an array member. However, with allocating more memory for
things like the struct hack (or flexible array members) by telling
the size of the original "array-able" type plus the required extra
memory, you could again "heal" it

Just for the record: I think this is really a stupid way of doing
it but it is feasible and can be implemented in standard C.

Just for the record: I still think that this is a really stupid way
to do it.

However, I took the liberty of implementing it:

http://kreativprojekt.gmxhome.de/downloads/overalloc.zip
(~6K, files are unix-like text files)

Compile with -DTEST/#define TEST for some basic tests.

Suggestions welcome. However, this is only a couple of hours'
work and certainly nothing with a well-thought-out interface.

Code with comments was too long to post here.


So, in the future, whenever someone asks for memsize(), you can
say "There is a really wasteful but standard conforming(*) way
of doing it; have a look at overalloc"
(*) If you use it correctly


Cheers
Michael

 

[Mabden]

If you read and post offline...

What is this "offline"?

Is that like "offphone" or "offwater"? I had an "offelectric" episode
once, but we went "oncandle" to fix it.

I pray there will never be "offbeer"...!

 

[pete]

I have certain standards for code that I post. Part of that
includes testing. At the very least, it will uncover typos,
omissions of headers, and so on. If I don't take the time that I
normally would to test code, I often mark it "untested" to warn
people. I don't see why this is a bad thing.

It's not very bad, but it doesn't mean much to me.
It doesn't affect whether or not I will test your code.
It only affects my expectations when I test your code,
but I don't consider my expectations to be very important.

 

[Ben Pfaff]

It's not very bad, but it doesn't mean much to me.
It doesn't affect whether or not I will test your code.
It only affects my expectations when I test your code,
but I don't consider my expectations to be very important.

Perhaps it is not important whether code I post is tested, if you
are the one reading it. Clearly, you have the knowledge needed
to evaluate my code on its own merits, regardless of what I say
about it. Statements about thoroughness of testing are not for
you. They are for the novices who frequent this newsgroup,
ill-equipped to evaluate the quality of responses to their
questions.

 

[pete]

Perhaps it is not important whether code I post is tested, if you
are the one reading it. Clearly, you have the knowledge needed
to evaluate my code on its own merits, regardless of what I say
about it. Statements about thoroughness of testing are not for
you. They are for the novices who frequent this newsgroup,
ill-equipped to evaluate the quality of responses to their
questions.

Thank you.

 

[Dave Thompson]

Take the following snippet:

int *ip;
char *cp;

ip=extra_malloc(sizeof *ip + sizeof char);

[snip: as an attempted substitute for malloc]

cp=ip + sizeof *ip;

FTR you meant (char*)ip + sizeof *ip
or (char*)(ip + 1) or (char*)&ip[1]

*cp='a';
*ip=1438;

- David.Thompson1 at worldnet.att.net