Does malloc() reuse addresses?

[avasilev]

I don't think this will help him. If he has multiple modules with a
pointer to an object and any one of them may free it at any time, with
a custom or non custom free, there is a design flaw that needs to be
fixed.

Yep, practically using a custom free function which checks and NULL-s
all possible references is not feasible. If nothing else, it is
error-prone. Imagine I add some other place where an instance is
referenced - I have to change the free function every time.

 

[Chris Torek]

Background: the issue involves using free()d pointers, as in the
following code fragment:

void *p1, *p2;
int i;
void *table[MAX];
...
p1 = malloc(N);
p2 = malloc(N);
table = p1;
free(p);
if (table == p2) ... do something ...

No no, Im no examining the moemory that hte pointer points to, I am
simply comparing the values of the pointers themselves, i.e. I am
comparing the addresses, not the contents of the memory that is pointed
to.

This may not help (although it happens to work on most machines today).
See <http://web.torek.net/c/numbers2.html>.

 

[Chris Torek]

(More on using free()d memory, and/or malloc() later returning a pointer
value that was at one point free()d.)

[There is a] global
variable instList, this is the pointer to the whole list. When an
instance is to be freed, it is first removed from the list, and then
free()-d (see code). So all pointers in the list are always valid.
Unfortunately I cannot keep track of all copies of these pointers that
fly around in the application. Thats why the whole problem arises.

Let me boil this down to some sample code:

/* xalloc - just like malloc, but panics (aborts) if out of memory */
extern void *xalloc(size_t);

struct obj {
struct obj *next;
... data ...
};
static struct obj *head; /* "global" list head */

/* create a new object, putting it in the list */
struct obj *newobj(void) {
struct obj *new = xalloc(sizeof *new);
new->next = head;
head = new;
...
return new;
}

/* delete an object, which must be in the list */
void deleteobj(struct obj *obj) {
struct obj *p, **pp;

for (pp = &head; (p = *pp) != NULL; pp = &p->next)
if (p == obj)
break;
if (p == NULL)
panic("asked to delete object not in list");

/* at this point, p==obj and it is in the list */
*pp = p->next;
free(p);
}

/* return true if the object is in the list */
int testobj(struct obj *obj) {
struct obj *p;

for (p = head; p != NULL; p = p->next)
if (p == obj)
return 1;
return 0;
}

Now you can call testobj() with a valid pointer, to see if it is
in the global list:

struct obj *p1, *p2;

p1 = newobj();
p2 = newobj();

if (testobj(p1)) ...
if (testobj(p2)) ...

These are all OK (albeit useless; obviously p1 and p2 *are* in
the list at this point).

Here is where things go wrong:

void f2(void) {
struct obj *p1 = newobj();

use(p1);
if (testobj(p1)) ...
...
}

where use() looks like this:

void use(struct obj *p) {
deleteobj(p);
}

Here, the problem is that use() got a copy of the value in p1, and
asked deleteobj() to delete the object. But its caller (f2()) then
continues to use the object.

This is not something that can be detected, in general. One might
as well ask f2() to detect code like this:

void use(struct obj *p) {
memcpy(p, 0xff, 10000); /* smash a bunch of memory, whee! */
}

The problem is not in f2(); the problem is in use(). It cannot be
solved in f2(); it must be solved in use(). The use() function
lights the fuse on a time-bomb; there is no predicting just when
it will go off.

This does not mean that debug routines -- code that gives you a
chance to set the bomb off faster -- are *useless*, just that they
can never be made perfect. The bomb may be trickier than the
defusing code.

If you could find a machine that actually used segment registers
to catch illegal use of free()d pointers, you would not even need
the debug routine. Instead of a delayed-fuse time-bomb, the original
use() -- which incorrectly deletes the object while it is still in
use) -- would set a *non*-delayed fuse that would trigger immediately
upon the return to f2(), when f2() attempts to do any other operation
on p1 (except of course something like "p1 = NULL").

Alas, nobody builds machines designed for debugging anymore.

Now, if you *want* to be able to "pseudo-delete" objects, and
have them only go away when everyone is *really* done with them,
you need a garbage collector, or something that is sufficiently
similar. In this case, you might change "obj" to add a reference
count to it, and have gainref() and deleteref() calls, with the
object going away when its last reference goes away:

struct obj {
int refcnt;
... data ...
};


/* create a new object */
struct obj *newobj(void) {
struct obj *new = xalloc(sizeof *new);
new->refcnt = 1;
...
return new;
}

/* gain an extra reference to an object */
void gainref(struct obj *p) {
p->refcnt++;
}

/* lose a reference to an object */
void deleteref(struct obj *p) {
if (p->refcnt <= 0)
panic("invalid refcnt in deleteref");
if (--p->refcnt == 0)
free(p);
}

Now a function can just add a reference at any time, and delete
one whenever it is done. When the reference count goes to zero,
the object goes away; but not until then.

Of course, a reference counting system only works if you do not
have loops in the data structures (or if those data structures do
not "count themselves", as it were, but this last is terribly
tricky). In this case, you might be better served by a language
that includes built-in garbage-collection.

 

[av]

(More on using free()d memory, and/or malloc() later returning a pointer
value that was at one point free()d.)

[There is a] global
variable instList, this is the pointer to the whole list. When an
instance is to be freed, it is first removed from the list, and then
free()-d (see code). So all pointers in the list are always valid.
Unfortunately I cannot keep track of all copies of these pointers that
fly around in the application. Thats why the whole problem arises.

Let me boil this down to some sample code:

/* xalloc - just like malloc, but panics (aborts) if out of memory */
extern void *xalloc(size_t);

struct obj {
struct obj *next;
... data ...
};
static struct obj *head; /* "global" list head */

/* create a new object, putting it in the list */
struct obj *newobj(void) {
struct obj *new = xalloc(sizeof *new);
new->next = head;
head = new;
...
return new;
}

/* delete an object, which must be in the list */
void deleteobj(struct obj *obj) {
struct obj *p, **pp;

for (pp = &head; (p = *pp) != NULL; pp = &p->next)
if (p == obj)
break;
if (p == NULL)
panic("asked to delete object not in list");

/* at this point, p==obj and it is in the list */
*pp = p->next;
free(p);

here p=0;

}

/* return true if the object is in the list */
int testobj(struct obj *obj) {
struct obj *p;

for (p = head; p != NULL; p = p->next)
if (p == obj)
return 1;
return 0;
}

Now you can call testobj() with a valid pointer, to see if it is
in the global list:

struct obj *p1, *p2;

p1 = newobj();
p2 = newobj();

if (testobj(p1)) ...
if (testobj(p2)) ...

These are all OK (albeit useless; obviously p1 and p2 *are* in
the list at this point).

Here is where things go wrong:

void f2(void) {
struct obj *p1 = newobj();

use(p1);
if (testobj(p1)) ...
...
}

where use() looks like this:

void use(struct obj *p) {
deleteobj(p);
}

Here, the problem is that use() got a copy of the value in p1, and
asked deleteobj() to delete the object. But its caller (f2()) then
continues to use the object.

this is not a problem if
void use(struct obj *p) { deleteobj(p); p=0; }
or you change deleteobj like above

otherwhise it is a bug but for me it has not segfault


it have to segfaul only when deference for write
it is possible i like to read memory out my program

This is not something that can be detected, in general. One might
as well ask f2() to detect code like this:

void use(struct obj *p) {
memcpy(p, 0xff, 10000); /* smash a bunch of memory, whee! */
}

The problem is not in f2(); the problem is in use(). It cannot be
solved in f2(); it must be solved in use().

no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p
no, f2 can see the problem when free p

The use() function
lights the fuse on a time-bomb; there is no predicting just when
it will go off.

time-bomp is only the code that not self check memory when free it

This does not mean that debug routines -- code that gives you a
chance to set the bomb off faster -- are *useless*, just that they
can never be made perfect. The bomb may be trickier than the
defusing code.

this is always true but i have 99% of probability to find there is
something wrong and where

If you could find a machine that actually used segment registers
to catch illegal use of free()d pointers, you would not even need
the debug routine. Instead of a delayed-fuse time-bomb, the original
use() -- which incorrectly deletes the object while it is still in
use) -- would set a *non*-delayed fuse that would trigger immediately
upon the return to f2(), when f2() attempts to do any other operation
on p1 (except of course something like "p1 = NULL").

Alas, nobody builds machines designed for debugging anymore.

Now, if you *want* to be able to "pseudo-delete" objects, and
have them only go away when everyone is *really* done with them,
you need a garbage collector, or something that is sufficiently

malloc has need a function for check memory when free
garbage collettor is wrong
it is good the c++ way (constructors/destructors)


don't understand the remain are not so smart
then the rest it seems too long and i'm lazy
can you summarize?

 

[av]

here p=0;

wrong it seems here i can not chage p
....

this is not a problem if
void use(struct obj *p) { deleteobj(p); p=0; }
or you change deleteobj like above

ok i make errors but all you too

 

[Michael Wojcik]

If you could find a machine that actually used segment registers
to catch illegal use of free()d pointers, you would not even need
the debug routine. Instead of a delayed-fuse time-bomb, the original
use() -- which incorrectly deletes the object while it is still in
use) -- would set a *non*-delayed fuse that would trigger immediately
upon the return to f2(), when f2() attempts to do any other operation
on p1 (except of course something like "p1 = NULL").

I can find one. In fact, I have a pretty good idea where it is right
now.

(It does not use "segment registers", as such, to detect this error,
but then the OP's claim that examining a pointer's value "just loads
it into a register" is not universally correct, so segment registers
are only one example of a possible source of trouble.)

Alas, nobody builds machines designed for debugging anymore.

IBM does. Hurrah for capability architectures; it's a pity we don't
have more of them.

 

[Michael Wojcik]

At the assembly
level a pointer is just a register value which can be manupulated just
as any other value, as long as no attempts are made to dereference it.

The C standard does not require this, and there are conforming
implementations for which it is not true.

Certainly, there are many conforming implementations where it *is*
true, and on those it is generally safe to examine an invalid
pointer value as long as it is not dereferenced; but such code is
not portable beyond those implementations. Whether that is an issue
is up to the authors and users of such code.

--
Michael Wojcik (e-mail address removed)

An intense imaginative activity accompanied by a psychological and moral
passivity is bound eventually to result in a curbing of the growth to
maturity and in consequent artistic repetitiveness and stultification.
-- D. S. Savage

 

[Chris Torek]

This may not help (although it happens to work on most machines today).
See <http://web.torek.net/c/numbers2.html>.

Oops, wrong URL -- should be http://web.torek.net/torek/c/numbers2.html

 

[Stephen Sprunk]

[snip]

Hm, thats the strange thing here - the code is part of a widely used
open source cross-patform library, which supports a huge diversity of
compilers and platforms. And this code works on all... So, as you say
it seems that the problem with reading a pointer to free-d memory
should be theoretical. But it is really interesting that nobody has
complained so far about this.

What open source tool kit did the code come from?

iaxclient, in libiax2 - iax.c
This is the way sessions are handled.

<OT>
libiax2 does not have a very diverse userbase; the majority of people will
be running it on Linux, with a smattering of *BSD, OSX, Win32, and Solaris
users. Those are all relatively forgiving platforms in general, and all
allow you to examine an invalid pointer without trapping in particular. Go
run the code on an AS/400 and you'll see a very different result.
</OT>

Examining an invalid pointer causes UB, period. That means that in some
cases, it will do exactly what you expect (i.e. nothing). However, it
doesn't preclude some implementation from doing something else.

As others have noted, since malloc() is allowed to return duplicate values
if there's been an intervening free(), the code you're working with is
broken even if the behavior _were_ defined. Either you're lucking out that
your code happens to assign those blocks to other data types and thus never
causes a conflict between two instances of the same type, or you're actually
getting false pointer links to the wrong instance of an object and it's
causing subtle bugs you'll never be able to trace.

The only obvious solution to this is to replace bare calls to free() with a
function that walks all of your data structures and NULLs out any pointers
that compare equal before actually free()ing the dead object.

S