use delete to destroy primitive/object types but memory is not freed

[Keith Thompson]

Hi, I am still here

g is passed by value to free( ) and this is why it continues to have the
same value -pointing to the same memory location- after free( ) is called (I
hope I ve got it right). What may cause the pointer to assume an
indeterminate value?

In C, the safest approach is to think of a pointer as an opaque
abstract entity that supports certain operations defined by the C
standard. If you happen to know something about how machine addresses
work in assembly language, that's great. 99% of the time a C pointer
will be implemented as a machine address, and operations on it will
work as you would expect. That knowledge can help you to understand
how C pointers work, and why they're defined as they are -- but the
language standard allows them to behave in ways that don't necessarily
match the "obvious" behavior of machine addresses.

A pointer object, like any object in C, has a value that can be viewed
as a sequence of unsigned bytes, and that can be displayed, for
example, in hexadecimal. There's no guarantee about what those bytes
are going to look like, but it can be instructive to examine them if
you want a more concrete example of how an implementation might work.

In the following program, I convert a pointer value to unsigned int
and display its value in hexadecimal. This happens to work as
"expected" on the platform I'm using, but don't count on this being
portable. The program assumes that it makes sense to display the
result in 8 hexadecimal digits; this happens to be true on the system
I'm using, but again, it's not guaranteed (there are plenty of systems
with 64-bit pointers, and other sizes are possible.)

#include <stdio.h>
#include <stdlib.h>
/*
* Warning: This program invokes undefined behavior.
*/
int main(void)
{
double *ptr;

ptr = malloc(sizeof *ptr);
*ptr = 9.25;
printf("After malloc(): ptr = 0x%08x, *ptr = %g\n",
(unsigned)ptr, *ptr);

free(ptr);
printf("After free(): ptr = 0x%08x, *ptr = %g\n",
(unsigned)ptr, *ptr);

*ptr = 111.125;
printf("After assignment: ptr = 0x%08x, *ptr = %g\n",
(unsigned)ptr, *ptr);

return 0;
}

When I compile and execute this program, it gives me the following
output:

After malloc(): ptr = 0x000209c0, *ptr = 9.25
After free(): ptr = 0x000209c0, *ptr = 9.25
After assignment: ptr = 0x000209c0, *ptr = 111.125

Now let's examine in a bit more detail what's going on here, both in
terms of the underlying hardware and in terms of the C language
standard.

The call to malloc() allocates memory space for a double object; we
assign the resulting address to the pointer object ptr. (In a
real-world program we'd want to check whether the malloc() call
succeeded, and probably bail out if it didn't.)

We then assign a value to the double object that ptr points to, and we
display (in a non-portable manner) the value of ptr and of what it
points to.

As it happens, pointers on the system I'm using are represented as
machine addresses (actually virtual addresses). malloc() allocated 8
(sizeof(double)) bytes of memory starting at address 0x000209c0.

The C runtime system has reserved that chunk of memory, guaranteeing
that it belongs to this program, that we can read and write it, and
that no other object overlaps it.

Now we call free(). By doing so, we're informing the runtime system
that we no longer need that chunk of memory, that we won't try to use
it again, and that the runtime system is now free to reallocate it for
other purposes. There's no guarantee about what the runtime system
will actually do with that chunk of memory; it could well remain
unused for the remainder of the execution of this program. Or it
could be immediately reallocated for use as temporary storage. By
calling free(), we're telling the runtime system that we don't care
what happens to that chunk of memory; we're done with it. (We're
*not* asking the runtime system to prevent us from trying to access it
again.)

But we still have the pointer value. Since free()'s argument is
passed by value, the variable ptr is referenced, but not modified. It
still contains the same bit pattern, 0x000209c0. (There's been some
debate about whether a sufficiently clever implementation might be
allowed to modify the value of ptr, but we'll assume that it can't.)

So what does 0x000209c0 mean? After the call to malloc(), it was the
address of a chunk of memory that we owned. After the call to free(),
in terms of the underlying hardware, it's still the address of the
same chunk of memory; the only difference is that it's memory that we
no longer own. The runtime system is free to do what it likes with
that chunk of memory, including marking it as read-only. If it
happens to do so, the assignment "*ptr = 111.125;" will likely crash
the program, triggering a segmentation fault or something similar.
But if it happens not to do anything with it immediately, attempts to
access it may still work.

Ok, so the chunk of memory allocated at 0x000209c0 is off-limits after
the call to free(). Attempts to refer to it may happen to work (as
they did when I ran my sample program), but they could just as easily
blow up.

But what about the contents of ptr itself? ptr isn't stored at
0x000209c0, it just points to it. The pointer object is in the local
stack frame of our main program, We should still be able to do
anything we like with the value of ptr, as long as we don't try to
dereference it, right?

Well, yes and no.

On the machine level, on *most* real-world implementations, that's
true. A pointer value is just a bunch of bits, and even though we no
longer own the memory it points to, we still own the pointer itself,
and we can still do things like compare it for equality to another
pointer value.

But the C language standard deliberately doesn't guarantee that.

What happens when we compare two pointer values? On some (probably
most) systems, we're just executing a machine instruction that does a
bit-by-bit comparison of the two values and tells us whether they're
equal. On others, though, there may be special machine instructions
for operating on address values. The program might load the values
into special-purpose address registers before comparing them, and the
very act of loading values into these registers might check whether
the pointers are valid, and trigger a trap if they aren't.

Here's another sample program:

#include <stdio.h>
#include <stdlib.h>
/*
* Warning: This program invokes undefined behavior.
*/
int main(void)
{
double *ptr;
double *copy;

ptr = malloc(sizeof *ptr);
if (ptr != NULL) {
printf("malloc() succeeded\n");
}

copy = ptr;
if (copy == ptr) printf("After malloc, ptr and copy are equal\n");
else printf("After malloc, ptr and copy are unequal (???)\n");

free(ptr);
if (copy == ptr) printf("After free, ptr and copy are still equal\n");
else printf("After free, ptr and copy are unequal (???)\n");

return 0;
}

and here's the output I got when I ran it:

After malloc, ptr and copy are equal
After free, ptr and copy are still equal

On the system I'm using, referring to the value of the pointer after
calling free() doesn't cause any problems; it works just as you might
expect. On a system with the kind of special handling of address
values that I described above, referring to the value of ptr after
calling free() could cause a trap as the value is loaded into a
special address register. Either system could have a conforming C
implementation; either behavior is consistent with the C language
standard, even though the latter might be surprising to many
programmers.

Before the call to free(), the bit pattern 0x000209c0 represents a
valid pointer value. After the call to free(), that same bit pattern
no longer represents a valid pointer value, and any attempt to
reference that value, even without dereferencing it, causes undefined
behavior.

You can't really learn about this kind of thing by running sample
programs and seeing whether they happen to work. Running any number
of sample programs is likely to give you the false impression that the
language makes guarantees that it really doesn't.

I still cant understand this :-(

The sizeof operator is a special case. Unless the operand is a
variable length array, the operand of sizeof is not evaluated. (Why?
Because the standard says it's not evaluated. Why does the standard
say so? Because there's no need to evaluate the operand; the compiler
needs to know the type of the operand, not its value, and the result
can be, and is, determined during compilation.) The expression
(sizeof *g) doesn't evaluate *g, so it doesn't cause any of the
problems that you might encounter if you did try to evaluate *g.

[...]

If there was a standardised code of conduct for use in the comp.lang.c, it
would have definitely describe this as an inappropriate behaviour
(and who am I to judge you,e?)

I don't understand why you have a problem with what I wrote above.
ERT made an absurd statement; I was mildly sarcastic in pointing out
the absurdity.

This is true . I clicked on the wrong newsgroup.

Fair enough; mistakes happen.

However Robert was kind enough to convert my code in C and answer my
question in terms of C in which I am also interested in. I was wondering how
is it possible to free( ), dereference the pointer and still access the data
which I had assigned before. His example answered exactly this. Then Keith
told me that I should not refer to or dereference a pointer; it may cause an
undefined behaviour. The bottom line is that may and I have learned a lot -
which is the whole point of newsgroups- even though I posted my question to
the wrong newsgroup. There is no need for people to get upset and be rude.
This is what Robert wants to communicate.

If that were all that Robert wants to communicate, we wouldn't have a
problem with him. But he repeatedly posts things here that have the
effect of disrupting this newsgroup. He often does so in a way that
*looks* like he's being kind and helpful to novices, but while doing
so he often posts subtle misinformation, some of which we're unable to
refute effectively because it's off-topic and outside our area of
expertise. When his misinformation is within the scope of
comp.lang.c, many of us feel obligated to spend the considerable time
necessary to correct his errors -- time that we'd much rather spend
doing something more constructive. Some of us are unwilling to let
his statements stand without response, because we're afraid that some
people will assume he's correct.

If you're so inclined, you might want to take a look through the
archives at groups.google.com. Look for things that ERT has posted,
and look at how we've responded to him. (There's a lot of it.) Some
of the responses are admittedly overreactions, but on the whole I
think we've done as good a job as can be expected in an anarchic forum
like this. If you can suggest a more effective way of dealing with
his behavior, we'd love to hear about it. But you really need to have
some understanding of ERT's history in comp.lang.c to understand why
we respond to him as we do.

And by the way, welcome to comp.lang.c. I hope you find it useful,
and I'm sorry that your most recent foray here has dumped you into the
middle of this brouhaha.

 

[jimjim]

Hello,

I really appreciate the help you have all given me

I also do appreciate that you all try to ensure that no one receives wrong
information in this newsgroup. If we all had the sense of duty many would
have been different.

However, I strongly believe that my statement stands even if we remove the
word Robert:
However ........ was kind enough to convert my code in C and answer my
question in terms of C in which I am also interested in. I was wondering how
is it possible to free( ), dereference the pointer and still access the data
which I had assigned before. With his example tried to answered it (ie
after free( ) the pointer doesnt "own" the particular the memory space).
Then Keith told me that I should not refer to or dereference a pointer; it
may cause an undefined behaviour. The bottom line is that may and I have
learned a lot - which is the whole point of newsgroups- even though I posted
my question to the wrong newsgroup. There is no need for people to get upset
and be rude. This is what ........ wants to communicate.

There is no reason to be rude in newsgroups or even get upset by some posts.
The whole idea is openess, sharing of ideas, providing help (which is what
.......... tried to do), co-operation, companionship......


Thanks again and I II speek to you soon

 

[jimjim]

P.S: I am just trying to have a constructive dialog. I am not trying to give
lessons to anybody (anyways I assume you are much older than me )

 

[Michael Wojcik]

Your reasoning is fallacious because no implementation of free()
does not, in fact, manipulate the segment table
as you have hypothesized:

The C implmentations for the AS/400 can invalidate pointers in free(),
and indeed do so. Each C pointer is a descriptor that names an address
space and offset, and free() (after validating the descriptor) marks the
address space as invalid. Subsequent attempts to dereference a pointer
to the freed area cause a "trap" (in OS/400 terms, the job is stopped
and a program-error message is sent to the operator's message log, if
no handler for the message is already in force).

While there may not have been any implementation such as Chris describes
for the iAPX family (and good luck proving that negative), there
certainly are equivalent ones for other architectures.

http://www.don-lindsay-archive.org/skeptic/arguments.html#hypothesis

You fail to understand (or deliberately misrepresent) the fallacy of
hypothesis contrary to fact. (Lindsay does a poor job of explaining
it - indeed, he offers almost no explanation whatsoever. I don't
believe rhetoric is well-served by this sort of treatment. It
encourages sophomores.)

Hypothesis contrary to fact is only a fallacy under restrictive
conditions: the hypothesis must be introduced to support an argument
that applies only to past factual matters. If the argument is itself
hypothetical, and touches on future possibilities, then the hypothesis
remains just that - a hypothesis, which (as it pertains to that which
has not yet occurred) is provisional. Or, as in this case, the
example given may be merely illustrative, and not offered as support
for the central claim at all.

There are only two points which are relevant here: the implementation
is allowed to invalidate a pointer passed to free(), and there are
mechanisms by which some implementations could do so.

 

[Keith Thompson]

The C implmentations for the AS/400 can invalidate pointers in free(),
and indeed do so. Each C pointer is a descriptor that names an address
space and offset, and free() (after validating the descriptor) marks the
address space as invalid. Subsequent attempts to dereference a pointer
to the freed area cause a "trap" (in OS/400 terms, the job is stopped
and a program-error message is sent to the operator's message log, if
no handler for the message is already in force).

[...]

Keep in mind that there are two different senses (in terms of the
effects) in which a pointer can become invalid.

The first is demonstrated by:

int *ptr = malloc(sizeof *ptr);
*ptr = 10;
free(ptr);
if (*ptr == 10) ...

The *ptr in the if statement after the free() dereferences ptr, which
invokes undefined behavior because it refers to memory that we no
longer own. On most real-world systems nothing bad will happen, and
*ptr will probably compare equal to 10, but on the AS/400 (if I
understand your description correctly) it could actually cause a trap.

The second is demonstrated by:

int *ptr = malloc(sizeof *ptr);
*ptr = 10;
free(ptr);
if (ptr == NULL) ...

Here we're not dereferencing ptr, so there's no issue of referring to
memory that we no longer own, but referring to the value of ptr itself
invokes undefined behavior. As I explained upthread, it's even more
unlikely that a real-world system will cause a trap, but the standard
specifically allows it to do so.

Question: does the AS/400's invalidation of pointers passed to free()
cause a trap when the pointer value itself is referenced (as in my
second example), or only when it's dereferenced (as in my first)?

In any case, keep in mind that pointer errors are the root cause of
many of the software bugs that enable viruses, root exploits, and
other security breaches. It's not implausible that, with pressure to
plug these holes and advances in hardware technology, future CPUs will
do much more checking in hardware, with the goal of catching errors as
early as possible. Avoiding undefined behavior that happens to do
what you expect on current systems doesn't just give you brownie
points for following the letter of the language standard; it makes it
more likely that your code will continue to work on the systems that
may be common 5 or 10 years from now.

 

[Keith Thompson]

(anyways I assume you are much older than me )

Oh, that makes me feel so much better about the whole thing. }

 

[Joe Wright]

Oh, that makes me feel so much better about the whole thing. }

Chuck Falconer recently alluded to having an elder but nobody believes
him. We properly revere our elders, jimjim you and the rest of us,
Chuck.

 

[Chris Torek]

[I have ERT killfiled so I only saw this in the form of the second
level quotes.]

<[email protected]> ...

accuses me of "hypothesis contrary to fact", which would only be
the case if I claimed some 80x86-based C systems actually *did* it,
rather than "make it possible".

The C implmentations for the AS/400 can invalidate pointers in free(),
and indeed do so. Each C pointer is a descriptor that names an address
space and offset, and free() (after validating the descriptor) marks the
address space as invalid. Subsequent attempts to dereference a pointer
to the freed area cause a "trap" (in OS/400 terms, the job is stopped
and a program-error message is sent to the operator's message log, if
no handler for the message is already in force).

As described, this is a somewhat weaker form of trapping than that
offered by the iAPX architecture. In this case, consider the code
fragment:

TYPE *p, *q, *r;
...
p = malloc(N * sizeof *p);
q = p;
use(p);
free(p);
r = q; /* allowed to trap */

would not in fact trap on the "r = q" assignment.

Trapping a reference to "*p" or "*q" or "*r" is, of course, also
possible, *does* happen on the AS/400, and *can* be made to happen
on a wide range of current systems using "malloc substitutes"
somewhat like Electric Fence. (I even did my own in-kernel one
for 4.xBSD many years ago, to catch what turned out to be the
use of a "struct proc *" pointer after the process had exited.)

It is, however, not excessively difficult to make the "r = q"
assignment trap on iAPX systems, via the hardware tests that occur
upon loading a segment register.

While there may not have been any implementation such as Chris describes
for the iAPX family (and good luck proving that negative) ...

I suspect none exist. The trick to making it work requires:

a) forcing *all* pointer-value loads to pass a segment number
through one of the segment registers; and

b) cooperation from the operating system (if there *is* an OS)
so that malloc() and free() can manipulate the CPU's
segment table.

Part (a) requires that the compiler emit instructions whose sole
purpose is to catch bugs. These instructions otherwise slow down
correct code. It appears to me that it is commercially impossible
to sell this as a feature: "Our compiler produces 20% slower runtime
code that catches your errors!"

Part (b) is not difficult if you have control over the OS code,
but it then encourages the use of the task-switch instruction[%]
to handle process scheduling inside the OS, and OS-writers have
been avoiding this instruction for decades because it is -- guess
what? -- slow.

[% TSS allows changing the I/O permission maps, segment selector
tables, and various other "CPU state" variables all in one fell
swoop, as it were -- but it costs about as much time as it takes
to change all those state variables. If one forces all processes
to live with a single set of segments and I/O maps, one can switch
between them faster by *not* changing those.

The VAX had similar issues, though 4.1BSD did use the "svpctx"
and "ldpctx" instructions there.]

 

[Michael Wojcik]

Question: does the AS/400's invalidation of pointers passed to free()
cause a trap when the pointer value itself is referenced (as in my
second example), or only when it's dereferenced (as in my first)?

To be honest, I don't remember, and I suspect it may depend on
circumstances. AIUI, use of an invalid pointer (whether it involves
dereferencing or simple referencing) is detected and rejected when
the OS/400 MI instruction "matptr" (materialize pointer) is executed.
(MI is the pseudo-assembly language that C and other customer-usable
languages compile to on the '400.)

The compiler generates a matptr for pointer dereferences, but I'm not
sure that it always does so for references.

Note that this also applies to Chris' post regarding pointer aliases.
In the case of:

p = malloc(size);
if (p)
{
q = p;
free(p);
r = q;
}

the '400 should trap that r=q if it does a matptr on q, since the
area named by q is no longer valid. But I don't know whether the
compiler will generate a matptr in this case.

I'll have to write some test programs and see. My AS/400 (which
lives in another state) appears to be down at the moment. (It's
configured to require operator intervention to reboot after a
power failure that outlasts the UPS, unfortunately. Must dig out
the docs and find out how to change that...)