Testing if a pointer is valid

J

jacob navia

Le 23/09/11 23:09, James Kuyper a écrit :
Since the object in question was automatic, this would involve a
radically bigger change to the language than simple adding the ability
to call gc_malloc().

Yes, and I clarified in the next message (that you choose to ignore)
that that is UB and lcc-win doesn't support it. Gc applies only to
objects allocated with gc_malloc() not to automatic objects.

But anything is OK so long as you can throw something to lcc-win
 
J

jacob navia

Le 23/09/11 23:13, Ben Bacarisse a écrit :
Whatever the reason, code that needs to be linked
against a collecting allocator is then tied to one and maybe that is
what makes people wary.

Sure, but then, that is true for EVERY library that you use.
If you use OpenGL your code is tied to it. If you use some
network library your code is tied to it.

Etc.

To get rid of the collector you just replace gc_malloc by
malloc and add all the calls to free().

To replace OpenGl you write your own graphic library.

To replace the network library you write your own TCP/IP
implementation.
 
I

Ian Collins

Not "your code" but there are conforming C programs that will fail with
some garbage collected implementations. The details too horridly
contrived to matter (think %p and it's guarantees) so I mention it only
because this is c.l.c.

In reality, the problem is that one does not link against a GC allocator
just for fun.

That's one of the better uses! I had great fun linking it against
system applications and watching the leak log! Mind you, this was nigh
on 20 years ago.
 
J

jacob navia

Le 23/09/11 23:22, James Kuyper a écrit :
The biggest conformance problem with using garbage collection is that
your code will not be portable to any implementation that doesn't
provide it, or which provides it by an incompatible mechanism. There's
nothing wrong with writing unportable code, so long as it is done
deliberately with a a proper understanding of the consequences.

The gc system written by Mr Boehm runs in all workstations, windows
machines, what have you.

It is distributed with some gcc distributions.

But yes, it is a library and if you do not like a dependency
just call free() when appropiate.
 
I

Ian Collins

Such linkage does not render your code non-conforming, it doesn't even
interfere with strict conformance. However, if you don't merely link to
the collector, but actually use it, that would interfere with strict
conformance. That's because the interface to the collector is not part
of any conforming implementation of the C standard library, and the
collector itself cannot be written in strictly conforming C. However,
very few programs are strictly conforming; possibly none that do
anything really useful.

Untrue, the main advantage of using a library is the library simply
interposes malloc and friends.
The biggest conformance problem with using garbage collection is that
your code will not be portable to any implementation that doesn't
provide it, or which provides it by an incompatible mechanism. There's
nothing wrong with writing unportable code, so long as it is done
deliberately with a a proper understanding of the consequences.

The code will be perfectly portable. It will just leak a lot...
 
J

James Kuyper

On 09/24/11 09:22 AM, James Kuyper wrote: ....

Untrue, the main advantage of using a library is the library simply
interposes malloc and friends.

Automatic garbage collection of malloc() allocations cannot be done by a
fully conforming implementation of C. See the requirements for
scanf("%p"), and consider the implications of having the only copy of a
pointer's value stored completely outside of any memory that the garbage
collector is capable of scanning. Of course, you don't need scanf("%p")
to achieve that; fwrite()/fread() is sufficient, unless the garbage
collector also scans files.
The code will be perfectly portable. It will just leak a lot...

The problem I mentioned above can be avoided by restricting garbage
collection to memory allocated through non-standard methods (for
instance, a call to gc_malloc()) - the standard makes no guarantees
about the lifetime of memory allocated in that fashion. It's those
non-standard methods, rather than the garbage collection itself, which
render the code non-portable.
 
I

Ian Collins

Automatic garbage collection of malloc() allocations cannot be done by a
fully conforming implementation of C. See the requirements for
scanf("%p"), and consider the implications of having the only copy of a
pointer's value stored completely outside of any memory that the garbage
collector is capable of scanning. Of course, you don't need scanf("%p")
to achieve that; fwrite()/fread() is sufficient, unless the garbage
collector also scans files.

I'm sure everyone uses that ridiculous corner case.
 
B

Ben Bacarisse

jacob navia said:
Le 23/09/11 23:13, Ben Bacarisse a écrit :

Sure, but then, that is true for EVERY library that you use.
If you use OpenGL your code is tied to it. If you use some
network library your code is tied to it.

Of course, but not every library has an standard C alternative.
To get rid of the collector you just replace gc_malloc by
malloc and add all the calls to free().
Simples!

To replace OpenGl you write your own graphic library.

To replace the network library you write your own TCP/IP
implementation.

The fact that replacing them is so hard makes the argument for using
them overwhelming. That's not always true for a collecting allocator.
 
J

James Kuyper

On 09/24/11 10:06 AM, James Kuyper wrote: ....

I'm sure everyone uses that ridiculous corner case.

I'm sure that almost no-one relies upon writing pointer values to a file
and reading them back later while they're still valid. Nonetheless, an
inability to handle "that ridiculous corner case" correctly does render
automatic garbage collection of malloc()'d memory non-conforming.
 
J

jacob navia

Le 24/09/11 14:02, James Kuyper a écrit :
I'm sure that almost no-one relies upon writing pointer values to a file
and reading them back later while they're still valid. Nonetheless, an
inability to handle "that ridiculous corner case" correctly does render
automatic garbage collection of malloc()'d memory non-conforming.

You have a misunderstanding: the collector only cares about pointers
allocated with gc_malloc(). Pointers allocated with plain malloc()
must be freed() as before, they are not concerned with the gc.

Implementation specific limitations to pointers are common place.
For instance doubles must be aligned in an eight byte boundary in
many machines. Nothing about those limitations are specified in the
standard.

Are then all uses of doubles in those machines "not C" (tm) ??

Similarly, pointers allocated with gc_malloc should be kept visible
to the collector, not Xored, whatever.
 
R

Rui Maciel

Keith said:
Certainly if I write something simple like:

void func(void) {
int local = 42;
return &local;
}

then I've made a beginner's mistake, and it's probably my own fault,
even though the compiler isn't required to diagnose it.

But I think you're ignoring the fact that many errors are *subtle*. Toy
examples, don't quite illustrate the point, but more realistic examples
are difficult to read.

Bugs are inevitable even in code written by highly skilled and
experienced programmers. Some languages do a better job than others of
helping to diagnose those errors. (C isn't one of those languages, but
there are valid reasons for the tradeoffs C makes.)

Yes, that is true; many errors are very subtle, and they are extremely hard
to pinpoint, even when employing specialized tools to weed them out.

Nonetheless, there is nothing subtle about assigning a global pointer to the
reference of a local object, and then expecting to access it beyond the
object's lifetime. A programmer must expressly write code for this effect,
and a programmer only does this if he doesn't know what he is doing.

Regarding how some languages do a better job than others of helping to
diagnose those errors, this is debatable. In this case, the C standard
leaves this behavior undefined to open the door for compiler developers to
implement any memory management policy they see fit. For this purpose, it
wouldn't be reasonable to mandate strict rules that interpreted this form of
memory access as being an error, as it would negate the motivation behind
leaving this behavior undefined. Yet, just because the standard doesn't
enforce any sanity check in this circumstance, this doesn't mean compiler
developers are barred from implementing them. In fact, some compilers do
test for this sort of error. For example, consider the following code:

<code>
int* test(void)
{
int test = 1;
return &test;
}

int main(void)
{
int *foo = test();

return 0;
}
</code>


When compiling this code with gcc, and without passing any flags, the
following warning is shown:

<message>
rui@Kubuntu:tmp$ gcc main.c
main.c: In function ‘test’:
main.c:5:2: warning: function returns address of local variable
</message>

When compiling this code with clang, the following warning is shown:
<message>
rui@Kubuntu:tmp$ clang main.c
main.c:5:10: warning: address of stack memory associated with local variable
'test' returned
return &test;
~^~~~
1 warning generated.
</message>


This diagnostic isn't mandated by the C standard, but at least two compilers
have implemented it. This doesn't mean that the C programming language
suddenly became safer with these platform-specific diagnostics, or that it
was less safe in the past. The language is still the same, the code is
still broken and the sole cause is still the programmer's incompetence.

By explicitly defining the limits where a specific behaviour must be
expected then the C standard clearly defines what represents valid C
code, which every competent C programmer is expected to understand.
[...]

Understanding the limits is one thing. Consistently avoiding them while
writing or maintaining complex software is quite another.

Some limits are pretty obvious and pretty hard to miss, unless a programmer
is incompetent. This example would fit into this category.

It's also important to mention that this doesn't mean that C is free from
any safety issue. Yet, some of these accusations are baseless and absurd,
and only contribute to grow a myth about a problem which is far greater, far
more widespread and far more serious than what it really is, and this while
absolving any incompetence which caused it, no matter how profound it might
have been. This safety myth has grown so much that nowadays, by the
comments some people make, it almost sounds like even the smallest hello
world program written in C represents a potential security breach which no
other programming language is capable of causing. And this doesn't benefit
anyone.


Rui Maciel
 
R

Rui Maciel

Keith said:
What about mistakes made by *competent* programmers?

I don't believe that a competent programmer believes it is reasonable to try
to access a local object beyond its lifetime, something which at least some
compilers warn about. As this was the example which was presented to
demonstrate that the C programming language is unsafe then, at least in this
case, this misconception can be put to rest.


Rui Maciel
 
R

Rui Maciel

Keith said:
I'd say that a language that doesn't require a diagnostic for a
particular error is less "safe" than a language that does. It's true
that a particular compiler might warn about it, but that's about the
safety of the compiler, not of the language.

The language is still the same, no matter what tricks compiler writers might
have implemented in a specific compiler. If a specific issue, which is
caused by a programmer expressly writing broken code, has safety
implications and if some compiler implementations are able to diagnose that
error and warn the programmer about it then it isn't reasonable to accuse
the language of being unsafe. Doing this would be shifting the blame from
the programmer to the tools which he employs, and this does nothing to solve
the real source of this problem.


Rui Maciel
 
R

Rui Maciel

Phil said:
And are such pointers "valid" anyway?
If the local variable's out of scope, I would assert that
a pointer to it was no longer valid. Dereferencability has
never been a sufficient condition for being valid, IMHO.

There is a corner case where this pointer could still be considered valid,
but this would need the compiler developers to have implemented some form of
garbage collection (which is possible and, I believe, the main reason to
leave this behaviour undefined) and the programmers should be well aware
that they would be taking advantage of a non-portable feature.

Other than this, such a pointer couldn't possibly be considered valid, and a
compiler could (or should) throw a warning due to this.


Rui Maciel
 
B

Ben Bacarisse

jacob navia said:
Le 24/09/11 14:02, James Kuyper a écrit :

You have a misunderstanding: the collector only cares about pointers
allocated with gc_malloc(). Pointers allocated with plain malloc()
must be freed() as before, they are not concerned with the gc.

I think you have a misunderstanding. This sub-thread is about an
implementation that collects malloc'd memory. Look at the very first
quote. That was prompted by Ian Collins's (now snipped)

| the main advantage of using a library is the library simply
| interposes malloc and friends.

It's been about whether an implementation that collects malloc'd memory
is conforming or not -- and it can't be because of several corner cases
-- for a few messages now. I don't think anyone has suggested that
using gc_malloc (in user code) makes an implementation non-conforming.
Implementation specific limitations to pointers are common place.
For instance doubles must be aligned in an eight byte boundary in
many machines. Nothing about those limitations are specified in the
standard.

That's not true. The standard says a number of things about these
limitations. For example, that corresponding signed and unsigned types
have the same alignment.

Similarly, pointers allocated with gc_malloc should be kept visible
to the collector, not Xored, whatever.

That's not the same as an alignment restriction. It's not something the
standard cares about at all. Code like some_allocator_i_just_wrote()
can do pretty much anything. If a GC'd pointer's pointed-to storage
goes away because the pointer was hidden, that's just a bug caused by
misuse of some API.

But its not quite as simple as this... There is an interaction between
the implementation and a GC system. For example, if the compiler
optimises away a pointer, storage may be erroneously freed. I don't
think this affects conformity, but might mean that a compiler needs to
know more about gc_malloc than it does about some_allocator_i_just_wrote
if everything is to work propoerly.
 
B

Ben Bacarisse

I dunno, it seems to me that there is nothing that forbids an
implementation from using "smart" pointers. Given that, the pointer
can contain a tag issued at allocation time. In turn there can be a
"database" that contains currently valid pointers. When accessing a
pointer, look it up and see whether it is valid.

So it seems to me that in principle you can't rule out a fulling
comforming garbage collector. There might be performance issues, but
that's not my department. Then, too, there might be something in the
standard that forbids it. I wouldn't know about that.

You'd need to say more. How and when is garbage collected?

I am pretty sure than some form of automatic collection can be done
without falling over the corner cases of obscured pointers (for example
freeing storage only pointed to be a pointer that goes out of scope) but
to do it in the general case seems hard. What method do you have in
mind for finding freeable storage?
 
K

Kenny McCormack

Richard Harter said:
Beyond that, I opine that you do not understand what it means to say
something is safe. Just because a language (or any tool for that
matter) is unsafe doesn't mean that it cannot be used safely by a
skilled and careful programmer. It means that the language permits
uncaught errors. Even "competent" programmers make mistakes - no one
says that competent programmers intend to do things like returning out
of scope pointers. C is unsafe in this regard because it can happen.

There is nothing wrong with that - most people understand that C was
never meant to be a safe language.

Rui's problem throughout this thread is that he refuses to accept that
"unsafe" != "bad". I.e., the rest of us understand that no moral judgement
is being made; we are not saying that C is a bad language. But Rui seems
obsessed with defending C's honor against the claim of unsafeness; no such
defense is warranted or necessary.

Another way to look at this is that if you find me & Kiki arguing the same
side of a C question/issue, you can probably be certain that that position
(the one Kiki & I are arguing for) is correct.

--
Windows 95 n. (Win-doze): A 32 bit extension to a 16 bit user interface for
an 8 bit operating system based on a 4 bit architecture from a 2 bit company
that can't stand 1 bit of competition.

Modern day upgrade --> Windows XP Professional x64: Windows is now a 64 bit
tweak of a 32 bit extension to a 16 bit user interface for an 8 bit
operating system based on a 4 bit architecture from a 2 bit company that
can't stand 1 bit of competition.
 
J

James Kuyper

I dunno, it seems to me that there is nothing that forbids an
implementation from using "smart" pointers. Given that, the pointer
can contain a tag issued at allocation time. In turn there can be a
"database" that contains currently valid pointers. When accessing a
pointer, look it up and see whether it is valid.

I presume that the function of the database is to ensure that no memory
pointed at by a valid pointer get collected? If so, the key issue
becomes - when does an entry get deleted from that data base? If you
wait until after free() has been called, it's a fully conforming
implementation of C; but it doesn't really qualify as a garbage
collector, in any conventional sense of the term.

If it gets deleted as soon as there are no copies of a given pointer
value stored anywhere in memory, it causes non-conforming behavior in
precisely the "ridiculous corner case" referred to above.

I can't see anyway to specify the rule for deleting the database entry
that allows such a system to be simultaneously a part of a conforming
implementation of C and a garbage collector. Could you explain the rule
you're thinking of?

In case you're not clear about the relevant issue, here's the
"ridiculous corner case", in all it's glory:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

static int func(void)
{
void *p = malloc(16);
if(p && printf("%p\n", p) > -1)
{
strcpy(p, "It worked!");
return 1;
}
return 0;
}

int main(void)
{
if(func())
{
// Insert sufficient processing here to allow:
// All copies of the value returned by malloc()
// to disappear from scannable memory.
// The garbage collector to notice that fact.

void *q;
if(scanf("%p", &q) == 1)
{
printf("%s\n", (char*)q);
free(q);
return EXIT_SUCCESS;
}
}
return EXIT_FAILURE;
}

I assume that any reasonably conventional garbage collector would
collect the memory shortly after exiting the scope of the variable 'p'.
If I'm mistaken about that, I'd appreciate an explanation.

If translated and executed with a fully conforming implementation of C,
and if the user types back exactly the same character string that was
printed out by the first printf(), the above program is required to
print out "It worked!" and to give a successful exit status (modulo
issues about whether any I/O at all is required to succeed).

If the memory is garbage-collected and put to some other use at any time
prior to the second printf() call, that won't work as it is required to
by the C standard.
 
W

Willem

James Kuyper wrote:
) In case you're not clear about the relevant issue, here's the
) "ridiculous corner case", in all it's glory:

<snip example using %p>

Wouldn't the 'xor-ed pointer linked list' trick be a much more
real-world example ? Even if there is no way to do that in conforming C.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,083
Messages
2,570,591
Members
47,212
Latest member
RobynWiley

Latest Threads

Top