Is it time for secure C ?

[Igmar Palsenberg]

Hello,

I just downloaded MS Visual Studio 2005 Express Beta. When I tried to
compile existing valid project, I get a lot of warnings like 'sprintf'
has been deprecated, 'strcpy' has been deprecated etc. I opened STDIO.H
and figured that one has to define a macro _CRT_SECURE_NO_DEPRECATE
to stop these warnings.

There is more when writing insecure programs. integer overflows,
unsignedness issues, corrupted heaps, etc.

The use of strcpy() itself isn't always unsafe, it's use is unsafe when
certain conditions are met. It's a matter of just thinking before you
type the lines of C. The path that MS chooses doesn't give 100% secure
programs, it just gives a false sense that programs are more secure.

Thinks like stack protection (stackguard / propolice in gcc), or the
recent guard is MSVC are good things, but don't match up with the best
think : Simply writing good code.

I'm using valgrind at the moment, and that simply learns you to write
better code : I'm seeing a decrease in errors that leads to invalid
memory reads / writes in my own code, and you learn what actually
happens, and where in the code it happens.

That beats all runtime stuff that prevents it from happening, since it
doesn't tell you why it is happening. Tools like valgrind tell that, and
help make better programmers.

Just my 2 cents.


Igmar

 

[Chris Torek]

This need not be the case.

There's a guy by the name of Donald Knuth who's written at least one
major software package that I know of and use regularly, and I think a
few others as well. I don't know of any bugs in any of them. ...

Programs like TeX and Metafont, for instance? DEK set up something
that few others would dare to do: anyone who found a bug (after the
initial release) got paid, with the amount doubling each time.

I happened to find the 9th or 10th bug in TeX, and have a now-expired
check (which I never cashed, of course ) for either $5.12 or
$10.24 -- I forget which, and would have to find it; I always meant
to get around to framing it and hanging it on the wall, but have
not, yet.

Any bugs that do exist are definitely NOT ones that a mere mortal
would come across in normal use.

(I found mine by reading the code -- one could command TeX to copy
an \hbox, and one particular sub-code was not handled in this "copy
list of nodes" function. The default case called TeX's internal
"panic" function.)

It does not take much experience as a programmer to realize that
a "double the reward for each successive bug" system makes releasing
buggy software a *very* expensive proposition. Bug #24 would be
worth $167 772.16, and bug #40 would be worth $10 995 116 277.76.
Imagine how much Microsoft would be paying out.

 

[P.J. Plauger]

There is more when writing insecure programs. integer overflows,
unsignedness issues, corrupted heaps, etc.

True. Other parts of Secure C address some of these issues, but
nothing takes the place of good design habits, careful code review,
and thorough testing.

The use of strcpy() itself isn't always unsafe, it's use is unsafe when
certain conditions are met. It's a matter of just thinking before you
type the lines of C. The path that MS chooses doesn't give 100% secure
programs, it just gives a false sense that programs are more secure.

If people were told, "Just use Secure C and you won't have to worry
your pretty head any further", I'd agree with you. Rather, the new
library is presented as part of a complete breakfast. It forces the
programmer to think more about the sizes of things, and it makes
more visible the decisions that programmer reached. "Just thinking"
is all you need to write safe code in assembly language, but you
need to do a helluva lot more thinking with error-prone tools than
with safer tools. Secure C is (just) a tool to help programmers
think more effectively about writing safe code.

Thinks like stack protection (stackguard / propolice in gcc), or the
recent guard is MSVC are good things, but don't match up with the best
think : Simply writing good code.

Once again, you're setting up a straw man. *Nobody* is proposing
that Secure C or anything else (with the possible exception of
Java licensed directly from Sun) will eliminate the need to code
carefully and will automatically eliminate all bugs. But tools
such as stack protection do indeed match up well with other
good disciplines. They just don't replace them.

I'm using valgrind at the moment, and that simply learns you to write
better code : I'm seeing a decrease in errors that leads to invalid
memory reads / writes in my own code, and you learn what actually
happens, and where in the code it happens.

That beats all runtime stuff that prevents it from happening, since it
doesn't tell you why it is happening. Tools like valgrind tell that, and
help make better programmers.

Yet another straw man. *Nobody* is proposing that runtime checks should
take the place of all other code-development tools. I've been advocating
good programming style for over 40 years now, and I've always emphasized
early checking over late bug detection; but I also happily embrace any
tools that mitigate the damage caused by bugs even as late as runtime.
The approaches are not in conflict.

Just my 2 cents.

Mine too.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[Dan Pop]

Anyway, my original point was simply that there's nothing *in the
language* that forbids safe pointers, it's just no one has bothered to
implement them.

s/bothered/successfully managed/

Big difference.

Dan

 

[Ben Pfaff]

It does not take much experience as a programmer to realize that
a "double the reward for each successive bug" system makes releasing
buggy software a *very* expensive proposition. Bug #24 would be
worth $167 772.16, and bug #40 would be worth $10 995 116 277.76.
Imagine how much Microsoft would be paying out.

However, DEK's reward system doesn't work like that as far as I
know. Instead, he doubled the reward at each release of TeX.
Typically a release fixed more than one bug.

 

[Minti]

How do *you* know? (And BTW, your lines are too long. Stick to
75 characters wide for Usenet, please.)

What do you bet Ada or Pascal or maybe COBOL?

Lucky you. WinXP Professional crashes about twice a week at work,
and more, lately. At home it's less of a problem, but it still crashes
every so often. ...OTOH and besides, crashing is a heck of a lot better
than the alternative, from a *security* point of view!

I would say that it could be possibly because of some faulty driver.

The Unix/Linux/network/mainframe/embedded/portable cave, yeah,
probably. Microsoft certainly doesn't mean anything in the world
of C standards, and it doesn't mean a whole lot more even in the
world of C applications programming. Just because they hire a lot
of programmers doesn't make them relevant.

Indeed it does not, it is just one of the members of the standards
committee or is it not.

You seriously believe that, do you? Check Google News recently;
even their bugfixes apparently need bugfixes!

So your Linux machine crashes once a century. Right? Well wait you use
Linux _once_ in a century. Every OS has it's own bugs, just because we
get to _learn_ more about MS bugs does not mean Linux does not have
any . It just has less of them, probably. WinXP crashes more than
Linux because of 3 reasons not 1

a) More people _care_ to crash Windows.
b) More people use it. Crashes are generally propotional to the
ammount of time we use an OS also most people who use computers don't
know the basics of using a computer.
c) It has more bugs. Probably.

 

[tweak]

Hello,

I just downloaded MS Visual Studio 2005 Express Beta. When I tried to
compile existing valid project, I get a lot of warnings like 'sprintf'
has been deprecated, 'strcpy' has been deprecated etc. I opened STDIO.H
and figured that one has to define a macro _CRT_SECURE_NO_DEPRECATE
to stop these warnings.

I started to search internet and found few links, and the following proposal

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1031.pdf

After looking into Whidbey Beta header files I started liking this. This is
something I have been using already for static and local buffers using
macro with strncpy() and vsnprintf(), only this is better.

Although this feature should be invoked by defining _USE_SECURE_LIBS
and not be used by default, that's easy to fix in CRTDEFS.H.

Anyway, I am just wondering if anybody knows about the status of this
proposal. And also would like to read some opinions.

Roman

A secure C standard (perhaps with some input from the openBSD
developers) would be nice since undefined behavior usually equals
exploitable.

Brian

 

[Malcolm]

Anyway, my original point was simply that there's nothing *in the
language* that forbids safe pointers, it's just no one has bothered to
implement them.

The language makes it very hard to implement them.

For instance consider this

char *p1 = malloc();
char *p2 = p1;

free(p1);
/* code must trap here */
foo(p2);

How are you going to prevent the programmer from doing this? If you trap,
that's almost as bad as an illegal memory write, and in any case you've got
to implement some scheme for marking p2 as invalid after the free().

You can of course eliminate a lot of overruns by implementing fat pointers
(each pointer contains the address itself and the upper and lower bound of
the object it points to). In some cases this might be worthwhile. However
the overhead may not be insignificant, and that leads you to the question
"why use C?" when we have languages that are designed to prevent direct
access to memory.

 

[Roman Ziak]

The language makes it very hard to implement them.

For instance consider this

char *p1 = malloc();
char *p2 = p1;

free(p1);
/* code must trap here */
foo(p2);

This is very easy to catch and I believe several compilers including VC
do implement that at least in _Debug_ configuration:

1) run-time keeps the list of free blocks and under certain circumstances
may check if the pointer being freed is on that list.

2) every valid memory block has a header which contains a _signature_.
There is still a slight chance of random data forming the signature, but
that is of much lesser chance. The method has almost no performance
hit, so it stay in effect even for _Release_ configuration.

Roman

 

[Malcolm]

This is very easy to catch and I believe several compilers including VC
do implement that at least in _Debug_ configuration:

1) run-time keeps the list of free blocks and under certain circumstances
may check if the pointer being freed is on that list.

I don't think you've looked at the example carefully enough. The call to
free() is valid. However p2 contains a copy. Therefore the compiler has to
keep a list of every pointer which is assigned a value derived from p1, and
tag it as "invalidated".

2) every valid memory block has a header which contains a _signature_.
There is still a slight chance of random data forming the signature, but
that is of much lesser chance. The method has almost no performance
hit, so it stay in effect even for _Release_ configuration.

You can use this to check if a pointer points to the head of a block
allocated with malloc(). Unfortunately C allows you to increment the
pointer, also it allows pointers to allocated blocks and pointers to stack
items to be passed arbitrarily to functions. Worst of all it allows a
pointer to ploint to one past the end of a valid object, so the pointer is
legal but cannot be dereferenced.
So what this means is that foo() cannot have startup code that says "check
if pointer points to allocated block", because the parameter might easily be
another type of pointer. So the check has to be done in the calling
function.

 

[Nick Landsberg]

I don't think you've looked at the example carefully enough. The call to
free() is valid. However p2 contains a copy. Therefore the compiler has to
keep a list of every pointer which is assigned a value derived from p1, and
tag it as "invalidated".


You can use this to check if a pointer points to the head of a block
allocated with malloc(). Unfortunately C allows you to increment the
pointer, also it allows pointers to allocated blocks and pointers to stack
items to be passed arbitrarily to functions. Worst of all it allows a
pointer to ploint to one past the end of a valid object, so the pointer is
legal but cannot be dereferenced.
So what this means is that foo() cannot have startup code that says "check
if pointer points to allocated block", because the parameter might easily be
another type of pointer. So the check has to be done in the calling
function.

Another, more complex case, might be:

char *p1 = malloc(SIZE);
char *p2 = p1;

free(p1);
char *p3 = malloc(SIZE);

AFAIK (correct me if I am wrong, please), there is
nothing which prevents an implementation of malloc()
from returning the same pointer value for
both p1 and p3 in the above example. It either may
or may not as I understand it.

Thus further code which does:

bar(p3);

followed by:

foo(p2);

Might look perfectly valid to some "malloc checker" code
inserted by the compiler or by anything else for
that matter, since both p2 and p3 are both pointing
at the same hunk of (valid) memory in this case.

Is that correct?

NPL

 

[Malcolm]

AFAIK (correct me if I am wrong, please), there is
nothing which prevents an implementation of malloc()
from returning the same pointer value for
both p1 and p3 in the above example. It either may
or may not as I understand it.

On many platforms this is probably what will happen, since the block you
have just freed is at the top of the "free list", which was why it was
allocated by the first malloc().
However we're discussing implementing "secure C" by adding bounds
information to pointers and the like. One thing you would naturally do is
rewrite malloc() to ensure that this problem can't happen, though exactly
how you would achieve that is anyone's guess.

 

[Richard Bos]

I don't think you've looked at the example carefully enough. The call to
free() is valid. However p2 contains a copy. Therefore the compiler has to
keep a list of every pointer which is assigned a value derived from p1, and
tag it as "invalidated".

You're missing the fact that a pointer need not be a linear, integral
address. There is nothing to stop a debugging implementation from
implementing pointers as a (block_id, index_in_block) couple, and a
memory block as a (block_id, size, memory) triple. If you then assign
block_ids in order, so that you won't re-use an id unless you've had,
oh, about 16 million malloc() calls in the interim, foo(p2) will try to
pass a pointer using a block_id which no longer exist, and cause a trap.
You can even ensure that pointer++ doesn't increase pointer.index beyond
block.size+1.

Sure, this will create some overhead. But what else is new? Checking for
overflow on all pointer accesses _does_ cause overhead, no matter how
you implement it. That doesn't mean that it's hard.

Richard

 

[Dan Pop]

A secure C standard (perhaps with some input from the openBSD
developers) would be nice since undefined behavior usually equals
exploitable.

Including <unistd.h> in a C program invokes undefined behaviour.
How much C programming on Unix can you do without invoking undefined
behaviour?

Dan

 

[Dan Pop]

On many platforms this is probably what will happen, since the block you
have just freed is at the top of the "free list", which was why it was
allocated by the first malloc().
However we're discussing implementing "secure C" by adding bounds
information to pointers and the like. One thing you would naturally do is
rewrite malloc() to ensure that this problem can't happen, though exactly
how you would achieve that is anyone's guess.

malloc is the "trivial" case. How about:

int *f1(void)
{
int i;
return &i;
}

void f2(int *p)
{
int j = 10;
*p = j;
}

int main()
{
f2(f1());
return 0;
}

In f2, p may very well point to a valid memory address, yet the code is
broken.

Dan

 

[Dan Pop]

So your Linux machine crashes once a century. Right? Well wait you use
Linux _once_ in a century. Every OS has it's own bugs, just because we
get to _learn_ more about MS bugs does not mean Linux does not have
any . It just has less of them, probably. WinXP crashes more than
Linux because of 3 reasons not 1

a) More people _care_ to crash Windows.
b) More people use it. Crashes are generally propotional to the
ammount of time we use an OS also most people who use computers don't
know the basics of using a computer.
c) It has more bugs. Probably.

d) It is less secure by design. More important than everything else.

Windows was designed no security concerns whatsoever. By the time they
wanted to add security, they realised that backward compatibility
requirements were limiting their options. Things got slowly better
but Windows is still lagging behind systems designed with security in
mind from the very beginning.

Dan

 

[Guillaume]

Windows was designed no security concerns whatsoever. By the time they

wanted to add security, they realised that backward compatibility
requirements were limiting their options. Things got slowly better
but Windows is still lagging behind systems designed with security in
mind from the very beginning.

This is not true of the NT line. It was designed with security in mind,
right from the start. Windows 2000/XP/2003 all come from the NT source
base, so it's basically the same thing. The security model was probably
not the best around, but it's there.

Of course Microsoft always favored functionality over security, and
that's probably the key matter. BUT your claim was simply not true.

One major difference is that Windows NT was not a real multi-user
OS like the Un*x have been for decades. And Windows 2003 still
compares pale in comparison with a Un*x-like in terms of multi-user
support and security.

Ultimately, it's the end-user who decides what is best. And I'd care
to say that "professional" users are more significant in terms of
quality judgement than the basic consumer.

One very good example is Google. It has been tremendously effective.
We can even say that it's a rare example in the IT world. When's
the last time you wanted to use Google and it was unavailable, or
buggy? I haven't seen that. It has never happened. For the record,
Google uses over 100,000 servers all over the world, and they all
run under Linux flavors. Something that obviously cannot be ignored.

 

[Keith Thompson]

One very good example is Google. It has been tremendously effective.
We can even say that it's a rare example in the IT world. When's
the last time you wanted to use Google and it was unavailable, or
buggy? I haven't seen that. It has never happened. For the record,
Google uses over 100,000 servers all over the world, and they all
run under Linux flavors. Something that obviously cannot be ignored.

I was briefly unable to connect to Google a week or two ago. (I think
there was a DNS problem, probably not something Google was responsible
for.)

 

[tweak]

Including <unistd.h> in a C program invokes undefined behaviour.
How much C programming on Unix can you do without invoking undefined
behaviour?

Dan

I never said anything about unix. I only said to take input from those
who have gone out of their way to create a secure OS. Perhaps they
would be good sources as to where all the holes are in C.

Brian

 

[Keith Thompson]

I never said anything about unix. I only said to take input from those
who have gone out of their way to create a secure OS. Perhaps they
would be good sources as to where all the holes are in C.

Dan's point, I think, is that undefined behavior often does not
indicate an exploitable hole or other bug. For example,
"#include <unistd.h>" invokes undefined behavior, but doesn't
generally indicate a bug (though on the DS9K it probably does make
demons fly out your nose).

One point of a secure C standard would be to minimize instances of
*dangerous* undefined behavior, not necessarily to eliminate all
undefined behavior.