Microsoft abandons the C language

K

Keith Thompson

Rui Maciel said:
If the person responsible for punching in the code intentionally disregards
fundamental coding practices that are covered in every programming 101
course, claiming that somehow they only mattered in the 1960s, and then
proceeds to mitigate the problems caused by that level of incompetence by
claiming that restarting the process is an adequate fix, then I believe we
can agree that it easily fits the "too lazy to do our jobs right" category.

Sure, if you radically change the question I'm asking, you can justify
any answer you like.
There may be software bugs whose fix presents a cost/benefit ratio that is
too high to justify fixing them, but that doesn't mean that a programmer
should be free to intentionally implement disastrous coding practices and
criticise those who actually try to avoid making that sort of mistake.

I said absolutely nothing about "intentionally implement[ing]
disastrous coding practices".

Every large project I've worked on has had hundreds of bugs reported
against it. Can you point to one that doesn't?

Unless you're willing to pay the *huge* price to write (nearly)
bug-free code (as NASA did for the space shuttle's on-board
software), there are going to be bugs. It's entirely possible
that some of those bugs are going to result in crashes. If a
system crashes once or twice a year, it doesn't necessarily imply
incompetence on the part of the developers -- nor does it imply that
fixing that particular bug (with the inevitable risk of introducing
*new* bugs) is the best use of finite resources.
 
I

Ian Collins

If you take the time to browse this thread down to its root, you will notice
that this particular branch of the thread started when VLAs were mentioned,
particularly why their use is frowned upon by some programmers.

If you get to that part, you will notice that "Don't make my brown eyes
China Blue" criticised and poke fun at those who took care of making sure
that the call stack wasn't overflowed, to which he insinuated that it was an
issue which only mattered in 1966, and then proceeded to refer to how he
solved this type of issue by restarting the process.

I believe we can agree that bursting the stack is not a mere resource
contention issue, and that it is a serious programming bug.

Then we should also agree that providing the means to allocate beyond
the the stack without any means for validating the allocation is a
serious specification bug.
 
N

Nick Keighley

Or Windows :)

perhaps we need to distinguish recovery of the system (or daemon or
apllication) and recovery of the current "transaction".

Consider a mobile radio system. rebooting may "recover" the system but
all the on-going calls have gone.
For the purposes of this discussion so far, I would say that if
restarting the application brings it back to a normal operational state,
that constitutes recovery.  And, if such recovery is possible, then by
definition the error was not "unrecoverable".

Regarding the acceptability (or inevitability) of crashes, there is
nothing about restarting the application that proper error-handling code
within the application could not have done itself.

in principle- not always in practice. Th system may have munged its
data structures so badly it may not know what state it is in.
 Since there was no
need to crash in the first place, the crash is not acceptable.

so Linux never panics?
I do recognize that there are "legitimate" reasons for an application to
crash, eg. bad hardware, OS problems, gamma rays, crashing into the
surface of Mars, etc.  Let's not go making excuses for preventable
crashes just because we're too lazy to do our jobs right.

you write perfect software?
 
N

Nick Keighley

There is a difference in the costs of the software developer, and any
consequential losses of any single customer out of thousands. The developer
won't care to spend a definite $1000 to save a possible $1000000 of a
customer.

It might make more sense to just give a refund to that one customer,
although that would be a dangerous precedent, so any liabilities (or lack
of) of the software ought to be carefully spelled out in the licence
agreement.

Of course if many customers are getting irate at such problems, then the
developer should do something about them. But it needs to make financial
sense for the developer not the customer.

remember the Ford Pinto? They calculated it would cost less to pay
compensation for accidents (the fuel tank could split in rear eand
collisions) than fix the problem. People died and then the internal
memo detailing the calculation got to court...
 
J

jacob navia

Le 03/09/12 23:25, Ian Collins a écrit :
Then we should also agree that providing the means to allocate beyond
the the stack without any means for validating the allocation is a
serious specification bug.

This is just NOT TRUE!

int fn(int elements)
{
int tab[elements < 1000 ? elements:1000];
}

There you have limited the number of stack bytes!

You can combine stack allocation with malloc allocation as I
have shown in a thread in comp.std.c:

int fn(size_t datasiz)
{
char databuf[datasiz<1000?datasiz:1];
char *p;
int result;

if (datasiz >= 1000) {
p = malloc(datasiz);
if (p == NULL {/*error handler here */
return EXIT_FAILURE;
}
}
else p = & databuf[0];
// Use p here
if (datasiz>=1000) free(p);
return result;
}

Here we have the best path: In MOST cases we will use a cheap stack
allocation. In the few cases where the data size is exceptionally big
we use the more costly heap allocation. We can handle an unbounded
amount of data.

I have repeated this several times but apparently you do not read those
messages

jacob
 
S

Stephen Sprunk

perhaps we need to distinguish recovery of the system (or daemon or
apllication) and recovery of the current "transaction".

Consider a mobile radio system. rebooting may "recover" the system but
all the on-going calls have gone.

Right. Of course, if you handle the error without crashing, then you
have the opportunity to save all of those ongoing calls.
in principle- not always in practice. Th system may have munged its
data structures so badly it may not know what state it is in.

Said munging is likely the root cause of the crash, and that is what
needs to be fixed first. If the crash persists, then you fix that as
well--on top of a foundation that allows for a graceful recovery.
so Linux never panics?

See below on OS problems. If the crash is solely due to instability in
the layers below, eg. OS or hardware, then you're just an innocent
victim; there is no bug to fix.

But, since you asked, it's been years since I've had a Linux system
panic on me. Heck, it's been nearly a month since I've had a Windows
system panic on me, which is quite an improvement over years past.
you write perfect software?

I am far from perfect. However, when I find a bug in my code, I don't
call it "acceptable" just because I don't feel like fixing it. I don't
have infinite resources, obviously, but the priority of crashes is
second only to security vulnerabilities. Customers aren't impressed by
new/fixed features if the product isn't reliable enough or secure enough
to use in the first place.

S
 
K

Keith Thompson

Stephen Sprunk said:
I am far from perfect. However, when I find a bug in my code, I don't
call it "acceptable" just because I don't feel like fixing it. I don't
have infinite resources, obviously, but the priority of crashes is
second only to security vulnerabilities. Customers aren't impressed by
new/fixed features if the product isn't reliable enough or secure enough
to use in the first place.

For problems with Windows systems, the canonical tech support question
is "Have you tried rebooting?". Customers have in fact accepted this
for decades.
 
K

Kaz Kylheku

Le 03/09/12 23:25, Ian Collins a écrit :
Then we should also agree that providing the means to allocate beyond
the the stack without any means for validating the allocation is a
serious specification bug.

This is just NOT TRUE!

int fn(int elements)
{
int tab[elements < 1000 ? elements:1000];
}

There you have limited the number of stack bytes!

But this is stupid, compared to just:

int tab[1000];

If there is an upper limit, then you should just allocate to that limit
in every stack frame.

This way you have a fighting chance of reproducing the worst-case stack usage
in testing, making sure that the program fits into what the system makes
available.

Like any fatal bug, you want to hit worst-case stack usage in testing, rather
than have your users run into it.
You can combine stack allocation with malloc allocation as I
have shown in a thread in comp.std.c:

int fn(size_t datasiz)
{
char databuf[datasiz<1000?datasiz:1];
char *p;
int result;

if (datasiz >= 1000) {
p = malloc(datasiz);
if (p == NULL {/*error handler here */

What makes you confident that databuf[datasiz < 1000?datasiz:1] will always
succeed? And whatever that reason is, why doesn't it make you confident that
databuf[999] will always succeed? (After all, datasiz could be 999).

If you're confident that databuf[datasiz < 1000?datasiz:1] will succeed,
but not that databuf[999] will succeed, the only reason for that would be
that you know that datasiz is either greater than 999, or sufficiently less
than 999. I.e. you know something about the value datasiz from doing a
complicated analysis of stack space usage which takes into account run-time
values.

I would rather not have to do that. It's easier to be confident that
databuf[999] will work, and once you have that assurance, there is little point
in writing databuf[datasiz < 1000?datasiz:1].
return EXIT_FAILURE;
}
}

The function will work just fine with:

char databuf[999];

Sure, if datasiz is significantly above 999 or below, you save stack
space with the VLA. But by doing so, you have created a function which needs a
suitable argument value in order to hit its worst-case stack usage, and that
must be hit by a test case.

The main advantage of the VLA is simplicity: you just declare it. No OOM
checks, no strategies for different sizes, no nothing. You've lost all that in
this function.

You're trying to "sell" the VLA as a stack-space saving tool, but the argument
is not very powerful.

The one aspect of this that I "buy" is that with a VLA, we can bring stack
frames closer together, eliminating the loose space created by worst-case
allocations. This brings about better locality for better caching and paging
performance. There is the question of whether this is offset by the additional
cost of the steps to allocate the VLA's, which depends on the execution
and data access patterns of the program.

How I might exploit this would be to have a compile macro which can be
configured to switch all fixed arrays to VLA's.

I would make sure that the program passes a battery of test cases using
statically-sized local arrays, never blowing its stack.

Production builds of the program would use dynamic sizes to compact the
stack.

This seems to land into the category of micro-optimization.
Here we have the best path: In MOST cases we will use a cheap stack
allocation.

Stack allocation is cheapest when it is constant. When the function is entered,
all the space it needs is reserved in one step. char databuf[1000] is even
cheaper than char databuf[datasiz ...].

(But then there is that aforementioned loss of access locality. Two things
locatd on either side of databuf could land into the same cache line if databuf
is small enough, and the whole stack fits into fewer pages, etc.)
char databuf[datasiz<1000?datasiz:1];
In the few cases where the data size is exceptionally big
we use the more costly heap allocation. We can handle an unbounded
amount of data.

This is by no means a new trick; you've just introduced the VLA into it.
 
E

Edek Pienkowski

Dnia Tue, 04 Sep 2012 12:16:24 -0500, Stephen Sprunk napisal:
Right. Of course, if you handle the error without crashing, then you
have the opportunity to save all of those ongoing calls.

I have some experience, so maybe I'll contribute a little.

That is not the point - at least for a reasonable HA system. It may
be a point for a literal crash a process at some very unlucky
point, but it is enough to have a message passing architecture. If
a process dies unexpectedly during handling x sessions, it is enough
for the HA framework to redo sending to another, with a few exceptions:
- some errors have a much better outcome when "not done" than when
"done twice", e.g. charging a customer for anything. It is actually
often better not to charge than to charge twice and handle complaints
- given the above, if the process in response to the message already
did some external changes, it might be better to fail by default.

Except for the exceptions, the message can be handled "again" by
another process; the process may die and respawn. Usually it
is possible to have many of them in message-passing architecture.
So crashing may not necessarily result in losing anything.

The main reason however for existence of such underlying systems
is not to justify errors, but only to handle unexpected situations,
such as for example a message from some other system which leads
to a fault, whatever the fault may be - unexpected may mean "not
handled gracefully"; or a memory leak, when a process must be killed
and restarted; or a hangup or infinite loop. With all these
it is better to kill the process than disrupt the whole system
(node). What is not recoverable by the process often is
recoverable by the whole system.
Said munging is likely the root cause of the crash, and that is what
needs to be fixed first. If the crash persists, then you fix that as
well--on top of a foundation that allows for a graceful recovery.

I'm not sure if you mean system-wide data structures or process-local
data structures. Different beasts, the former may have lots of safety
mechanisms (starting with transactionality).
See below on OS problems. If the crash is solely due to instability in
the layers below, eg. OS or hardware, then you're just an innocent
victim; there is no bug to fix.

Except if you bundle hardware with your software.
But, since you asked, it's been years since I've had a Linux system
panic on me. Heck, it's been nearly a month since I've had a Windows
system panic on me, which is quite an improvement over years past.

I had some linuxes panic during last year, but I would not call them
versions stable enough for production.
I am far from perfect. However, when I find a bug in my code, I don't
call it "acceptable" just because I don't feel like fixing it. I don't
have infinite resources, obviously, but the priority of crashes is
second only to security vulnerabilities. Customers aren't impressed by
new/fixed features if the product isn't reliable enough or secure enough
to use in the first place.

The HA mechanisms considerably improve the situation from customer point
of view. The customer may not even know anything went wrong (except
for 0,01% failure rate). And in any given large system there will
be some bugs left, due to basic statistics.
 
A

Angel

For problems with Windows systems, the canonical tech support question
is "Have you tried rebooting?". Customers have in fact accepted this
for decades.

Because M$'s marketing department did such an awesome job convincing not
only the generic public but also many software vendors that there are no
alternatives. Gotta admire them for that, I guess.

As a Linux admin, I often have to convince my customers that no, the
server is not down and no, rebooting will not miraculously cure their
problems. And that they should be caning their programmer/vendor
instead until he fixes the rampant memory- and/or file descriptor leaks
that bring their application down every other day...
(How I wish I was kidding...)
 
J

jacob navia

I (e-mail address removed) answered
Stack allocation is cheapest when it is constant. When the function is entered,
all the space it needs is reserved in one step. char databuf[1000] is even
cheaper than char databuf[datasiz ...].

At most by one or two instructions

All your arguments are the same: trying at all cost to say that VLAs
are bad because they do not fit in the C++ model.

Other similar arguments:

(e-mail address removed)
If there is an upper limit, then you should just allocate to that
limit in every stack frame.
This way you have a fighting chance of reproducing the worst-case
stack usage in testing, making sure that the program fits into what
the system makes available.

Of course!

#ifdef DEBUG
#define TABSIZ 1000
#else
#define TABSIZ (datasiz<1000?datasiz:1)
#endif

But obviously that is wrong too because... whatever.

Another phony argument:
(e-mail address removed)
If you're confident that databuf[datasiz < 1000?datasiz:1] will
succeed, but not that databuf[999] will succeed, the only reason for
that would be that you know that datasiz is either greater than 999,
or sufficiently less than 999.

I am of course not sure that databuf[1000]succeeds, just as I am not
sure that
int i;
will succeed you see?

But I showed that it is very well possible to put an arbitrary limit
(if you feel that the stack could be too small)

Under windows default stack size starts at 1MB. THOUSAND times more
than the 1000 bytes I used as limit. Linux has an even greater stack.
In MANY modern systems stack size is no longer a problem unless you
put huge arrays in recursive functions.

This is a well known FACT. But here we are, discussing about 1000
bytes, or why the "stack will explode" etc etc.

And when I point out that stack usage CAN be limited then it is
always "not enough".

Well, if you want c++ you know where to find it!
 
K

Kaz Kylheku

I (e-mail address removed) answered
Stack allocation is cheapest when it is constant. When the function is entered,
all the space it needs is reserved in one step. char databuf[1000] is even
cheaper than char databuf[datasiz ...].

At most by one or two instructions

All your arguments are the same: trying at all cost to say that VLAs
are bad because they do not fit in the C++ model.

VLA's fit well into the C++ model because in C++ they could take storage from
the free store and throw std::bad_alloc exceptions.
Under windows default stack size starts at 1MB. THOUSAND times more
than the 1000 bytes I used as limit. Linux has an even greater stack.
In MANY modern systems stack size is no longer a problem unless you
put huge arrays in recursive functions.

This observation undermines your argument that we should replace small
fixed-size arrays like char databuf[1000] with VLA definitions which tune
their size to make it even smaller.

If we have the space for the full size (which we damn well should, to prepare
for worst case behaviors in the program), the only reason to shrink it would be
for better caching (as I already observed).

Also in my response I made exactly the same comment as you already: that
a debugging version of a program can use fixed arrays, and then a production
build can compact the stack usage with VLAs. I also noted that this could be
faster due to improved locality (fewer pages of stack, better cache line
density). This is really such a marginal use of VLA's. And anyway, you can
achieve such a thing by reserving a chunk of space on the stack and then
dynamically pulling from it. E.g. GNU "obstacks".

I do not make phony arguments, thank you very much.
 
I

Ian Collins

Le 03/09/12 23:25, Ian Collins a écrit :
Then we should also agree that providing the means to allocate beyond
the the stack without any means for validating the allocation is a
serious specification bug.

This is just NOT TRUE!

int fn(int elements)
{
int tab[elements< 1000 ? elements:1000];
}

There you have limited the number of stack bytes!

Limited, but you still can't tell if table[999] is within the current
stack. Note I specifically stated "without any means for validating the
allocation".
You can combine stack allocation with malloc allocation as I
have shown in a thread in comp.std.c:

int fn(size_t datasiz)
{
char databuf[datasiz<1000?datasiz:1];
char *p;
int result;

if (datasiz>= 1000) {
p = malloc(datasiz);
if (p == NULL {/*error handler here */
return EXIT_FAILURE;
}
}
else p =& databuf[0];
// Use p here
if (datasiz>=1000) free(p);
return result;
}

Here we have the best path: In MOST cases we will use a cheap stack
allocation. In the few cases where the data size is exceptionally big
we use the more costly heap allocation. We can handle an unbounded
amount of data.

Unless there are only 999 bytes free on the stack. Your example proves
the point: you test the malloc, but assume the VLA.
 
S

Stephen Sprunk

For problems with Windows systems, the canonical tech support question
is "Have you tried rebooting?". Customers have in fact accepted this
for decades.

Microsoft's customers have "accepted" it because they (believe they)
don't have a choice.

As one blogger put it, "Microsoft creates their own gravity"; what
applies to them does _not_ apply to anyone else.

S
 
J

jacob navia

Le 05/09/12 00:13, Ian Collins a écrit :
Your example proves the point: you test the malloc, but assume the VLA.

int fn(void)
{
int i;
}

You do not test for "int i;"

We should ban then all stack allocation?

What are you saying actually? I do not understand what you want.
 
E

Edek Pienkowski

Dnia Tue, 04 Sep 2012 16:46:55 -0700, Keith Thompson napisal:
Edek Pienkowski said:
That is not the point - at least for a reasonable HA system.
[...]

HA means "High Availability", yes?

Yes.

I always forget not to use acronyms outside of context.
 
M

Malcolm McLean

בת×ריך ×™×•× ×¨×‘×™×¢×™, 5 בספטמבר 2012 07:59:24 UTC+1, מ×ת jacob navia:
Le 05/09/12 00:13, Ian Collins a écrit :


int fn(void)

{

int i;

}


You do not test for "int i;"

We should ban then all stack allocation?

What are you saying actually? I do not understand what you want.
What he is saying is that you test whether the call to malloc() has succeeded,
but simply assume that the vla allocation has succeeded.
But it's not a very good point. If you set a cap on the amount that can be
allocated on the stack, you can know that the stack won't be breached,
assuming non-recursive functions. That does raise the issue, however, why
use a vla at all, since you might as well allocate the maximum as a fixed
buffer? Arguably a vla provides better documentation.
 
I

Ian Collins

Le 05/09/12 00:13, Ian Collins a écrit :

int fn(void)
{
int i;
}

You do not test for "int i;"

We should ban then all stack allocation?

What are you saying actually? I do not understand what you want.

VLAs are a vulnerability waiting to happen. Not as bad as gets, but
dodgy none the less. Yes your suggestion provides some degree of
protection, although it is rather too cumbersome for general use.

I'd guess that where operation requires a large buffer, the overhead of
allocating the buffer is probably noise, so dynamic allocation is a
better choice.

I will concede the most uses for VALs are small buffers, but things that
start out small tend to grow...
 
I

Ian Collins

בת×ריך ×™×•× ×¨×‘×™×¢×™, 5 בספטמבר 2012 07:59:24 UTC+1, מ×ת jacob navia:
What he is saying is that you test whether the call to malloc() has succeeded,
but simply assume that the vla allocation has succeeded.
But it's not a very good point. If you set a cap on the amount that can be
allocated on the stack, you can know that the stack won't be breached,
assuming non-recursive functions. That does raise the issue, however, why
use a vla at all, since you might as well allocate the maximum as a fixed
buffer? Arguably a vla provides better documentation.

It may provides better documentation, but it precludes static analysis
of stack usage.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Staff online

Members online

Forum statistics

Threads
474,077
Messages
2,570,569
Members
47,206
Latest member
MalorieSte

Latest Threads

Top