Qry : Behaviour of fgets -- ?

[Chris Dollin]

"Conforming program" is a term of little value, and was defined
in the C standard for political reasons. A "strictly conforming
program", however, does not trigger undefined behavior.

"Conforming programs" on "conforming implementations" can do
all sorts of things beyond what the C standard specifies,
including operations that make monitors catch on fire, etc.

Francis's point was that such consequences actually do occur
sometimes when "undefined behavior" is triggered.

Yes; but my esteemed twin is asking why we think those
consequences can't arise even when the program doesn't
exhibit UB. `i += 1;` may increment `i`, as it should,
/and/ set fire to the monitor.

I'd reject such an implementation on QoI grounds, not
on "not a C implementation" grounds.

 

[Bart van Ingen Schenau]

[some examples of UB that actually happened]

What part of the Standard prevents those things from happening when
a conforming program executes -- better, which stop an implementation
that does these things from being a conformant implementation?

How about clause 5.1.2.3/5, which loosely translated states that, when
observing the side-effects of a program, the user may not be able to
tell the difference between execution on the abstract machine and
execution on the actual machine.

Bart v Ingen Schenau

 

[Rainer Weikusat]

Rainer Weikusat wrote:
[...]

The issue is perfectly clear.

That

i = 3;
i = ++i;

means 'the value of i is now 3' is wrong,

[...]

So, how many bug-reports have you filed with the compiler builders that
their behaviour for the code above is incorrect?

I wasn't talking about code but about the defined meaning of certain
statements. The statement above has no defined meaning. Ergo: Every
claim that its defined meaning would be ... must be wrong. It doesn't
matter if the claim is something technically sensible for certain C
implementation (like the example I gave above), or some absurd piece
of non-sense intended to be entertaining (like the 'nasal daemons'):
They are both wrong.

BTW, how did you manage to miss that:

| This is completely orthogonal to the question what conceivable
| behaviour of C-implementations when processing this piece of non-C
| could be. And C is again very simple in this respect: Whatever the
| behaviour may be, it does not matter.

 

[Rainer Weikusat]

Rainer Weikusat wrote:
[...]

This is completely orthogonal to the question what conceivable
behaviour of C-implementations when processing this piece of non-C
could be. And C is again very simple in this respect: Whatever the
behaviour may be, it does not matter. No reason to discuss it.

There are two reasons to discuss it: to answer a question about
whether a given behavior is allowed (when the behavior is undefined,
the answer is always "yes"), and as a didactic tool for impressing
students with just how big a risk they take everytime they execute
code with undefined behavior. Giving humorous and extreme examples is
helpful for making the lesson more memorable, and is technically
correct,

That something weird and presumably dangerous happens if a source file
containing 'character sequences looking like C' which have no defined
behaviour is processed by a conforming C implementation is not
required by the C-standard. Insofar is a lesson trying to establish a
belief like that using sufficently weird make-ups designed to do so
at best useless (student is bright) and at worst bad (student is
stupid and just repeats what he has been told).

I have encountered people who believed exactly that.

 

[Kenneth Brody]

Chris Dollin wrote: [...]

I don't know that it's /infinitely/ many. The universe may be
finite. Even if it isn't, we don't have access to more than
a finite amount of it.

Certainly there are a /great many/ counterexamples.

So you're saying that there is a /finite/ number of counterexamples?

If the accessible universe is finite, yes.

Are you saying that it is, in theory, possible to enumerate them all?

Yes. Enumerate all possible universe states. The number of counterexamples
is no bigger than this.

Oops: I've an implicit assumption that states are discrete.

I /think/ I could weasel that into the finiteness, but I won't.

What if the universe has an inifinite number of possible states?

Yes, there are a finite number of particles in the universe, and
each of them probably has a finite number of states. Unless,
that is, "state" includes "position and velocity". (Unless you
are to argue that there are two non-identical locations where a
particle can be for which there is no location between them.)

So... How does a clc discussion on fgets() turn into a quantum
physics discussion?

[...]

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Kenneth Brody]

Chris Dollin said:



Where?

What? You don't keep a scratch universe handy?

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Richard Harter]

Very cute, but he/she won't get it.

 

[Keith Thompson]

I've redirected followups to comp.std.c.

Rainer Weikusat wrote: [...]

The issue is perfectly clear.

That

i = 3;
i = ++i;

means 'the value of i is now 3' is wrong,

[...]

So, how many bug-reports have you filed with the compiler builders that
their behaviour for the code above is incorrect?

I wasn't talking about code but about the defined meaning of certain
statements. The statement above has no defined meaning.

No meaning is defined for it by the C standard.

Ergo: Every
claim that its defined meaning would be ... must be wrong.

Every claim that its meaning is defined by the C standard would be
wrong. I've seen no such claim here.

Its meaning could be defined by something other than the C standard.
For example, it could be defined by the documentation of some specific
implementation. A note following the C standard's definition of the
phrase "undefined behavior" says this explicitly (C99 3.4.3p2):

NOTE Possible undefined behavior ranges from ignoring the
situation completely with unpredictable results, to behaving
during translation or program execution in a documented manner
characteristic of the environment (with or without the issuance of
a diagnostic message), to terminating a translation or execution
(with the issuance of a diagnostic message).

Detailed discussions of any behavior defined by things other than the
C standard are off-topic (at least in comp.{std,lang}.c), but examples
are useful as illustrations of what the C standard permits.

It doesn't
matter if the claim is something technically sensible for certain C
implementation (like the example I gave above), or some absurd piece
of non-sense intended to be entertaining (like the 'nasal daemons'):
They are both wrong.

A claim that the behavior is defined would be wrong. A claim that
some specific behavior is permitted is perfectly correct.

BTW, how did you manage to miss that:

| This is completely orthogonal to the question what conceivable
| behaviour of C-implementations when processing this piece of non-C
| could be. And C is again very simple in this respect: Whatever the
| behaviour may be, it does not matter.

What makes you think anyone missed it?

 

[Keith Thompson]

Followups redirected to comp.std.c.

That's a claim which should be easy to prove: Assuming you are not
deliberatetly misunderstanding me, you appear to claim that 'undefined
behavious' is actually behaviour defined by the C-standard. Since 'no
requirements' isn't a definition of any specific behaviour, this
definition or set of definitions must be in some other location, so
please cite where the standard provides a definition of the allowed
semantics for undefined behaviour.

NB: A positive definition is required.

You misunderstand.

The *phrase* "undefined behavior" is defined by the standard.

The *behavior* of a program that exhibits undefined behavior is not
defined by the standard.

I haven't seen anyone here make any claim that's inconsistent with
that.

A program may behave in some particular way whether that behavior has
been *defined* (i.e., documented) or not.

More correctly (as I have written several times now): There is no such
thing as 'acceptable behaviour' for these case (provided you cannot
magically come up with a positive definition of this beahviour)
because no acceptable behaviour is defined.

I believe you're mistaken. No *unacceptable* behavior is defined.
By implication, *any* behavior is acceptable.

Presumably, a real-world
implementation of C will have 'some behaviour' in such situation and
whatever this might be is not relevant for determining the standard
conformance of this implementation.

I agree completely with that.

Assuming that the C-standard is relevant for defining the C language,
it is the only location where 'behaviour' can be defined and you are
basically presenting a false dichotomy: There is no such 'other
location'.

Yes, there certainly can be other locations where behavior might be
defined. An implementation's documentation might specify the behavior
of some programs whose behavior is not defined by the C standard. The
standard says this explicitly in C99 3.4.3p2 (in a non-normative
note).

I strongly suspect that we're in agreement about the nature of
undefined behavior, but we're just expressing it in different ways.

 

[Ben Bacarisse]

The meaning of the program is unimportant in this context, its
behaviour is.

Just to clarify. The meaning is the behaviour, but with the all
undetectable variations in behaviour collapsed down into one. The
subtle shift in emphasis when one moves from operational semantics
(describing behaviour) to denotational semantics (ascribing meaning to
program texts) is that one can avoid all the "how" questions.

In C (keep this on topic) the standard describes, in operational
terms, what happens as a program executes. But then, so as not to
constrain implementations has an "as if" rule to allow other behaviour
provided that the result is essentially the same -- i.e. (in
denotation terms) if the meaning is the same.

 

[Ben Bacarisse]

The same argument means the following are UB:

char buf[] = "hello";
strncpy(buf, "j", 5);
buf[1] = 'e';
puts(buf);

Why do you think the above code should invoke UB ?
It should just output j e and a new-line on stdout

Yes of course! Duh!

I was looking for another function that took a "no more than n"
parameter and which does not specify (explicitly) that it does not
write any more location than the minimum required (like fgets in the
discussion). I stupidly switched from strncat to strncpy because I
thought I could make the string cuter! How daft is that? What I
should have given is something like:

char buf[4] = "\0\0c";
strncat(buf, "a", 3);
buf[1] = 'b';
puts(buf);

We all know what should happen, but I can't see how the wording rules
out the same memset(buf, 0, 3) suggested for fgets. Similarly, the
second example should have been:

char buf[2] = "";
strncat(buf, "x", 20);

Given the operational description, only two bytes are written, but the
same goes for fgets -- the standard only states what is read and
written but it does not preclude other modifications.

In my search for examples, I saw that the text for mbstowcs includes
the assurance: "No more than n elements will be modified in the array
pointed to by pwcs" so my hope that I might have missed some blanket
"no function writes to any object unless explicitly stated" seems
unfounded.

Sorry to have messed up this already messy thread with a duff example.

 

[Ben Bacarisse]

Ben Bacarisse wrote:

Because I have a liking for denotational semantics, given a C program
X which has some UB elements in it, I think E[X] = _|_ [1].

UB isn't bottom -- that's a different sense of undefined.

Not entirely different. I know it is somewhat different which is why
I only said that this is how I *think* about UB. The standard says
what UB means (not what programs having UB mean, of course, just what
the term means) and I am not suggesting an alternative. I am saying
how I think of it and why I think that is scary enough.

(To see why, consider our old friend `i = i++;`. If UB was bottom, then
this expression would compute bottom; it would not terminate; any
implementation that gave any value to `i` and continued would be
non-conformant.

E[X] = _|_ does not (always) mean X does not terminate. In the lambda
calculus, non-termination is the most common way to get _|_, but
there are others (E["tail []"] = _|_ in most semantics, for example).

I'd have thought that in /most/ semantics, `tail []` is an error value
or invokes a continuation.

[I have honoured your "followup" this time, but I think it often
better, when one is saying essentially "you are wrong", to limit
*that* message rather than suggesting a limited "followup". Last time
I wanted to defend myself to all how saw your "you are mistaken"
reply!]

OK, maybe not most. Some. The point is that the semantics of
a language may choose to say nothing about some programs. This can be
done with partial functions or with _|_.

Machines implement bottom as non-termination; I'm almost completely
sure of this, but it has been a long time.

I don't follow this. Bottom is just the least element of a poset
(formally) and the least defined value in the set of "meanings" (to be
rather vague about it). Why would one implement it? (I agree that if
one wanted to, while (1); would be a fine way to do it.)

I'm beginning to believe that nasality, being invisible to the standard,
isn't a true (even if good) capturing of UB!

I think that because the standard say "anything" people are happy to
include things way outside what program might or usually do. It sound
so easy to say "anything" (or we impose no constraints on the
behaviour) but as soon as one start to illustrate that, arguments can
start. "No meaning" is much harder to illustrate!

Consider this question: which part of the standard prohibits `i += 1;`
from causing a demon to fly out of your nose?

None. And of course it should not since if 'i' is an 'int', UB is
possible (on overflow). That is the clc answer, of course! Assuming
you mean 'i += 1;' when i is, say, 0, then I think it follows from a
kind of Occam's razor. An operational description like that in the
standard is never entirely unambiguous, but there is no need to add
possibilities that essentially rob the document of all meaning!

 

[Flash Gordon]

Rainer Weikusat wrote, On 14/09/07 18:08:

Rainer Weikusat wrote:
[...]

This is completely orthogonal to the question what conceivable
behaviour of C-implementations when processing this piece of non-C
could be. And C is again very simple in this respect: Whatever the
behaviour may be, it does not matter. No reason to discuss it.

There are two reasons to discuss it: to answer a question about
whether a given behavior is allowed (when the behavior is undefined,
the answer is always "yes"), and as a didactic tool for impressing
students with just how big a risk they take everytime they execute
code with undefined behavior. Giving humorous and extreme examples is
helpful for making the lesson more memorable, and is technically
correct,

That something weird and presumably dangerous happens if a source file
containing 'character sequences looking like C' which have no defined
behaviour is processed by a conforming C implementation is not
required by the C-standard.

No one has claimed that it is. Only that such behaviour, and other
behaviour which is just plain weird, is *permitted* by the C standard.

Insofar is a lesson trying to establish a
belief like that using sufficently weird make-ups designed to do so
at best useless (student is bright)

A bright student should understand the point. There is even evidence
that bright students *do* understand the point, namely responses to such
messages from said students indicating they have understood it.

and at worst bad (student is
stupid and just repeats what he has been told).

I have encountered people who believed exactly that.

People who cannot understand the concept of undefined behaviour and the
anything, even things they would never imaging, is possible where there
is undefined behaviour should not be programming in C.

 

[pete]

The meaning of the program is unimportant in this context,
its behaviour
is. IOWs what it does matters. The point of UB is that executing a
program with UB can result in anything. The Abstract machine is
irrelevant because the program is no longer C and so is outside the
requirements of the Abstract Machine.

Are you saying that a C program
that contains undefined behavior, is not a C program?

 

[Charlie Gordon]

The same argument means the following are UB:

char buf[] = "hello";
strncpy(buf, "j", 5);
buf[1] = 'e';
puts(buf);

Why do you think the above code should invoke UB ?
It should just output j e and a new-line on stdout

Yes of course! Duh!

I was looking for another function that took a "no more than n"
parameter and which does not specify (explicitly) that it does not
write any more location than the minimum required (like fgets in the
discussion). I stupidly switched from strncat to strncpy because I
thought I could make the string cuter! How daft is that? What I
should have given is something like:

char buf[4] = "\0\0c";
strncat(buf, "a", 3);
buf[1] = 'b';
puts(buf);

We all know what should happen, but I can't see how the wording rules
out the same memset(buf, 0, 3) suggested for fgets. Similarly, the
second example should have been:

char buf[2] = "";
strncat(buf, "x", 20);

Given the operational description, only two bytes are written, but the
same goes for fgets -- the standard only states what is read and
written but it does not preclude other modifications.

Actually, the wording for strncat is unambiguous about how many characters
can be changed in the destination buffer: .

C99 7.21.3.2p2
The strncat function appends not more than n characters (a null character
and
characters that follow it are not appended) from the array pointed to by s2
to the end of
the string pointed to by s1. The initial character of s2 overwrites the null
character at the
end of s1. A terminating null character is always appended to the
result.264) If copying
takes place between objects that overlap, the behavior is undefined.

264) Thus, the maximum number of characters that can end up in the array
pointed to by s1 is
strlen(s1)+n+1.

The wording is not cristal clear, but s1 is only changed by appending
characters from s2 and the null and following characters are not appended,
and then a (single) terminating null character is always appended.

In my search for examples, I saw that the text for mbstowcs includes
the assurance: "No more than n elements will be modified in the array
pointed to by pwcs" so my hope that I might have missed some blanket
"no function writes to any object unless explicitly stated" seems
unfounded.

Sorry to have messed up this already messy thread with a duff example.

What a great suggestion! Duff's examples would really mess this thread up
beyond repair ;-)

 

[kuyper]

That's a claim which should be easy to prove: Assuming you are not
deliberatetly misunderstanding me, you appear to claim that 'undefined
behavious' is actually behaviour defined by the C-standard.

No. He made no such claim. The phrase "undefined behavior" is defined
by the C standard, but the meaning it defines for that phrase is
precisely that the C standard does NOT define the behavior referred to
by that phrase.

... Since 'no
requirements' isn't a definition of any specific behaviour,

Correct. A definition of a specific behavior is neither given nor
needed nor even desireable in this context. What is defined is a
category of behavior, namely the universal category, which contains
all behaviors.

... this
definition or set of definitions must be in some other location, so

No. "no requirements" means that any specific behavior qualifies.

please cite where the standard provides a definition of the allowed
semantics for undefined behaviour.

"no requirements" => therefore all semantics are allowed.

NB: A positive definition is required.

Tough luck; you may require it, but the standard chose to define it
negatively.

More correctly (as I have written several times now): There is no such

Repeating it doesn't make it so.

thing as 'acceptable behaviour' for these case (provided you cannot
magically come up with a positive definition of this beahviour)
because no acceptable behaviour is defined.

"no requirements" means that all behavior is acceptable. You've
exactly reversed the intended meaning of the phrase "undefined
behavior". Could you recommend alternative wording for that
definition which would give it the same meaning, as you understand it,
that the rest of us consider to be given by the current wording?

... Presumably, a real-world
implementation of C will have 'some behaviour' in such situation and
whatever this might be is not relevant for determining the standard
conformance of this implementation.

True, and the reason why it's not relevant is precisely because that
behavior is acceptable. If no behavior were acceptable, the occurance
of any actual behavior (including doing nothing) would render the
implementation non-conforming.

....

Assuming that the C-standard is relevant for defining the C language,
it is the only location where 'behaviour' can be defined and you are

Incorrect, the C standard very explicitly allows that some code has
behavior which is defined by the implementation, but that's just a
single counterexample. For the more general case, look at section 1 of
the standard, titled "Scope". It specifies what things are within the
scope of the standard, and what things are not. If you compare those
two lists, I think it's clear that a very large portion of the things
that should properly be called the "behavior" of a program lie outside
the scope of the standard. To take a very simple case, an important
aspect of any program's behavior is how fast it executes; the C
standard does not specify anything, anywhere, about how fast a program
should execute; that's outside the scope of the standard.

 

[kuyper]

I wasn't talking about code but about the defined meaning of certain
statements. The statement above has no defined meaning. Ergo: Every
claim that its defined meaning would be ... must be wrong.

It has no meaning that is defined by the C standard. The C standard's
statements about such code implicitly allow for the possibilty that
it's meaning might be defined by something other than the C standard,
such as the particular implementation that is used to translate and
execute the code. It could also have behavior that isn't defined by
anything, but just happens without having been defined. It is those
other things we've been talking about. We haven't been talking about
the code have a meaning defined by the C standard itself.

 

[kuyper]

That something weird and presumably dangerous happens if a source file
containing 'character sequences looking like C' which have no defined
behaviour is processed by a conforming C implementation is not
required by the C-standard. ...

That is most certainly true; one of the most frequent mistakes made
with respect to UB is the idea that it's necessarily wierd and
dangerous.

... Insofar is a lesson trying to establish a
belief like that using sufficently weird make-ups designed to do so
at best useless (student is bright) and at worst bad (student is
stupid and just repeats what he has been told).

The syntax of that last sentence is mixed up, so I'm not quite sure
what you're saying, but I think I've got the gist of it. I assume
you're not a native speaker of English, so I'm not criticizing you,
just pointing out the problem. My Spanish is much worse than your
English, and of the five foreign languages I know, Spanish is the one
I'm most fluent in.

Any good teacher should emphasize that UB can include all of the
following (and many other things as well):
1. Wierd and/or seriously dangerous behavior,
2. Behavior that is exactly what you incorrectly assumed the code was
required to have. This is also dangerous, because it hides a code
defect.
3. Useful behavior that could not be achieved by code with standard-
defined behavior. The only way such code could be useful is if
something other than the C standard defines the behavior, for example
the particular implementation that you are using to translate and
execute the code. This can also be dangerous, but is also perfectly
legitimate as long as it's well-understood that this code is not
intended to be portable. Such code should only be written by experts
who have enough experience to know when it's appropriate to write non-
portable code. Newbies should generally avoid writing such code, if
they can.

 

[Bart van Ingen Schenau]

[fup set to c.l.c]

Rainer Weikusat wrote:
[...]

The issue is perfectly clear.

That

i = 3;
i = ++i;

means 'the value of i is now 3' is wrong,

[...]

So, how many bug-reports have you filed with the compiler builders
that their behaviour for the code above is incorrect?

I wasn't talking about code but about the defined meaning of certain
statements. The statement above has no defined meaning.

So far, I am with you and agree.
The extreme examples of behaviour resulting from UB are also mostly
meant to wean people (mostly newbies) from the notion that all computer
programs have predictable and/or consistent behaviour. This to avoid
that they start to rely on whatever behaviour their computer shows for
UB.

Ergo: Every
claim that its defined meaning would be ... must be wrong.

And I have nobody seen making such a claim.
There is a vast difference between 'The behaviour is defined to be ...'
and 'In some instances you might see behaviour such as ..., among a
myriad of other possibilities'.

Every example that is given for behaviour that results from UB is meant
in the way of the second statement: one of a myriad of possibilities.
And the behaviour can't be wrong, because if there is no defined
behaviour, there is also no definition that forbids certain behaviours.

It doesn't
matter if the claim is something technically sensible for certain C
implementation (like the example I gave above), or some absurd piece
of non-sense intended to be entertaining (like the 'nasal daemons'):
They are both wrong.

They are wrong if they are meant to give a statement of what *will*
happen (as in a definition).
They are not wrong if they only give an example of what is possible,
without excluding any other possibilities.

BTW, how did you manage to miss that:

| This is completely orthogonal to the question what conceivable
| behaviour of C-implementations when processing this piece of non-C
| could be. And C is again very simple in this respect: Whatever the
| behaviour may be, it does not matter.

I did not miss it. I chose not to comment on it, but now I will.

All examples that are given for behaviour that results on encountering
UB are just what you mention in this paragraph: examples of behaviour
that a C implementation could exhibit when processing the incorrect
code.
Most people, who have questions regarding code that invokes UB, do not
have the experience needed to understand the intricacies behind UB.
They are often still in the stage where they believe that if the
compiler can generate an executable, then the code must be correct.
Then we have to get them to accept that there is a difference between
compilable code and correct code and that there are classes of errors
that the compiler can't diagnose and which cause UB.
The next question is usually 'what behaviour do I get on UB?' and then
it is not enough to just say that it is undefined. This is where the
examples involving the DS9K come in.

Bart v Ingen Schenau

 

[Ben Bacarisse]

"Ben Bacarisse" <[email protected]> a écrit dans le message de
(e-mail address removed)...

I was looking for another function that took a "no more than n"
parameter and which does not specify (explicitly) that it does not
write any more location than the minimum required (like fgets in the
discussion). I stupidly switched from strncat to strncpy because I
thought I could make the string cuter! How daft is that? What I
should have given is something like:

char buf[4] = "\0\0c";
strncat(buf, "a", 3);
buf[1] = 'b';
puts(buf);

We all know what should happen, but I can't see how the wording rules
out the same memset(buf, 0, 3) suggested for fgets. Similarly, the
second example should have been:

char buf[2] = "";
strncat(buf, "x", 20);

Given the operational description, only two bytes are written, but the
same goes for fgets -- the standard only states what is read and
written but it does not preclude other modifications.

Actually, the wording for strncat is unambiguous about how many characters
can be changed in the destination buffer: .

C99 7.21.3.2p2
The strncat function appends not more than n characters (a null character
and
characters that follow it are not appended) from the array pointed to by s2
to the end of
the string pointed to by s1. The initial character of s2 overwrites the null
character at the
end of s1. A terminating null character is always appended to the
result.264) If copying
takes place between objects that overlap, the behavior is undefined.

264) Thus, the maximum number of characters that can end up in the array
pointed to by s1 is
strlen(s1)+n+1.

The wording is not cristal clear, but s1 is only changed by appending
characters from s2 and the null and following characters are not appended,
and then a (single) terminating null character is always appended.

OK. I have always taken "it does this" to mean "... and nothing else"
but doubt has been raised about fgets and mbstowcs has an explicit
"only n chars written" clause. Do you feel that fgets is similarly
unambiguous?