P
S
Sidney Cadot
Wojtek said:
I didn't mean to mandate a classic "jump to subroutine" or something
like that; the notion I try to convey is that the OS specific part
should not be optimized away if it cannot be proven it has no side
effects. However, this hinges more on the definition of side effects
than on anything else.
My point is that for something like fopen(), the compiler can never be
able to prove this, unless it is given extra assumptions.
The C standard (erroneously, I think) provides those extra assumptions
by giving a list of cases it calls "side effects". If none of these
cases is relevant, the compiler may optimize at will.
By this rule, and given a strict interpretation of the rules, it is
permissible for a C compiler to optimize away fopen() here:
#include <stdio.h>
int main(void)
{
fopen("somefile", "rb");
return 0;
}
.... Following Mr. Pop's argument, it would even be permissible to
optimize this away if the fopen() would have a side effect (in the
traditional, non-C sense of the word) such as altering a file attribute.
My point is more that the C standard should not be readable in a way
that would condone this.
Admittedly, the 'volatile function' suggestion I gave is not a good
solution, as you point out. I think the root cause of the problem is in
the way the standard (incompletely) lists possible side effects.
I would think that a wording can be found that satisfies all, yes.
If not, I think it would at least be possible to do better than the
current 5.1.2.3 #2, which I think is just plain silly; it is easy to
give an example that isn't covered but should be covered (see above).
You are right, the volatile function idea is not good.
Indeed. However, I have a hard time seeing how a compiler could prove
that fopen() is side-effect free without the Standard lending a hand, as
it does now, with its flawed attempt to define a side effect.
Ok, but what part in POSIX specifically mandates that fopen() must not
be optimized away? It would have to specifically mention that the notion
of a file, as understood by its C compiler, is to be augmented by
considering file attributes to be covered by the phrase "modifying a
file". Basically, POSIX would need to override the C standard in this
respect.
Forget I said this ...
Best regards,
Sidney
W
Wojtek Lerch
Sidney Cadot said:
The C standard doesn't know the concept of the "OS specific part". It's all
just "the implementation". If the OS knows that something can be optimized
away, the implementation knows that it can be optimized away. How much of
that knowledge is made available to the compiler, on an implementation that
has a compiler, is an implementation detail that the C standard shouldn't be
concerned about.
If you don't like the term "side effects", perhaps we could call it
"required behaviour". The C standard requires that implementations behave
in a way that the C standard describes as required behaviour. Since the C
standard doesn't talk about file attributes, it neither requires nor forbids
fopen() to modify any attributes that files may have on some
implementations. On an implementation that does have file attributes, it's
up to the implementation to define how fopen() affects them, and to make
sure that it indeed affects them that way. I really can't understand why
you think it's silly that the C standard chooses not to mention that on an
implementation that defines how fopen() affects file attributes, fopen()
must affect them the way the implementation defines it. I think it would be
silly if it did.
....
You keep making the distinction between the compiler and the rest of the
implementation. That distinction does not exist in the C standard. The
compiler, in an implementation that has a compiler, is free to know as much
about the rest of the implementation as its authors want it to, and
therefore doesn't need to make any extra assumptions. If a call to fopen()
isn't required to produce any significant effects in some detectable
situations, the implementor is free to write a compiler that detects those
situations and optimizes those calls away. I really don't understand why
you think that allowing this freedom to the implementor is a bad idea.
Sure, but it's not *required* to optimize. If an implementor wants to have
a filesystem that guarantees that any successful call to fopen() changes
some attributes of a file, then even though the C standard doesn't refer to
those attribute changes as "side effects" and doesn't require them, he's
free to make his implementation behave that way. Of course, that means that
he'll have to make sure that his compiler doesn't optimize away the part of
fopen() that makes his implementation behave that way. Otherwise, the
implementation would break its own promise that calling fopen() changes the
attributes. But I don't think it makes sense to complain that the C
standard doesn't say that implementations are required to keep their own
promises.
Of course, provided that the implementation doesn't claim conformance to a
document that requires altering a file attribute. The C standard is not
such a document.
One more time: why not? What kind of problems are created by condoning it?
And where exactly do you want to draw the line?
On a typical POSIX system, writing data to a file causes the file's
attributes to be modified, and also causes the data to be transmitted
through some hardware and stored on some physical media. Do you think that
the C standard should consider both those things to be "side effects" and
forbid implementations to optimize them away? Even if the file is deleted
before its contents make it to the physical disk? If not, what kind of
wording would you use to describe the difference in general terms, in a way
suitable for a language standard?
I'm really curious what you think such wording might look like.
You keep complaining and calling it silly, but I still haven't seen an
explanation of why you think the standard should cover it. When I asked you
why you think it's such a big deal, your answer was that it wasn't really
such a big deal to you, personally. That's not very convincing.
....
What kind of problems does it create?
....
No, POSIX has no authority to change the C standard's definition of
"modifying a file". But POSIX can, and does, add its own requirements to
the semantics of fopen(). In particular, this is what POSIX says about how
fopen() modifies the POSIX file attributes:
If mode is w, wb, a, ab, w+, wb+, w+b, a+, ab+, or a+b, and the file did not
previously exist, upon successful completion, the fopen() function shall
mark for update the st_atime, st_ctime, and st_mtime fields of the file and
the st_ctime and st_mtime fields of the parent directory.
If mode is w, wb, w+, wb+, or w+b, and the file did previously exist, upon
successful completion, fopen() shall mark for update the st_ctime and
st_mtime fields of the file. The fopen() function shall allocate a file
descriptor as open() does.
D
Dan Pop
In said:
Well, here are the guarantees for binary streams:
3 A binary stream is an ordered sequence of characters that can
transparently record internal data. Data read in from a binary
stream shall compare equal to the data that were earlier written
^^^^^
out to that stream, under the same implementation. Such a stream
may, however, have an implementation-defined number of null
characters appended to the end of the stream.
As you probably know, it's not the actual implementations that dictate
how to interpret the C standard. The specifications of both text and
binary streams clearly reflect the same intent: with certain explicitly
mentioned exceptions, a program is supposed to be able to read back
what it wrote to a file. And there are plenty of multi-user
implementations that satisfy this condition, even if the Unix-based ones
don't.
Dan
S
Sidney Cadot
Wojtek said:
I do agree with you that this is how it should be. However, with Dan
Pop, I am unfortunately unable to find a justification for this entirely
reasonable position in the standard itself.
Let's not, shall we, it is confusing enough as it is, with the C
standard defining side effects as a subset of things that are normally
understood as side effects.
I have no problem whatsoever with the term 'side effects', but I do have
a problem with the definition given of it in the standard.
If I substitute back "side effects" here for "required behavior", I get
something that doesn't make sense to me.
After the "and": I'd like to see a standard quote supporting that.
Ok, well, we disagree on this.
I'm all for implementor freedom, as long as decent semantic guarantees
are made. I do not see anything in the standard that takes away the
freedom of the implementor to optimize an fopen away.
Yes, but it *may*. My point is that the standard should explicitly
forbid this for calls that have side effects (in the normal, not
c-standard sense).
Sure, and he will in practice. My point is that the implementor may
choose to ignore the side effects, and can now point to the standard to
justify this. I think that is bad.
I miss your point, here, I'm afraid.
I agree with you 100% on this, but I don't find a reference to the
possibility of externally defined by an entity outside of the c
standard, in the c standard.
It permits to optimize an fopen() away even if the fopen() would have a
side effect (in the traditional, non-C sense of the word) such as
altering a file attribute.
I cannot offer a wording, but that doesn't mean that people more clever
than I am couldn't think of one.
IMHO, the current wording is not ok.
Me too.
Well excuse me, but then you haven't read very well. The reason is that
it is currently possible to interpret the standard as Mr. Pop does,
basically, as in: there are circumstances where an implementation may
optimize away an fopen(), even if the plaform has last-touch attributes
or something similar.
Well, what can I say.
In practice: none. In theory: see above and before.
Hmmm, I was looking for fopen("somefile", "rb").... No mention of that?
Best regards,
Sidney
K
Keith Thompson
Assume an implementation that allows two C programs to run
concurrently. Assume that both programs access the same disk file,
one writing to it, the other reading from it. Assume that coherence
is maintained somehow (e.g., that the second program will never see a
partial write performed by the first). As far as I can tell, such an
implementation would not violate the guarantees for binary streams.
The "data that were earlier written out to that stream" were not
necessarily written by the current program.
(If the writing process isn't a C program, and doesn't access the disk
file though a "stream", things get a bit more confusing.)
The standard doesn't say anything about parallel processes, but it
doesn't specifically exclude them. The quoted paragraph could be read
either way; I'm deliberately trying to read it in a way that doesn't
exclude reasonable real-world implementations.
Perhaps what the standard *should* say is that data in external files
is "volatile", or that it's implementation-defined whether such data
is "volatile". I'm not necessarily using the word "volatile" in the
exact sense defined by the standard; the general idea is that the
contents of a file may change due to circumstances beyond the control
of the current program.
D
Douglas A. Gwyn
Keith said:
The actions of other processes/tasks lies outside the scope
of the standard. It describes the effect of a program "in
isolation". Also, a strictly conforming program has only
one thread of execution. (That isn't explicit either, but
it is inherent in the specification.)
No, we don't want to try to describe such environmental
complexities, especially since the only purpose would be
to exclude them.
I don't think any C implementer has misunderstood any of
this.
D
Dan Pop
In said:
You're confusing "stream" and "file". A stream exists only in the
context of the program creating it. It is created by an fopen()
function call and destroyed by a corresponding fclose() function call,
or as part of program termination. Depending on the way it was created,
a stream may or may not alter the contents of the associated file. What
the semantics of a stream explicitly exclude is having two streams
connected to the same file in write mode, at the same time, whether the
two streams belong to the same program or not (note that you don't need
a multitasking/multithreading environment in order to have a conformance
problem here: if the same program can simultaneously open the same file
in write mode twice, the same problem can arise). The issue can be
trivially handled by the implementor, using the file locking primitives
of the underlying OS.
Your scenario is perfectly possible, but it has exactly zilch to do
with our discussion, since the same file is accessed by two *different*
streams. The program opening the stream in read mode can get anything
at all without having the implementation's conformance affected in any
way. It is the program that opened it in read/write mode that has some
guarantees, and these guarantees are not affected by the existence of
the other program that accesses the file in read-only mode.
The quoted paragraph talks about streams, not about files, so you're
misreading it.
The standard says what it says and there are plenty of implementations
that conform to its requirements. Submit a DR if you think that it
should say something else, so that typical Unix implementations would
have a fighting chance to conform. Right now, they are hopelessly broken.
Dan
K
Keith Thompson
Douglas A. Gwyn said:
Then perhaps it's a good thing I'm not a C implementer.
}Suppose I have two C programs, Foo and Bar. Foo opens an existing
file and reads some data from it. Bar opens the same file and updates
it. Foo then seeks and re-reads the data (that would be identical to
the initial data if Bar hadn't intervened), and gets something
different from what it saw the first time. Is the implementation
non-conforming, or are the programs just acting outside the scope of
the standard? Does the standard forbid this behavior or does it
merely ignore it?
C
CBFalconer
Douglas A. Gwyn said:
ISTM that the situation, barring any use of seeks, splits into
three cases, all easily implemented.
a. The program writes, and then reads back, its own file. The
standard insists that it read back what it wrote.
b. The program writes, and then abandons, the file. What further
use is made of that is outside the provenance of the standard.
c. The program reads a file provided from somewhere else. All
that is strictly necessary is that repeated reads produce the same
result. Even this is not strictly necessary, since there is no
requirement for rewind to work.
The case involving seeks is neatly covered by the lack of
insistence that seeks work. All the other cases are covered by
the fact that almost any OS can provide exclusive access to any
given file.
J
James Kuyper
Sidney Cadot said:
It's not a requirement of the standard; it's outside the scope of the
standard. It's simply a matter of proper assignment of responsibility.
....
The implementor can also point to the standard and say "it doesn't say
I can't kill you - therefore, I can kill you". Is that problem within
the scope of the standard, or is it more properly a matter for the
legal code to address?
....
Except in the most indirect possble ways, the C standard also fails to
mention that a C program could cause data to be read from a modem. If
an implementation translated a program in such a way that, when
executed, it caused data to be read from a modem, it would be a
non-conforming implementation?
....
So? That's a problem with the implementation's failure to comform to
platform requirements, not a problem with the C standard.
Yes, but you haven't explained why you think that's the C standard's
responsibility. It seems to me that it's a platform-specific problem;
for instance, on POSIX platforms, it's the responsibility of the POSIX
standard.
W
Wojtek Lerch
But you don't seem to be willing to suggest a better definition, or
even to explain what you think is wrong with the existing one without
using terms that have no meaning in the C standard, like the "OS
specific part" of a function call, or "side effects in the tradidional
sense".
....
There is none. The C standard doesn't care. That's the point. Is it
really that difficult?
If an implementation wants to say that it has file attributes and to
promise that every successfull call to fopen() modifies them, it's its
choice whether to make that promise or not. If it does make that
promise, it's reasonable to expect that it actually behaves that way.
If it then sometimes optimizes a fopen() call away, that's a
discrepancy between what the implementation has promised and how it
behaves. You don't need a quote from the C standard to complain to
the implementor about the discrepancy. The discrepancy is all you
need to justify a complaint, even if it doesn't break the
implementation's conformance to the C standard.
....
I don't suppose you would be willing to explain how you imagine the C
standard should define "decent semantic guarantees", would you?...
Sorry, but what exactly do you mean by "calls that have side effects"?
If the implementation optimizes a call away, then it doesn't have
side effects. Unless the standard requires that call to have some
specific side effects, how exactly is the implementor supposed to know
that the call *should have* some side effects, according to you?
What do you mean by "ignore the side effects"? If he optimizes fopen()
away, there are no side effects left to ignore. Do you mean he can
choose to ignore *his own promise* that fopen() always modifies file
attributes? And that the current wording of the C standard can somehow
help him justify breaking the promise? I don't think that would make a
lot of sense. Just because the C standard doesn't require that
implementors make that promise in the first place, that's no
justification to make it but then break it.
....conformance to
a
Sorry, you lost me here. The possibility of what?
If running the unoptimized code of fopen() had a side effect of
causing a disk drive to move its head and read some data, do you
believe that the C standard should forbid optimizing that side effect
away, too? Should the C standard forbid filesystems to cache data?
....
No, but I think you'll have a hard time trying to convince them that
they need to.
Yeah, I've heard you...
....
What's a "platform"?
How can you know that it has last-touch attributes if not all calls to
fopen() are guaranteed to change the attributes?
If the implementation doesn't claim that every successful call to
fopen() modifies file attributes, why is the optimization a bad thing?
....
how
Funny, isn't it? It seems that POSIX doesn't mind if a compiler
optimizes your fopen() call away, either...
even to explain what you think is wrong with the existing one without
using terms that have no meaning in the C standard, like the "OS
specific part" of a function call, or "side effects in the tradidional
sense".
....
There is none. The C standard doesn't care. That's the point. Is it
really that difficult?
If an implementation wants to say that it has file attributes and to
promise that every successfull call to fopen() modifies them, it's its
choice whether to make that promise or not. If it does make that
promise, it's reasonable to expect that it actually behaves that way.
If it then sometimes optimizes a fopen() call away, that's a
discrepancy between what the implementation has promised and how it
behaves. You don't need a quote from the C standard to complain to
the implementor about the discrepancy. The discrepancy is all you
need to justify a complaint, even if it doesn't break the
implementation's conformance to the C standard.
....
I don't suppose you would be willing to explain how you imagine the C
standard should define "decent semantic guarantees", would you?...
Sorry, but what exactly do you mean by "calls that have side effects"?
If the implementation optimizes a call away, then it doesn't have
side effects. Unless the standard requires that call to have some
specific side effects, how exactly is the implementor supposed to know
that the call *should have* some side effects, according to you?
What do you mean by "ignore the side effects"? If he optimizes fopen()
away, there are no side effects left to ignore. Do you mean he can
choose to ignore *his own promise* that fopen() always modifies file
attributes? And that the current wording of the C standard can somehow
help him justify breaking the promise? I don't think that would make a
lot of sense. Just because the C standard doesn't require that
implementors make that promise in the first place, that's no
justification to make it but then break it.
....conformance to
a
Sorry, you lost me here. The possibility of what?
If running the unoptimized code of fopen() had a side effect of
causing a disk drive to move its head and read some data, do you
believe that the C standard should forbid optimizing that side effect
away, too? Should the C standard forbid filesystems to cache data?
....
No, but I think you'll have a hard time trying to convince them that
they need to.
Yeah, I've heard you...
....
What's a "platform"?
How can you know that it has last-touch attributes if not all calls to
fopen() are guaranteed to change the attributes?
If the implementation doesn't claim that every successful call to
fopen() modifies file attributes, why is the optimization a bad thing?
....
says about
how
that?
Funny, isn't it? It seems that POSIX doesn't mind if a compiler
optimizes your fopen() call away, either...
S
Sidney Cadot
James said:
That's fine by me. In the same vein, I think the standard has no
business defining side effects by enumeration (including file
modification, and excluding reading).
I think your analogy is faulty. As far as C is concerned, the standard
/is/ the law. In this law, an attempt is made to define side effects
that is (when read literally) is incomplete. That's all there is to it,
as far as I am concerned.
I sincerely think you mis-understand my position. I don't particularly
/mind/ the C standard being vague on some issues that are very hard to
define in a water-tight fashion (although, if possible, it should be
done); "side-effects" are a prime example. However, instead of being a
bit vague on the concept of side effects (as it is, for example, with
respect to files) the standard is quite specific (but, alas, incomplete
from a practical point of view) with regard to what is considered a side
effect and what not.
That's one way of looking at it. But I would appreciate a hint in the
standard that some side-effects can be imposed on library routines by
the operating environment, in additions to the short-list given in the
standards. With a few words to that effect, I'd be satisfied.
That would be fine by me. A couple of words with regard to this
separation of responsibilities in the standard couldn't hurt, though.
Best regards,
Sidney
S
Sidney Cadot
"Unable" would be a better word. I think I cannot provide something you
couldn't shoot holes in.
Having said that, suppose we were having this discussion in a universe
where the C standard didn't provide a definition, and upon your prompt
I would provide to you this definition:
"Accessing a volatile object, modifying an object, modifying a file, or
calling a function that does any of those operations are all side
effects, which are changes in the state of the execution environment."
.... would you accept this?
Well, I think it is impossible to talk about what's wrong with the
standard in this respect without reference to such notions; it's a bit
of a bootstrapping problem.
Not at all. It's just that I would have appreciated a disclaimer stating
something like this:
"The execution environment may impose semantics / side effects in
addition to (but not contradicting) the semantics and side effects as
defined in this standard, for the following I/O related functions: ..."
Is that so much to ask?
Again, fine by me. I'd appreciate a little note that implementations may
define additional semantics on top of the C standard.
No, because I cannot give something that is good enough to withstand
critique. However, I have given one example where currently the C
standard doesn't provide decent semantic guarantees, in my opinion.
That is answerable. A call has a side effect if there is a detectable
effect in the outside world of the call having been made, i.e., there is
a detectable difference between the effects on the execution environment
of a run of the program and a run of a derived program that is equal to
the original, except that this one call did not take place.
Something along those lines, at least.
Which definition of "side effect" are you using here?
He couldn't, and doesn't have to. What is important is that he knows
from the standard that the call *may* have a side effect.
The C standard makes no provisions for "external" promises as it is.
The standard tries quite hard to provide a situation where compliancy of
an implementation can be judged based on the text of the standard alone,
i.e., it is (or attempts to be) self-contained.
I would argue that this fails (where else) with its handling of the
concept of side effects. These cannot be properly defined without
references to "outside the C box", I think; so then you're left with a
number of options. My preference would be to be "intentionally vague"
rather than an (IMHO) misguided attempt to provide a definition.
Of course I've said the same thing in different words a dozen times now.
Please note that I'm perfectly happy to accept that some things are
defined outside the C scope (e.g. in POSIX), but the C standard should
write a couple of lines about that and its relation to such things,
that's all I ask.
I was writing gibberish there, I'm sorry. What I intended to say was:
"I don't find a reference to the possibility of applicable external
definitions by an entity outside of the c standard, in the c standard."
No. You are hitting on precisely the reason why it is so hard to define
"side effects" in the way that one would like. But you and I have an
intuitive idea. I'd prefer a sound definition, but on the other hand I
prefer a somewhat vague notion over a faulty definition.
That's true
Well, that's something.
I presume your question is designed to convey the idea that the C
standard doesn't have the notion of a platform, as currently written?
That explains the difficulties with defining side effects, then.
I agree with you that it isn't.
That's funny, indeed. What has POSIX to say of the access timestamp?
Best regards,
Sidney
D
Douglas A. Gwyn
Keith said:
The implementation is not nonconforming on that account.
There *is* no "second program" in the model covered by the
C standard.
D
Douglas A. Gwyn
CBFalconer said:
But we don't require that the implementation ensure exclusive access.
D
Dan Pop
In said:
Which means that, a conforming C implementation must behave *as if* there
is no second program, hence the semantics of streams.
Dan
J
James Kuyper
Sidney Cadot said:
No, the C standard is not all of the law that applies to a C program.
If you use a C program to embezzle money, the C standard isn't the
relevant law you've broken. If you've written a C compiler that
secretly steals the user's credit card information, it isn't the C
standand that says you've done something wrong. And if an
implementation promises that file attributes will be updated by
successfull calls to fopen(), then it isn't the C standard that is
violated if that implementation optimizes away the fopen() calls. It's
that implementation's own promises that are being broken.
D
Douglas A. Gwyn
Dan said:
No.
P
pete
James Kuyper wrote:
If I can read a C program
and pronounce all of the required diagnostics for translation,
and then when it's time to run the program
if I claim that all my streams are closed,
can I be a conforming implementation ?
If I can read a C program
and pronounce all of the required diagnostics for translation,
and then when it's time to run the program
if I claim that all my streams are closed,
can I be a conforming implementation ?