The Semantics of 'volatile'

[kid joe]

In the absence of information to the contrary, using volatile is
essentially always necessary when making use of extralinguistic
mechanisms. This conclusion follows from 5.1.2.3 p 5, which gives
the minimum requirements that a conforming implementation must
meet (and explained in more detail in the earlier comments on the
Black Box model). What I mean by 'necessary' is that, if volatile
is omitted, a developer has no grounds for complaint if something
doesn't work, just as there are no grounds for complaint if one
expects 'sizeof(int) <= sizeof(long)' and it isn't, or that signed
arithmetic will wrap on overflow (to pick an example at each end of
the spectrum).

For a long time, the "information to the contrary" was supplied
implicitly by a largely shared (and mostly correct) understanding of
how optimization is done and of the various machine environments in
which programs execute. Basically, volatile worked the same way
everywhere, within certain error bars. As time went on, optimizers
got smarter, and machine environments got more diverse, to the point
where the shared understanding is no longer a reliable indicator.
That's why there's confusion about what volatile means, and why
the community is starting to have discussions about what it does
mean and what it should mean. The Posix stance on threads is one
example of that.



I concur, except that I think the <OT></OT> marking isn't necessary
in this case. As a generic statement, an observation that volatile
is neither necessary nor sufficient for thread-shared data is (IMO)
quite apropos in comp.lang.c. It's only when the discussion starts
being limited to particular threading models that it becomes a
significant impedance mismatch for CLC.

Hi Tim,

Its been said that volatile is the multithreaded programmers best friend.

Cheers,
Joe

 

[Nobody]

The problem is that the compiler is *not* free to do this (as far as I can
see). Surely clearing the buffer *is* a side effect?

That seems to me to be what C99 says:

5.1.2.3 Program execution

...

[#2] Accessing a volatile object, modifying an object,
modifying a file, or calling a function that does any of
those operations are all side effects,10) which are changes
in the state of the execution environment. Evaluation of an
expression may produce side effects. At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place. (A summary of the sequence points is given in annex
C.)

This implies that all writes are side-effects, as are reads of volatile
objects.

If buffer originally contained non-zero values, and another thread
was monitoring its contents via a "volatile char *", it should be
guaranteed to see the elements being cleared in ascending order, with
buffer_ready only being set after all elements of buffer were cleared.

 

[Tim Rentsch]

[SNIP]

I think we're agreeing. Conformance to the standard is orthogonal
to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

No more so than any other part of the standard. "Volatile" means
"volatile",
not "do what you damned well please".

I think you need to make up your mind. If what you mean by
"correctness in the implementation" depends on implementing
volatile in the way the Standard says, then it is not orthogonal
to "conformance to the standard". Either the two conditions
/are/ orthogonal, in which case judging correctness is independent
of the Standard, or judging correctness depends on the Standard,
in which case they are /not/ orthogonal. It can't be both.
Or maybe you mean something different by "orthogonal" than
how the term is usually meant? If so maybe you could explain
that.

If the implementation can conform to the standard and not generate the
desired result, conformance is orthogonal to correctness. Isn't that your
argument? That implementation-defined means anything and everything? (Don't
bother. That was rhetorical. I'm outa this one.)

I wasn't making an argument. I was only clarifying my earlier
comments. Those comments were based just on your initial comment
(the most nested quoted portion above); they weren't offering
any conclusions about implementation-defined behavior.

Is what you're saying any different than the specification of
volatile bothers you because it allows things that you think it
shouldn't allow? I'm having trouble getting any other meaning
out of it.

 

[Tim Rentsch]

Are you both referring to the following paper?

"Volatiles Are Miscompiled, and What to Do about It"
http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

That is the one I meant. Their first example (2.1) is wrong I think:

======================================================================

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."

======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

Clearing the buffer is a side-effect, certainly, as the Standard uses
the term. More generally, the writes to buffer are accesses, and
so at end of 'buffer_ready = 1;' those accesses must be complete (at
least, that's one way of reading 5.1.2.3 p 5, although some people
consider another reading to be more consistent with how that clause
is supposed to be read).

The example is meant to illustrate "what does volatile mean". If it does
not mean what they think it does, the other claims seem suspect.

Points to consider:

1. What they are saying may be right, but they just may have said
it poorly.

2. Certainly some knowledgeable people consider an alternative
reading of 5.1.2.3 p 5 more appropriate, which would agree with
the conclusion following the above example, even though the
phrasing about side-effects is wrong (or at least misleading).

3. The other claims may not depend on the mistakes made in this
example.

Certainly I would agree that other comments in the paper deserve
scrutiny. But if you're asking whether the statements cited
above make it reasonable to simply dismiss the rest of the paper,
I would have to say No. At the very least, it is useful to
consider their model for what 'volatile' must imply, and
see what evidence gets turned up under those assumptions.

 

[Tim Rentsch]

The problem is that the compiler is *not* free to do this (as far as I can
see). Surely clearing the buffer *is* a side effect?

That seems to me to be what C99 says:

5.1.2.3 Program execution

...

[#2] Accessing a volatile object, modifying an object,
modifying a file, or calling a function that does any of
those operations are all side effects,10) which are changes
in the state of the execution environment. Evaluation of an
expression may produce side effects. At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place. (A summary of the sequence points is given in annex
C.)

This implies that all writes are side-effects, as are reads of volatile
objects.

If buffer originally contained non-zero values, and another thread
was monitoring its contents via a "volatile char *", it should be
guaranteed to see the elements being cleared in ascending order, with
buffer_ready only being set after all elements of buffer were cleared.

The statement about seeing elements cleared in ascending order is
wrong. Even under the most stringent reading of 5.1.2.3 p 5 and the
description of volatile in 6.7.3 p 6, the stores into buffer are
not guaranteed to occur in any particular order, because the
assignements to buffer are not made through a volatile-qualified
type. Hence the reads in the other thread, even though made through
a volatile-qualified access, might not see the same storage order.

Furthermore, there is disagreement about whether the changes to buffer
must occur before the change to the volatile variable 'buffer_ready'
completes, in /every/ conforming implementation. Certainly they must
in some implementations, but the more general statement is open to
different interpretations, even assuming the same model for what
constitutes a volatile-qualified access.

 

[Guest]

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines.  

I think Quality of Implementation (QoI) is the usual term.

Though I consider the entire "Quality" industry to be be based a
willfully
wrong re-definition of "quality" ("compliance with a standard").

"Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification

--
Nick Keighley

"The quality I have in mind is all-absorbing,
not just a way of doing things but a way of being."
a PHB

 

[Guest]

I suppose they could be, depending on how generically the terms
were meant.  In the particular case, I don't think they were,
because of different assumptions about what determines the
behavior on the other accesses.  For hardware memory registers, it
could be just about anything;  for inter-process synchronization,
the natural assumption would be that the other process would also
be a C program

why? People have been programming in multiple languages since about
1959 what evidence was there that they were going to stop in 1989?

(and so would view 'volatile' in the same way as

 

[Richard Bos]

That is the one I meant. Their first example (2.1) is wrong I think:

It is, but not for the reason you think it is.

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."
======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

No. That is, yes, it's a side effect, but it's not a side effect _on the
volatile object_. buffer_ready is volatile, so all accesses to it must
be done according to the abstract machine; but no other objects are
volatile, so they may be shuffled as you like.
The error in the example is that the developer has not properly
described his own intent. Clearly, from the text, his intent was that
buffer_ready was volatile _with respect to the buffer_; equally clearly,
from the code, that's not what he has written. What he should do, if he
wants the relative accesses of buffer_ready _and_ buffer itself to be
done in the exact order of the abstract machine, he should make them
both volatile, not just one or the other.

Richard

 

[Boudewijn Dijkstra]

Op Tue, 09 Jun 2009 10:06:38 +0200 schreef

Though I consider the entire "Quality" industry to be be based a
willfully wrong re-definition of "quality" ("compliance with a
standard").

Tool vendors cannot sell tools that translate developer intentions into
quality code (yet). They can however sell tools that help automate the
part of a quality _process_ that deals with standards compliance. The
fact that some people make the mistake of thinking that "compliance with a
standard" automagically causes quality, doesn't make the tool vendors the
bad guys.

 

[Guest]

Op Tue, 09 Jun 2009 10:06:38 +0200 schreef  


Tool vendors cannot sell tools that translate developer intentions into  
quality code (yet).  They can however sell tools that help automate the  
part of a quality _process_ that deals with standards compliance.  The  
fact that some people make the mistake of thinking that "compliance with a  
standard" automagically causes quality, doesn't make the tool vendors the  
bad guys.

I wasn't particularly picking on tool vendors.
What is ISO 9000 all about?

 

[Rich Webb]

That is the one I meant. Their first example (2.1) is wrong I think:

It is, but not for the reason you think it is.

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."
======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

No. That is, yes, it's a side effect, but it's not a side effect _on the
volatile object_. buffer_ready is volatile, so all accesses to it must
be done according to the abstract machine; but no other objects are
volatile, so they may be shuffled as you like.

No.

5.1.2.3 Para 2 "... Evaluation of an expression may produce side
effects. At certain specified points in the execution sequence called
sequence points, all side effects of previous evaluations shall be
complete and no side effects of subsequent evaluations shall have taken
place."

There's a sequence point at the end of "buffer = 0;"

*If* the compiler can determine that there are no other accesses to
buffer[] then it would be allowed to optimize-away the entire
expression, since "[a]n actual implementation need not evaluate part of
an expression if it can deduce that its value is not used and that no
needed side effects are produced."

However, it is not permitted to arbitrarily change the order of
evaluation of successive sequence points. Otherwise the compiler could,
legally, re-order the expressions in, say, alphabetical order and still
be conforming.

 

[Boudewijn Dijkstra]

Op Tue, 09 Jun 2009 14:25:53 +0200 schreef

I wasn't particularly picking on tool vendors.
What is ISO 9000 all about?

Don't know, but seems like the same difference to me.

 

[Chris M. Thomasson]

Are you both referring to the following paper?

"Volatiles Are Miscompiled, and What to Do about It"
http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

That is the one I meant. Their first example (2.1) is wrong I think:

======================================================================

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."

======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

The example is meant to illustrate "what does volatile mean". If it does
not mean what they think it does, the other claims seem suspect.

The code is totally busted if your on a compiler that does not automatically
insert a store-release memory barrier before volatile stores, and
load-acquire membars after volatile loads. I assume another thread will
eventually try to do something like:


int check_and_process_buffer() {
if (buffer_ready) {
/* use buffer */
return 1;
}
return 0;
}



AFAICT, MSVC 8 and above is the only compiler I know about that
automatically inserts membars on volatile accesses:


http://groups.google.com/group/comp.lang.c/msg/54d730b2650c996c



Otherwise, you would need to manually insert the correct barriers for a
particular architecture. Here is a portable version for Solaris that will
work on all arch's support by said OS:


#include <atomic.h>


volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
membar_producer();
buffer_ready = 1;
}


int check_and_process_buffer() {
if (buffer_ready) {
membar_consumer();
/* use buffer */
return 1;
}
return 0;
}

 

[Nobody]

The problem is that the compiler is *not* free to do this (as far as I can
see). Surely clearing the buffer *is* a side effect?

That seems to me to be what C99 says:

5.1.2.3 Program execution

...

[#2] Accessing a volatile object, modifying an object,
modifying a file, or calling a function that does any of
those operations are all side effects,10) which are changes
in the state of the execution environment. Evaluation of an
expression may produce side effects. At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place. (A summary of the sequence points is given in annex
C.)

This implies that all writes are side-effects, as are reads of volatile
objects.

If buffer originally contained non-zero values, and another thread
was monitoring its contents via a "volatile char *", it should be
guaranteed to see the elements being cleared in ascending order, with
buffer_ready only being set after all elements of buffer were cleared.

The statement about seeing elements cleared in ascending order is
wrong. Even under the most stringent reading of 5.1.2.3 p 5 and the
description of volatile in 6.7.3 p 6,

I never mentioned 5.1.2.3 p 5 or 6.7.3 p 6. I did mention 5.1.2.3 p 2,
which you decline to address.

Maybe I'm misinterpreting it; if you think so, say so (saying *why* would
also be useful).

the stores into buffer are
not guaranteed to occur in any particular order, because the
assignements to buffer are not made through a volatile-qualified
type.

5.1.2.3 p2 (which no-one seems to want to mention) seems to imply that
volatile makes no difference to writes, only to reads:

[#2] Accessing a volatile object, modifying an object, ....
are all side effects ....
At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place.

Modifying an object is a side-effect, and side-effects are supposed to
have completed at the end of an expression statement (e.g. "buffer=0;").

AFAICT, most of the problems with "volatile" appear to rely upon ignoring
5.1.2.3 p2, which may be why everyone seems to avoid mentioning 5.1.2.3 p2.

Furthermore 5.1.2.3 p3 says:

[#3] In the abstract machine, all expressions are evaluated
as specified by the semantics. An actual implementation
need not evaluate part of an expression if it can deduce
that its value is not used and that no needed side effects
are produced (including any caused by calling a function or
accessing a volatile object).

IOW, if an implementation wishes to elide any side-effects as "unneeded",
the onus is on the implementation to deduce that the side-effects really
are unneeded (e.g. if the value isn't used inside the translation unit and
there is no way it could be used from outside of the translation unit).

 

[Tim Rentsch]

why? People have been programming in multiple languages since about
1959 what evidence was there that they were going to stop in 1989?

Because of the context in which the discussions were taking place,
namely, trying to standardize C. It's much easier to specify
inter-process synchronization if it's limited to processes all
implemented in C, because the C language is under the control of
those standardizing the language. I'm sure other environments
would have been considered, but the emphasis would be on just
intra-language semantics, because that's within their scope and
under their control.

Of course, this is just my guess based on second-hand information.
Other people may have other guesses, and I wouldn't want to argue
that one guess is better than another.

 

[Tim Rentsch]

I think Quality of Implementation (QoI) is the usual term.

Certainly QoI is /a/ term, and I think it makes up part of what
the earlier poster was talking about. But I don't think it
captures the whole story (of course, I don't know for sure since I
don't know exactly what he was thinking, but that's what I think).
Anyway that's why I used the less definite terms in my comments.

 

[Tim Rentsch]

The problem is that the compiler is *not* free to do this (as far as I can
see). Surely clearing the buffer *is* a side effect?

That seems to me to be what C99 says:

5.1.2.3 Program execution

...

[#2] Accessing a volatile object, modifying an object,
modifying a file, or calling a function that does any of
those operations are all side effects,10) which are changes
in the state of the execution environment. Evaluation of an
expression may produce side effects. At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place. (A summary of the sequence points is given in annex
C.)

This implies that all writes are side-effects, as are reads of volatile
objects.

If buffer originally contained non-zero values, and another thread
was monitoring its contents via a "volatile char *", it should be
guaranteed to see the elements being cleared in ascending order, with
buffer_ready only being set after all elements of buffer were cleared.

The statement about seeing elements cleared in ascending order is
wrong. Even under the most stringent reading of 5.1.2.3 p 5 and the
description of volatile in 6.7.3 p 6,

I never mentioned 5.1.2.3 p 5 or 6.7.3 p 6. I did mention 5.1.2.3 p 2,
which you decline to address.

Maybe I'm misinterpreting it; if you think so, say so (saying *why* would
also be useful).

Yes, 5.1.2.3 p 2 certainly bears on the discussion, and it would
be good to address it.

It's important to understand, when considering how 'volatile' behaves,
that there are two "machines" under consideration: the physical
machine, and the abstract machine.

The physical machine is the computer as we experience it in our
program and how they behave. (Note: I'm speaking as though there is
only one persective on a physical machine, but in actuality there are
(at least) several. I'm going to ignore these distinctions for the
moment.) A physical machine always does /something/ -- possibly only
probabilistically, but still something -- and we can find out what it
does through experimentation. The physical machine exists in the
physical universe, and we can discover what it does in different
situations.

The abstract machine is a conceptual notion; it has no physical
existence but "exists" mainly in the minds of implementors. The
abstract machine is sort of a mathematical tool for defining
behavior -- C is defined in terms of how the "abstract machine"
behaves, not how a physical machine behaves.

The first and most important point of contact between the abstract
machine and the physical machine is the so-called "as-if" rule.
What this rule says, basically, is that the physical machine can
do anything at all, as long as the 'outputs' of a program match
what would happen if the physical machine and abstract machine
were always in lock step agreement.

The second point of contact between the abstract machine and
the physical machine is volatile-qualified access. Basically,
using volatile places additional restrictions on how aligned
(or unaligned) the abstract machine and the physical machine
may be.

The question you raised (about another thread monitoring the state of
different elements in the 'buffer' array) is concerned with the
physical machine. The reason for this is, the abtract machine
concerns only what happens /inside/ an implementation, so what happens
for another thread is determined not by the abstract machine but by
the physical machine. Threads are not a part of C; the Standard
doesn't say anything about them (at least not directly).

The paragraph you mention (5.1.2.3 p 2) imposes a requirement on the
/abstract/ machine, not on the /physical/ machine. In the abstract
machine the writes to 'buffer' must occur before the subsequent
assignment to 'buffer_ready'. However, they don't have to actually
occur that way in the physical machine. In fact, frequently they
don't, because (to name one example) stores done in a particular order
can be rearranged by the memory management unit. The stores are /in
order/ as seen by the abstract machine, but /out of order/ as seen by
the actual memory -- that is, the physical machine of the other
thread.

So, what the other thread sees has to match what the abstract machine
does (as explained in 5.1.2.3 p 2) /only if/ the physical machine is
required to match the abstract machine through additional requirements
that occur because of using 'volatile'. Because (as I explained
earlier) the use of 'volatile' in the example is not enough to make
the 'buffer' writes in the /abstract/ machine match up with what
happens in the /physical/ machine, in the physical machine (which is
what the other thread sees) those writes can happen in any order.

Does that all make sense?

the stores into buffer are
not guaranteed to occur in any particular order, because the
assignements to buffer are not made through a volatile-qualified
type.

5.1.2.3 p2 (which no-one seems to want to mention) seems to imply that
volatile makes no difference to writes, only to reads:

[#2] Accessing a volatile object, modifying an object, ...
are all side effects ...
At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place.

Modifying an object is a side-effect, and side-effects are supposed to
have completed at the end of an expression statement (e.g. "buffer=0;").

AFAICT, most of the problems with "volatile" appear to rely upon ignoring
5.1.2.3 p2, which may be why everyone seems to avoid mentioning 5.1.2.3 p2.

Again, 5.1.2.3 p 2 is talking only about the abstract machine, not
about the physical machine. Using 'volatile' doesn't affect what
happens in the abstract machine (except for the two special cases
named explicitly in the Standard, setjmp/longjmp and signal
handlers). Using 'volatile' does impose additional requirements on
how and where the physical machine and the abstract machine must be
in alignment, but those requirements do not extend to imposing
5.1.2.3 p 2 in each previous statement (that doesn't use a
volatile-qualified access) before a volatile access. There are
different opinions about just how lax or how strict these additional
requirements are, but even in the most strict interpretation it's
only required that all the assignments to 'buffer' be completed
before the store into the (volatile) buffer_ready; the previous
stores don't have to be done in any particular order in the
/physical/ machine, even though they must occur in a particular
order in the /abstract/ machine.

Furthermore 5.1.2.3 p3 says:

[#3] In the abstract machine, all expressions are evaluated
as specified by the semantics. An actual implementation
need not evaluate part of an expression if it can deduce
that its value is not used and that no needed side effects
are produced (including any caused by calling a function or
accessing a volatile object).

IOW, if an implementation wishes to elide any side-effects as "unneeded",
the onus is on the implementation to deduce that the side-effects really
are unneeded (e.g. if the value isn't used inside the translation unit and
there is no way it could be used from outside of the translation unit).

In a sense this paragraph is just a special case of the "as if"
rule -- in the abstract machine certain operations are required
to happen, and in a particular order, but in the physical machine
they don't have to happen in that order, or even happen at all,
/provided/ the end result is "as if" they happened as the abstract
machine would do them.

Note that using 'volatile' either would, or might, (some people
would say "would", others would only say "might") force some
expressions to be evaluated that could remain unevaluated if
'volatile' weren't used. (I think most people would say "would",
and personally I believe that's the most defensible interpretation.
However I don't want to dismiss the considered statements of
those who have expressed the less restrictive viewpoint here.)

 

[Phil Carmody]

That is the one I meant. Their first example (2.1) is wrong I think:

It is, but not for the reason you think it is.

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."
======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

No. That is, yes, it's a side effect, but it's not a side effect _on the
volatile object_. buffer_ready is volatile, so all accesses to it must
be done according to the abstract machine; but no other objects are
volatile, so they may be shuffled as you like.
The error in the example is that the developer has not properly
described his own intent. Clearly, from the text, his intent was that
buffer_ready was volatile _with respect to the buffer_; equally clearly,

There is no "volatile with respect to something else". There's
"volatile", and that's it.

from the code, that's not what he has written. What he should do, if he
wants the relative accesses of buffer_ready _and_ buffer itself to be
done in the exact order of the abstract machine, he should make them
both volatile, not just one or the other.

I strongly disagree with your interpretation of the spec.

Phil

 

[Tim Rentsch]

That is the one I meant. Their first example (2.1) is wrong I think:

It is, but not for the reason you think it is.

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."
======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

No. That is, yes, it's a side effect, but it's not a side effect _on the
volatile object_. buffer_ready is volatile, so all accesses to it must
be done according to the abstract machine; but no other objects are
volatile, so they may be shuffled as you like.
The error in the example is that the developer has not properly
described his own intent. Clearly, from the text, his intent was that
buffer_ready was volatile _with respect to the buffer_; equally clearly,
from the code, that's not what he has written. What he should do, if he
wants the relative accesses of buffer_ready _and_ buffer itself to be
done in the exact order of the abstract machine, he should make them
both volatile, not just one or the other.

I would like to offer some counterpoint.

First, 6.7.3 p 6 (defining volatile) says, in part:

An object that has volatile-qualified type may be modified
in ways unknown to the implementation or have other unknown
side effects. Therefore any expression referring to such an
object shall be evaluated strictly according to the rules of
the abstract machine, as described in 5.1.2.3.

5.1.2.3 includes this paragraph (p2):

Accessing a volatile object, modifying an object, modifying
a file, or calling a function that does any of those
operations are all side effects,11) which are changes in the
state of the execution environment. Evaluation of an
expression may produce side effects. At certain specified
points in the execution sequence called sequence points, all
side effects of previous evaluations shall be complete and
no side effects of subsequent evaluations shall have taken
place.

Because access to volatile requires (per 6.7.3 p 6) that 5.1.2.3
be faithfully observed, all the stores to buffer must be
completed (although in no particular order) before the (volatile)
store into buffer_ready.

Also, 5.1.2.3 says this (in p 8):

The least requirements on a conforming implementation are:

-- At sequence points, volatile objects are stable in the
sense that previous accesses are complete and subsequent
accesses have not yet occurred.

Notice the wording -- "previous accesses must be complete". It
doesn't say "previous volatile accesses". It says "previous
accesses."

Admittedly, the wording here is ambiguous; it could mean that
previous accesses to the same volatile object be complete (and
similarly for subsequent accesses). However, access to a
volatile object requires evaluation per 6.7.3 p 6, and therefore
per 5.1.2.3 p 2. So, completing an access to a volatile object
also means that all the assignments done in all previous
expressions must have been completed before the volatile access
side-effect occurs.

 

[Tim Rentsch]

John Devereux wrote:

FreeRTOS.org wrote:

I once wrote an article on compiler validation for safety critical
systems. In return somebody sent me a paper they had published
regarding different compilers implementation of volatile. I forget
the numbers now, but the conclusion of their paper was that most
compilers don't implement it correctly anyway!

If it was the same one posted here a few months ago, it started out with
a very basic false assumption about what volatile *means*. Casting the
rest of the paper into doubt as far as I can see.

Are you both referring to the following paper?

"Volatiles Are Miscompiled, and What to Do about It"
http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

That is the one I meant. Their first example (2.1) is wrong I think:

======================================================================

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."

======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

The example is meant to illustrate "what does volatile mean". If it does
not mean what they think it does, the other claims seem suspect.

The code is totally busted if your on a compiler that does not automatically
insert a store-release memory barrier before volatile stores, and
load-acquire membars after volatile loads. I assume another thread will
eventually try to do something like:


int check_and_process_buffer() {
if (buffer_ready) {
/* use buffer */
return 1;
}
return 0;
}



AFAICT, MSVC 8 and above is the only compiler I know about that
automatically inserts membars on volatile accesses:


http://groups.google.com/group/comp.lang.c/msg/54d730b2650c996c



Otherwise, you would need to manually insert the correct barriers for a
particular architecture. Here is a portable version for Solaris that will
work on all arch's support by said OS:


#include <atomic.h>


volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
membar_producer();
buffer_ready = 1;
}


int check_and_process_buffer() {
if (buffer_ready) {
membar_consumer();
/* use buffer */
return 1;
}
return 0;
}

Again, thank you for posting some excellent specific examples.

I would like to add one comment. Despite the differences, both
the MSVC 8 implementation and the Solaris implementations can
be conforming. The reason is the last sentence in 6.7.3 p 6,

What constitutes an access to an object that has
volatile-qualified type is implementation-defined.

Presumably the MSVC implementors and the Solaris implementors
reached different conclusions about how to define what
constitutes an access to a volatile-qualified object. Or, to put
that in the language I used earlier, what memory regime will be
aligned to under 'volatile'. It's possible, for example, that
the Solaris notion of volatile makes it work with some thread
implementations but not inter-process communication (or other,
differently implemented thread packages). (I'm only guessing
here; certainly I wouldn't call myself a Solaris expert.) In
any case, whichever choice is "better", both are allowed under
6.7.3 (provided of course the implementation-defined choice is
documented with the implementation).