No way to add volatile access?

[Johannes Schaub (litb)]

How is it possible in C++ to access a specific port address using volatile
semantics? Here is a wrong way:

int volatile &port = *(int volatile*)0x318f;

This doesn't work according to the Standard, because the observable behavior
does only include read and writes to volatile objects. It does not include
access to potentially non-volatile objects through volatile lvalues.

How can this be solved? It seems a bit weird to me that the main purpose of
volatile was to read out mapped device addresses, but that volatile actually
seems to fail to supply that guarantee?

The only way to me seems to be to do this like

extern int volatile port;

And to map that using the linker to a specific address, but this looks to me
like an ugly solution. What if we compute the address at runtime and want to
do volatile accesses to it?

 

[Joshua Maurice]

How is it possible in C++ to access a specific port address using volatile
semantics? Here is a wrong way:

int volatile &port = *(int volatile*)0x318f;

This doesn't work according to the Standard, because the observable behavior
does only include read and writes to volatile objects. It does not include
access to potentially non-volatile objects through volatile lvalues.

How can this be solved? It seems a bit weird to me that the main purpose of
volatile was to read out mapped device addresses, but that volatile actually
seems to fail to supply that guarantee?

The only way to me seems to be to do this like

  extern int volatile port;

And to map that using the linker to a specific address, but this looks to me
like an ugly solution. What if we compute the address at runtime and want to
do volatile accesses to it?

Meh? I suggest you reread the standard.

Quoting C++03:
1.9 Program execution / 7

Accessing an object designated by a volatile lvalue (3.10), modifying an object, calling a library I/O function, or calling a function that does any of those operations are all side effects, which are changes in the state of the execution environment. Evaluation of an expression might produce side effects. At certain specified points in the execution sequence called sequence points, all side effects of previous evaluations shall be complete and no side effects of subsequent evaluations shall have taken place.7)

In your example, you are accessing the object through a volatile
lvalue, so it is an observable aspect of the abstract machine.

volatile has basically 3 C++03 standardized uses:
1- signal handlers
2- setjump and longjump nonsense
3- MMIO.
I imagine that the main driving point of volatile was to provide the
implementation with a hook to let users access MMIO.

However, I wouldn't focus too much on what the C++ standard has to say
here. Because volatile is relatively infrequently used, you're much
more likely to hit compiler bugs. Also, regulars on here have said
that volatile alone is generally insufficient to do MMIO, and that you
need to read whatever documents you have for the platform in question.

 

[Johannes Schaub (litb)]

Meh? I suggest you reread the standard.

Quoting C++03:
1.9 Program execution / 7

In your example, you are accessing the object through a volatile
lvalue, so it is an observable aspect of the abstract machine.

It isn't. It is just a side-effect. Modifying a non-volatile object is a
side effect too, yet it isn't an observable behavior.

volatile has basically 3 C++03 standardized uses:
1- signal handlers
2- setjump and longjump nonsense
3- MMIO.
I imagine that the main driving point of volatile was to provide the
implementation with a hook to let users access MMIO.

However, I wouldn't focus too much on what the C++ standard has to say
here. Because volatile is relatively infrequently used, you're much
more likely to hit compiler bugs.

At work, we had problems with these cases and bugged the intel bugtracker.
At intel, they said this exact thing (that only volatile for variables have
such observable behavior, and not just an access path) to defend their
position. And i think the Standard is clearly on their side, yet i find
documents (like "C++ and the Perils of double checked locking") that claim
volatile access alone can provide observable behavior.

I feel slightly lost, thus I made this question.

 

[Joshua Maurice]

It isn't. It is just a side-effect. Modifying a non-volatile object is a
side effect too, yet it isn't an observable behavior.



At work, we had problems with these cases and bugged the intel bugtracker..
At intel, they said this exact thing (that only volatile for variables have
such observable behavior, and not just an access path) to defend their
position. And i think the Standard is clearly on their side, yet i find
documents (like "C++ and the Perils of double checked locking") that claim
volatile access alone can provide observable behavior.

I feel slightly lost, thus I made this question.

As an asinine work-around, you could use placement new to construct a
new volatile object at the desired memory location. That should work
right?

 

[Joshua Maurice]

It isn't. It is just a side-effect. Modifying a non-volatile object is a
side effect too, yet it isn't an observable behavior.



At work, we had problems with these cases and bugged the intel bugtracker..
At intel, they said this exact thing (that only volatile for variables have
such observable behavior, and not just an access path) to defend their
position. And i think the Standard is clearly on their side, yet i find
documents (like "C++ and the Perils of double checked locking") that claim
volatile access alone can provide observable behavior.

I feel slightly lost, thus I made this question.

Quoting 1.9 Program execution / 6

The observable behavior of the abstract machine is its sequence of reads and writes to volatile data and calls to library I/O functions.6)

I guess it depends on your interpretation of volatile data. Does
"volatile data" mean "volatile lvalue" or "volatile object"? I would
assume that it means "volatile lvalue", but of course it's not
anywhere where I can find.

As for the uses of the term data in the standard, I see several
usages:
- Used in the idiomatic phrase "data member" to differentiate a member
sub-object from a member function. In this case, "data" could derive
meaning as being equivalent to "object", or it could derive meaning as
"an object which contains data (which can be directly read or
written)" as opposed to a function.
- Used to refer to the value of an object or objects, aka a particular
sequence of bits making up the value of the object.
- Used to refer to the contents of a file, aka a particular sequence
of bits stored in the file.

Searching for "observable behavior", "abstract machine", and
"volatile" don't immediately turn up any better references either.

I don't see any sort of clear cut way to figure out what they meant by
"volatile data". I guess it's another thing you can add to the list
"Things where the C++ standard is uselessly vague."

As I mentioned in a preceding post, I guess you could use placement
new to construct a volatile object at that particular memory location,
guaranteeing that you meet whatever definition of "volatile data" in
1.9 Program execution / 6.

 

[Pavel]

How is it possible in C++ to access a specific port address using volatile
semantics? Here is a wrong way:

int volatile&port = *(int volatile*)0x318f;

This doesn't work according to the Standard, because the observable behavior
does only include read and writes to volatile objects. It does not include
access to potentially non-volatile objects through volatile lvalues.

How can this be solved? It seems a bit weird to me that the main purpose of
volatile was to read out mapped device addresses, but that volatile actually
seems to fail to supply that guarantee?

The only way to me seems to be to do this like

extern int volatile port;

And to map that using the linker to a specific address, but this looks to me
like an ugly solution. What if we compute the address at runtime and want to
do volatile accesses to it?

Try to find out *compiler* memory barrier macros/builtins for your
particular compiler (do not confuse with CPU instructions executing
memory barriers). I have ones for GNU somewhere work.. something like

__asm__ __volatile__ ("" ::: "memory")

or similar.

Hope this will help
-Pavel

 

[Gil]

How is it possible in C++ to access a specific port address using volatile
semantics? Here is a wrong way:

int volatile &port = *(int volatile*)0x318f;

This doesn't work according to the Standard, because the observable behavior
does only include read and writes to volatile objects. It does not include
access to potentially non-volatile objects through volatile lvalues.

How can this be solved? It seems a bit weird to me that the main purpose of
volatile was to read out mapped device addresses, but that volatile actually
seems to fail to supply that guarantee?

The only way to me seems to be to do this like

  extern int volatile port;

And to map that using the linker to a specific address, but this looks to me
like an ugly solution. What if we compute the address at runtime and want to
do volatile accesses to it?

not sure I follow your question, you're saying that if I do

volatile const char * port = ( char * )0x318f;

some compiler would still be able to cache the value of *port in a
register instead of reading it directly from the physical port every
time it is accessed?
because I doubt it is so.

 

[Johannes Schaub (litb)]

Quoting 1.9 Program execution / 6

I guess it depends on your interpretation of volatile data. Does
"volatile data" mean "volatile lvalue" or "volatile object"? I would
assume that it means "volatile lvalue", but of course it's not
anywhere where I can find.

As for the uses of the term data in the standard, I see several
usages:
- Used in the idiomatic phrase "data member" to differentiate a member
sub-object from a member function. In this case, "data" could derive
meaning as being equivalent to "object", or it could derive meaning as
"an object which contains data (which can be directly read or
written)" as opposed to a function.
- Used to refer to the value of an object or objects, aka a particular
sequence of bits making up the value of the object.
- Used to refer to the contents of a file, aka a particular sequence
of bits stored in the file.

Searching for "observable behavior", "abstract machine", and
"volatile" don't immediately turn up any better references either.

I don't see any sort of clear cut way to figure out what they meant by
"volatile data". I guess it's another thing you can add to the list
"Things where the C++ standard is uselessly vague."

As I mentioned in a preceding post, I guess you could use placement
new to construct a volatile object at that particular memory location,
guaranteeing that you meet whatever definition of "volatile data" in
1.9 Program execution / 6.

Note that C++0x cleaned up and is exceedingly clear. It just says "Access to
volatile objects are evaluated strictly according to the rules of the
abstract machine.".

Thanks for the placement-new idea. I think that's the desired solution!

 

[Johannes Schaub (litb)]

not sure I follow your question, you're saying that if I do

volatile const char * port = ( char * )0x318f;

some compiler would still be able to cache the value of *port in a
register instead of reading it directly from the physical port every
time it is accessed?
because I doubt it is so.

I doubted it first too, but that's what C++0x says very clearly and what i
think C++03 is saying too.

Note that the notion of "side effect" is completely unrelated to this. My
suspicion is that mere access using volatile lvalues is a side-effect to
grant compilers to produce undefined behavior for the following without
doing heroic efforts to determine a correct execution if those lvalues
really point to non-volatile variables:

*port + *port

Which are two unsequenced side-effects on the same scalar object. Yet, to
make it not observable side-effect, the compiler optimizer is granted to
make good optimizations all over. Whether the latter is intended - i've no
idea

 

[Alf P. Steinbach /Usenet]

* Johannes Schaub (litb), on 31.08.2010 08:41:

I doubted it first too, but that's what C++0x says very clearly and what i
think C++03 is saying too.

Where?

It's certainly not the intention of 'volatile'.

However, even though the generated machine code is prohibited from doing that
kind of optimization, the processor itself may very well do it. And I don't
think the standard requires the machine code to force the processor to do the
external reads. I think it merely prohibits the machine code optimization.

Note that the notion of "side effect" is completely unrelated to this. My
suspicion is that mere access using volatile lvalues is a side-effect to
grant compilers to produce undefined behavior for the following without
doing heroic efforts to determine a correct execution if those lvalues
really point to non-volatile variables:

*port + *port

Which are two unsequenced side-effects on the same scalar object. Yet, to
make it not observable side-effect, the compiler optimizer is granted to
make good optimizations all over. Whether the latter is intended - i've no
idea

Not sure about this.


Cheers,

- Alf

 

[Stuart Redmann]

It isn't. It is just a side-effect. Modifying a non-volatile object is a
side effect too, yet it isn't an observable behavior.

At work, we had problems with these cases and bugged the intel bugtracker.
At intel, they said this exact thing (that only volatile for variables have
such observable behavior, and not just an access path) to defend their
position. And i think the Standard is clearly on their side, yet i find
documents (like "C++ and the Perils of double checked locking") that claim
volatile access alone can provide observable behavior.

I feel slightly lost, thus I made this question.

I'm a little bit confused here: You're trying to do MMIO on a specific
memory address? I think there are many problems involved with this:
(1) your C++ compiler may optimize away read/write operations, (2) the
pre-fetching unit of your CPU may optimize away reads/writes, and (3)
the cache may interupt your reads and writes. AFAIK, you can only
influence issues (1) and (2) by your own code, caching issues must be
solved through the OS. Your post relates to issue (1), but this is the
issue that can be checked most easily by inspecting the generated
machine code. This leaves (2) and (3). Have you taken these into
consideration?

BTW, using memory mapped IO without any library support seems to a bit
strange to me (unless you're developing drivers).

Regards,
Stuart

 

[Vladimir Jovic]

Stuart Redmann wrote:

[I hope I didn't snip anything relevant]

I'm a little bit confused here: You're trying to do MMIO on a specific
memory address? I think there are many problems involved with this:
(1) your C++ compiler may optimize away read/write operations, (2) the
pre-fetching unit of your CPU may optimize away reads/writes, and (3)
the cache may interupt your reads and writes. AFAIK, you can only
influence issues (1) and (2) by your own code, caching issues must be
solved through the OS. Your post relates to issue (1), but this is the
issue that can be checked most easily by inspecting the generated
machine code. This leaves (2) and (3). Have you taken these into
consideration?

volatile + placement new should solve that issue (as Joshua Maurice
suggested). At least, it works in my case (I did that for the platform I
am working on).

BTW, using memory mapped IO without any library support seems to a bit
strange to me (unless you're developing drivers).

I guess he is. Otherwise, why would you access a pretty much random
memory location?

 

[Stuart Redmann]

On 31 Aug. Vladimir Jovic wrote:

[I hope I didn't snip anything relevant]

volatile + placement new should solve that issue (as Joshua Maurice
suggested). At least, it works in my case (I did that for the platform I
am working on).

I don't know for sure, but I thought you'd have to add specific
instructions in order to add memory barriers (see Pavel's posting
elsethread), at least if you're reading the same address that you're
writing to.

I guess he is. Otherwise, why would you access a pretty much random
memory location?

The other postings of Johannes indicated to me that he wants to solve
some different problem. Who knows, maybe he just wanted to perform
MMIO as a kind of exercise. BTW, I don't think that you're able to
develop drivers under C++.

Regards,
Stuart

 

[Vladimir Jovic]

On 31 Aug. Vladimir Jovic wrote:

[I hope I didn't snip anything relevant]

volatile + placement new should solve that issue (as Joshua Maurice
suggested). At least, it works in my case (I did that for the platform I
am working on).

I don't know for sure, but I thought you'd have to add specific
instructions in order to add memory barriers (see Pavel's posting
elsethread), at least if you're reading the same address that you're
writing to.

Yes, he said something about clober :
http://ibiblio.org/gferg/ldp/GCC-Inline-Assembly-HOWTO.html#ss5.3

"If our instruction modifies memory in an unpredictable fashion, add
"memory" to the list of clobbered registers."

That seams to be very OS and platform specific.

The other postings of Johannes indicated to me that he wants to solve
some different problem. Who knows, maybe he just wanted to perform
MMIO as a kind of exercise. BTW, I don't think that you're able to
develop drivers under C++.

What stops you from developing a driver under c++? In my experience it
works fine.

 

[James Kanze]

[...]

Quoting 1.9 Program execution / 6

I guess it depends on your interpretation of volatile data.
Does "volatile data" mean "volatile lvalue" or "volatile
object"? I would assume that it means "volatile lvalue", but
of course it's not anywhere where I can find.

Formally, any use of the original reference is undefined
behavior, as far as the standard is concerned. As far as the
language is concerned, there is no object (known to the
language) at the address 0x318F. The intent, of course, is that
each implementation define this undefined behavior in a way that
makes sense for that implementation.

From a QoI point of view, I would expect an implementation to
treat port as if it referred to a volatile object. Regardless
of how it was initialized.

As for the uses of the term data in the standard, I see several
usages:
- Used in the idiomatic phrase "data member" to differentiate a member
sub-object from a member function. In this case, "data" could derive
meaning as being equivalent to "object", or it could derive meaning as
"an object which contains data (which can be directly read or
written)" as opposed to a function.
- Used to refer to the value of an object or objects, aka a particular
sequence of bits making up the value of the object.
- Used to refer to the contents of a file, aka a particular sequence
of bits stored in the file.

Searching for "observable behavior", "abstract machine", and
"volatile" don't immediately turn up any better references either.

I don't see any sort of clear cut way to figure out what they
meant by "volatile data". I guess it's another thing you can
add to the list "Things where the C++ standard is uselessly
vague."

In this case, I think, intentionally. The intent of the
standard with regards to volatile is largely to provide a hook
for implementations to define additional behavior not defined in
the standard.

 

[James Kanze]

On Aug 30, "Johannes Schaub" wrote:

[...]

I'm a little bit confused here: You're trying to do MMIO on a specific
memory address?

That's what volatile was introduced for. It's its raison
d'être.

I think there are many problems involved with this:
(1) your C++ compiler may optimize away read/write operations, (2) the
pre-fetching unit of your CPU may optimize away reads/writes, and (3)
the cache may interupt your reads and writes. AFAIK, you can only
influence issues (1) and (2) by your own code, caching issues must be
solved through the OS. Your post relates to issue (1), but this is the
issue that can be checked most easily by inspecting the generated
machine code. This leaves (2) and (3). Have you taken these into
consideration?

The intent of volatile is that the compiler should do whatever
is necessary to inhibit all of these issues. (In practice, most
compilers I know only handle (1), which makes volatile more or
less useless in some cases. Check your hardware, however---it's
quite possible that the hardware has been designed to recognize
addresses that are memory-mapped, and inhibit (2) and (3)
automatically. But don't count on the compiler getting (2) or
(3) right, regardless of the intent of the standard.)

BTW, using memory mapped IO without any library support seems
to a bit strange to me (unless you're developing drivers).

Well, someone has to write the library if you're going to have
library support. (He's obviously not developing a user process
under Windows, because memory mapping will mean that he can't
access any memory mapped IO anyway.)

 

[James Kanze]

On 31 Aug. Vladimir Jovic wrote:

I don't know for sure, but I thought you'd have to add
specific instructions in order to add memory barriers (see
Pavel's posting elsethread), at least if you're reading the
same address that you're writing to.

You might. Or the compiler might insert them automatically (fat
chance, but I'd argue that that is the intent). Or the target
platform might be a small, embedded processor with no fancy
pipeline or cache. Or it might recognize memory mapped
addresses, and inhibit the pipeline and cache when they are
accessed. (The only platform I know well for this is Sparc,
which does require memory barriers in this case. And neither
Sun CC nor g++ insert them, so you do need some assembler.)

[...]

BTW, I don't think that you're able to
develop drivers under C++.

Why not (except for possible problems with volatile)? There are
certainly some features that you'd want to avoid---I don't think
that an exception would be a good idea in an interrupt handler.
And there are probably a few things where you'd need additional
assembler (saving and restoring context in an interrupt handler,
for example---or inserting membar instructions if they're
needed). But globally, I don't see any real reason to avoid C++
for this.

 

[James Kanze]

* Johannes Schaub (litb), on 31.08.2010 08:41:

[...]
[...]

Where?

It's more a question of: where does it say that a compiler can't
do this? (Such optimizations are certainly allowed for
variables without the variable.)

Johannes has already pointed out: the "guarantees" with regards
to volatile concern accesses to volatile objects, not accesses
through volatile lvalues (which may be a defect in the
standard). And as far as I can tell, there is no C++ object at
0x318F (so formally, the code has undefined behavior).

It's certainly not the intention of 'volatile'.

Totally agreed. The intent, however, is to provide a hook for
implementations to define some specific behavior.

However, even though the generated machine code is prohibited
from doing that kind of optimization, the processor itself may
very well do it. And I don't think the standard requires the
machine code to force the processor to do the external reads.
I think it merely prohibits the machine code optimization.

The standard says that "what constitutes an access is
implementation defined" (from memory, but I think that's
basically correct). One of the major intents behind volatile is
memory mapped IO, so from a QoI point of view, the
implementation should define "access" in a way that makes memory
mapped IO work, say by defining it to be a read or write cycle
on the main memory/IO bus. And then, of course, generate code
conform to this definition, with fences or membars if necessary.
In practice, compilers are not conform in this point, in that
they don't define what they mean by access. So who knows what
they do---they're keeping it a deep dark secret. (By studing
generated code, I can say that certain compilers consider
"access" to be "executed a load or a store instruction". Which
certainly doesn't conform to the intent, and which, I suspect,
if documented would make people think and ask questions.)

I suspect it's rather to ensure that something happens in cases
like:
int a;
int volatile *p;
a = p;
and a is not used later in the program.

Adding volatile does NOT add a sequence point, and so does NOT
change undefined behavior into defined.

 

[Johannes Schaub (litb)]

On 31 Aug. Vladimir Jovic wrote:

[I hope I didn't snip anything relevant]

volatile + placement new should solve that issue (as Joshua Maurice
suggested). At least, it works in my case (I did that for the platform I
am working on).

I don't know for sure, but I thought you'd have to add specific
instructions in order to add memory barriers (see Pavel's posting
elsethread), at least if you're reading the same address that you're
writing to.

I guess he is. Otherwise, why would you access a pretty much random
memory location?

The other postings of Johannes indicated to me that he wants to solve
some different problem. Who knows, maybe he just wanted to perform
MMIO as a kind of exercise. BTW, I don't think that you're able to
develop drivers under C++.

I'm not developing drivers or anything like that. The port thing was just an
example to make my point. It's my colleague that writes such stuff and I
asked purely out of interest.

Thanks for all your insights!

 

[Johannes Schaub (litb)]

I suspect it's rather to ensure that something happens in cases
like:
int a;
int volatile *p;
a = p;
and a is not used later in the program.

Quite possible, but then it would inevitably be a defect that they define
the observable behavior not for accesses using merely volatile qualified
lvalues. I tried to not assume a defect, which is why i suspected another
thing. However, taking into account the Standard could be defective here, I
agree with you that your explanation seems more likely.

Adding volatile does NOT add a sequence point, and so does NOT
change undefined behavior into defined.

I didn't say so. In fact I said much the opposite. I said that we have two
side effects on the same scalar object (accessing the same object thru a
volatile qualified lvalue) and that these are not sequenced relative to each
other. Which gives undefined behavior. Detecting at compile time whether a
volatile qualified lvalue refers to a non-volatile object or not isn't
solvable, which is why I suspected this reason behind them not restricting
side-effects to accesses thru references to actual volatile objects.

I avoid the term "sequence point" with in context of c++0x, because it
doesn't exist anymore. Cheers!