how long is double

[f]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

In debug version, the print out is:
-12.25938471938358500000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

In the release version, the print out is:
-12.25938471938358300000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

The above sum = a + b + c is just a part of my computation. I found
that my whole computation crushed in the debug version because some
number became zero and another number divide this number. But this did
not happened in the release version.

Why?

Thanks,

ff

 

[Cy Edmunds]

I have this

double sum, a, b, c;
sum = a + b + c;

You used a, b, and c without initializing them. Was this your intent?

printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

In debug version, the print out is:
-12.25938471938358500000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

In the release version, the print out is:
-12.25938471938358300000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

Looks the same to me. And where is that trailing comma coming from?

The above sum = a + b + c is just a part of my computation. I found
that my whole computation crushed in the debug version because some
number became zero and another number divide this number. But this did
not happened in the release version.

Perhaps you can coax The Great Carsoni out of retirement. Or post some code.

Why?

Thanks,

ff

Your massive crossposting is very bad usenet manners.

 

[Andrew Koenig]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

In debug version, the print out is:
-12.25938471938358500000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

In the release version, the print out is:
-12.25938471938358300000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

The above sum = a + b + c is just a part of my computation. I found
that my whole computation crushed in the debug version because some
number became zero and another number divide this number. But this did
not happened in the release version.

Why?

Apparently the discrepancy is between -12.25938471938358500000
and -12.25938471938358300000, a difference of 2 in the 17th significant
digit.

I'm guessing--and this is only a guess--that in production mode, the
compiler tells the machine to compute a+b+c by using its extended-precision
intermediate register (80 bits, if I remember correctly), and in debug mode
it actually truncates the intermediate result to 64 bits. That might well
give a discrepancy of about this degree. I'd have to look at the binary
representations of the numbers to be sure, and I'm too lazy right now --
but if your computation is so brittle that such a small deviation crashes
it, you may want to rethink it.

 

[Jack Klein]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

In debug version, the print out is:
-12.25938471938358500000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

In the release version, the print out is:
-12.25938471938358300000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

The above sum = a + b + c is just a part of my computation. I found
that my whole computation crushed in the debug version because some
number became zero and another number divide this number. But this did
not happened in the release version.

Why?

Thanks,

ff

If you value accuracy in floating point calculations, do not use
Visual C++ under any circumstances. When Microsoft changed from
16-bit to 32-bit operating systems and compilers, they made changes to
their floating point code for the purpose of providing compatibility
on all the different processors for which they would provide Windows
NT, most of which never happened.

In particular, they made these two changes:

1. In their 16-bit compilers, the long double type used the full 80
bit extended precision of the Intel floating point hardware. In their
32-bit compilers, long double is the same as double and uses the 64
bit double precision mode of the floating point hardware. There is no
way in Visual C++ at all to utilize the higher precision mode build
into the hardware.

2. In most cases, their math code sets the floating point hardware
control bits to limit precision to 64 bits, instead of using the 80
bit format normally used internally by the FPU. That means that you
lose precision on calculations entirely inside the FPU, and not just
when you store values to RAM.

Microsoft made the decision years ago that programmers were not
trustworthy to decide for themselves whether they were better off with
the highest precision the Intel FPU can provide, or they should
sacrifice performance and accuracy for compatible floating point
results on other processors that nobody actually bought Windows NT on.
They made the decision for you, and took away your control over the
precision of your results.

If you want the maximum precision and accuracy that the Intel FPU is
capable of providing, you have to give up on Visual C++ and switch to
another compiler, such as Borland or GNU, that gives you extended
precision long double and doesn't truncate the floating point control
bits.

 

[Thomas Mang]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

Your code yields undefined behavior, because sum, a, b and c have never
been initialized before using them for computations / printing.
So the outcome of the program is undefined, thus anything might happen -
and different outcome between debug and release version is just one
possibility.


regards,

Thomas

 

[Francis Glassborow]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

I found that the debug version and release version of the same code
give me different result. I am using VC++ 6.0.

In debug version, the print out is:
-12.25938471938358500000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

In the release version, the print out is:
-12.25938471938358300000 = -11.43596388399630500000,
-0.07591666113607631300, -0.74750417425120252000,

The above sum = a + b + c is just a part of my computation. I found
that my whole computation crushed in the debug version because some
number became zero and another number divide this number. But this did
not happened in the release version.

Why?

FP arithmetic is very sensitive to such things as rounding mode and
order of evaluation. On x86 architectures there are considerable
differences between calculations done entirely in register and ones
where the intermediate results are written back to memory. My guess is
that in debug mode more intermediate results are being written back and
thereby are being stripped of guard digits.

For example your problem with '0' can be the consequence of subtracting
two values that are almost equal and are actually 'equal' within the
limits of the precision supported by memory values (which often have
lower precision than register values). This is an interesting case
because it means that the heavily optimised (minimum of writing back)
release version works as naively expected while the debug version that
adheres strictly to the semantics of the abstract C++ machine fails.

 

[P.J. Plauger]

Microsoft made the decision years ago that programmers were not
trustworthy to decide for themselves whether they were better off with
the highest precision the Intel FPU can provide, or they should
sacrifice performance and accuracy for compatible floating point
results on other processors that nobody actually bought Windows NT on.
They made the decision for you, and took away your control over the
precision of your results.

If you want the maximum precision and accuracy that the Intel FPU is
capable of providing, you have to give up on Visual C++ and switch to
another compiler, such as Borland or GNU, that gives you extended
precision long double and doesn't truncate the floating point control
bits.

Well, yes, but... You're correct that Microsoft has settled on a
conservative floating-point model. The reasons you attribute for
the choice are doubtless uncharitable, at best. Moving to a compiler
that supports 80-bit arithmetic does not, however, ensure superior
floating-point results. We've found, for example:

-- that the freely available Borland compiler computes long double
floating-point literals only to double precision, thereby making
a hash of our long double math functions

-- that two popular packagings of GNU C++ on Windows, Mingw and
Cygwin, either fail to set the FPU mode optimally or let it flap
in the breeze, thereby making a hash of some double and all long
double calculations

The OP was surprised at a change of the least-significant bit in
a 53-bit result. Old floating-point hands know that the slightest
rearrangement of operations can yield such a difference. This does
not signal the End of Western Civilization as We Know It. Nor
will applying the kneejerk Anything But Microsoft fix stave off
that inevitable end. If you want good floating-point results,
you have to:

a) test the quality of your environment,

b) know how to fix it if it's broken, and

c) know what's good when you see it.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[Servé Lau]

If you want the maximum precision and accuracy that the Intel FPU is
capable of providing, you have to give up on Visual C++ and switch to
another compiler, such as Borland or GNU, that gives you extended
precision long double and doesn't truncate the floating point control
bits.

What about intel's compiler?

I do think however that the code is merely an uninitialized variable
problem.

 

[Ron Natalie]

I have this

double sum, a, b, c;
sum = a + b + c;
printf("%.20f = %.20f, %.20f, %.20f", sum, a, b, c);

Most likely the difference is that the debug mode does some fstores to put
things back into
memory locations. The floating point registers on the pentium are really 80
bits wide.

 

[Francis Glassborow]

What about intel's compiler?

I do think however that the code is merely an uninitialized variable
problem.

I very much doubt it, were that the case the results would have been
totally different. Much more likely that the OP snipped out too much
code. This is re-inforced by the rest of his article.

 

[Balog Pal]

Most likely the difference is that the debug mode does some fstores to put
things back into
memory locations. The floating point registers on the pentium are really 80
bits wide.

IMHO a more likely explanation can be that the 3 numbers in a + b + c are
added in a different order. And that is a possible source of difference at
the last bit of the precision.

We have the builtin op + working on builtin types -- the compiler is allowed
to rearrange the operations it knows to be associative and commutative, or
even precalculate the result if able, isn't it?

I recall an old discussion wwhere the compile time environment was different
from the runtime environment, including the fp precision and the outcome was
way more different.

Paul

 

[Francis Glassborow]

IMHO a more likely explanation can be that the 3 numbers in a + b + c are
added in a different order. And that is a possible source of difference at
the last bit of the precision.

I am not sure that the compiler has that much freedom when the order
produces different results. This is not the same as the requirements re
order of evaluation of sub-expressions (i.e. that there is no
requirement)

 

[Martijn Lievaart]

On Mon, 05 Jan 2004 11:18:40 -0500, Francis Glassborow wrote:

[ f'up comp.lang.c++ ]

I am not sure that the compiler has that much freedom when the order
produces different results. This is not the same as the requirements re
order of evaluation of sub-expressions (i.e. that there is no
requirement)

I think so. if b is almost -c and a is much smaller, more precision is
lost is a is added to b first than when b is added to c first. There is
nothing the compiler can do about this. If this order changes due to
optimization switches may be seen as a QOI issue, but even that is a
bridge to far for me.

M4

 

[Francis Glassborow]

On Mon, 05 Jan 2004 11:18:40 -0500, Francis Glassborow wrote:

[ f'up comp.lang.c++ ]

I am not sure that the compiler has that much freedom when the order
produces different results. This is not the same as the requirements re
order of evaluation of sub-expressions (i.e. that there is no
requirement)

I think so. if b is almost -c and a is much smaller, more precision is
lost is a is added to b first than when b is added to c first. There is
nothing the compiler can do about this. If this order changes due to
optimization switches may be seen as a QOI issue, but even that is a
bridge to far for me.

The Standard requires that additions are evaluated on a left to right
basis. There are no alternatives. In the case of integer operations the
compiler can usually get away with the 'as if' rule and reorder them but
in the case of floating point calculations where re-ordering produces a
different result it manifestly cannot do that and conform.

Want to quote the Standard to support a contention that re-ordering is
allowed (other than by the as-if rule)?

 

[Ron Natalie]

The Standard requires that additions are evaluated on a left to right
basis. There are no alternatives. In the case of integer operations the
compiler can usually get away with the 'as if' rule and reorder them but
in the case of floating point calculations where re-ordering produces a
different result it manifestly cannot do that and conform.\

The standard says NO SUCH THING. With little exception
the order of operations in C++ is unspecified. Do not confuse
parsing association/precedence with evaluation.

 

[Martijn Lievaart]

The standard says NO SUCH THING. With little exception the order of
operations in C++ is unspecified. Do not confuse parsing
association/precedence with evaluation.

I couldn't find it either. Both you and Francis normally know what you are
talking about, so I'll wait for Francis answer before deciding for sure.

M4

 

[Balog Pal]

I am not sure that the compiler has that much freedom when the order
produces different results. This is not the same as the requirements re
order of evaluation of sub-expressions (i.e. that there is no
requirement)

Suppose I have expression, like

double d = 1.1 + x + 2.2 + 3.3; // we have double x in scope

here the compiler must generate code that will do 3 separate additions an
that order? And emitting code equivalent to expression

double d = x + 6.6;

is not really allowed, just in practice we allow the compiler doing it
anyway with some switch?

IIRC the question for this thread used MSVC which has the /Op [Improve Float
Consistency] option. Which the original poster likely dind't use
consistently.

Paul

 

[James Curran]

According to the Note in the Standard (1.9.15, pg 7, PDF Pg 33):
"operators can be regrouped according to the usual mathematical
rules....[caveat about machines in which overflows produce an
exception]...the above expression statement can be rewritten by the
implementation in any of the above ways because the same result will occur."

I guess the key point is the meaning of "same result". I was told
(pre-C89) that a C compile could assume infinite precision when reordering
floating point expressions, something not ruled by that statement, and not
addressed (as far as I could see) elsewhere in the Standard.

--
Truth,
James Curran
Home: www.noveltheory.com Work: www.njtheater.com
Blog: www.honestillusion.com Day Job: www.partsearch.com
(note new day job!)

 

[Ed Avis]

If you want good floating-point results, you have to:

a) test the quality of your environment,

Is there a good test suite that will work for this? (Both for
detecting hardware failures/bugs and C++ compiler stupidities.)

Not that a test program would eliminate the need to know what you're
doing, of course.

 

[Ron Natalie]

"

I couldn't find it either. Both you and Francis normally know what you are
talking about, so I'll wait for Francis answer before deciding for sure.

The fourth paragraph of section 5 of the C++ standard:
Except where noted, the order of evaluation of operands of individual operators and subexpressions of individual

expressions, and the order in which side effects take place, is unspecified.