converting float to double

[Mark McIntyre]

10 billion dollars *is* a large amount.

Not as far as financial markets are concerned.

Bear in mind that to someone from say Ethiopia, a hundred bucks is a
large amount.

The existence of larger amounts is not relevant,
assuming that's what your last sentence
implies

What my last sentence indicated was that banks *routinely* perform
significantly larger transactions, and in that context, 10Bn is small.

(it contains too many names I've never heard before to be
sure).

If you can be *rsed, look at http://www.lchclearnet.com/

A press release indicates a transaction volume of 408Tn in 2005.
LCH.Clearnet is merely one of many such exchanges. Its fairly easy to
see that a year's global business would overflow 64-bits, and to
retain enough accounts for GAAP rules would be way beyond it...

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[William Hughes]

What of the system is broken?


The current Turkish Lira would do pretty well. The previous one would
be bad, especially if you want to keep up to figures until the nearest
Kuru. But when somebody complains that there is a difference of one
cent in the result, that only means that the calculations are done
using the wrong data type. If you expect that anybody in Turkey before
the introduction of the new Lira bothered about anything less than
25000 Lira, you are wrong. (I still remember the 10,000,000 Lira tip
for the waiter after a small lunch for four.)

No financial program can cope with hyperinflation when it does not
regularly adjust to the inputs. You should know the smallest amount
that legally can be distinguished and base your calculations on that;
using integers. Picking the Turkish Lira is a red herring. You
could equally well have picked the Hungarian Pengo from just after
the war. If I have it right, at some moment they had notes of
1,000,000,000,000,000,000,000,000 Pengo. Or the Zimbabwian dollar
notes that have printed on them an ultimate date of validity (which
is not more than six months after introduction).

The whole point is that in most financial transactions it is precisely
defined how fractions of something should be rounded. Any attempt to
be slightly imprecise (using floating point) will fail at some point
or another.

No. The question is not whether the imprecise answer is the
correct answer (this is highly unlikely), but whether
the correct answer can be determined from the
imprecise answer.

You have to find out what the correct answer is and calculate
that. It is unlikely to be the native answer, whether calculated
using integer or floating point. The fundamental questions are:

i: can you determine the correct answer from
the calculations done using your chosen data type

ii how difficult is it to determine the correct answer.

For i there is no clear advantage to integer calculations (though it
is clear that there are native floating point type and native integer
types that do not satisfy i).

For ii, there may be small advantage to integer calculations, but
given that calculations with non-integer results (e.g. compound
interest) are needed, this advantage is, at best, quite small.

I would advocate making any system flexible and fault tolerant.
In this case it means taking input in many forms and doing
the same thing if the input is

1 cent
1.000001 cents
0.999999 cents

(yes this means that calculations with Turkish Lira are not
exact to the lira) and failing gracefully when the numbers
become large.

On the balance, I would expect a floating point approach
to be preferable, but double, while carrying any needed
practical precision, may not have the precision needed to
obtain the correct accounting answer. [Note the question
is not, "do native calculations give the accounting answer",
they do not. The question is "Can the accounting answer
be determined from the native calculations?"]

- William Hughes

 

[Dik T. Winter]

> Dik T. Winter wrote: ....
>
> No. The question is not whether the imprecise answer is the
> correct answer (this is highly unlikely), but whether
> the correct answer can be determined from the
> imprecise answer.

And the answer is: if the calculations are more than trivial certainly
not.

> You have to find out what the correct answer is and calculate
> that. It is unlikely to be the native answer, whether calculated
> using integer or floating point. The fundamental questions are:

Using integer arithmetic you are sure you get the correct answer
(supposing, of course, that you properly scale everything).

> i: can you determine the correct answer from
> the calculations done using your chosen data type
> ii how difficult is it to determine the correct answer.
>
> For i there is no clear advantage to integer calculations (though it
> is clear that there are native floating point type and native integer
> types that do not satisfy i).

If a native integer type is not sufficient, use something larger.

> For ii, there may be small advantage to integer calculations, but
> given that calculations with non-integer results (e.g. compound
> interest) are needed, this advantage is, at best, quite small.

Compound interest is best calculated using yearly interest calculation
with proper rounding after each year, otherwise you will almost
certainly *not* get the correct answer.

> (yes this means that calculations with Turkish Lira are not
> exact to the lira) and failing gracefully when the numbers
> become large.

And this means that if the Turkish law requires something to be
calculated to the Lira that your system is not good enough.

 

[William Hughes]

And the answer is: if the calculations are more than trivial certainly
not.

??. You cannot do non-trivial calculations to a precision
greater that 1/2 of your smallest currency unit?

Using integer arithmetic you are sure you get the correct answer
(supposing, of course, that you properly scale everything).

No, you need a large enough integer type as well (scaling
will not help if you are using char). Also, how do you
do compound intererst calculations "using integer arithmetic"
and how does the fact that you are using integer arithmetic
help you to determine a "correct' answer based on a non-mathematical
standard of correctness.

If a native integer type is not sufficient, use something larger.

Indeed. But why can't this "something larger" be a floating
point type with sufficient precision? If there is sufficient
precision you can still determine the correct answer.

Compound interest is best calculated using yearly interest calculation
with proper rounding after each year, otherwise you will almost
certainly *not* get the correct answer.

This can be done as conveniently using floating point arithmetic
as using fixed point arithmetic. In either case you have to
do rounding.

And this means that if the Turkish law requires something to be
calculated to the Lira that your system is not good enough.

If this is a requirement, such a system can be written
as easily using high precision floating point as large
integer. However, note that such a law would
regularly require inputs to be specified with an error
of less than 10^-9. It is unlikely to be honoured or
enforced.

- William Hughes

 

[Random832]

And the answer is: if the calculations are more than trivial certainly
not.

Converting a floating point input to an integer _once_ IS trivial.

 

[Ernie Wright]

Fixing the root of the problem appears to be off the table.

We are therefore wasting time by solving some other problem we invented.

- Ernie http://home.comcast.net/~erniew

 

[rdilipk]

It's inherently possible to do this portably.

My guess (and it's only a guess) is that his vendor is providing raw
data of type "float", probably in 32-bit IEEE format in some
particular byte order. If Dilip has already been successful in
storing this incoming data in objects of type "float", then that part
of the problem is solved.

The remaining problem is what to do with the information once he has
it.

This is absolutely **spot-on**! I shudder to think what would happen
if you are armed with hard facts (as opposed to guessing)

But thanks to the many wise men who contributed to this thread now I
have a far better understanding of my problem.

 

[Dilip]

No, the "two decimal places" refers to the desired value,
a value known by the supplier of the stock prices.
What he has is a float that approximates the desired value.
As you note, this will rarely be exact.

This is **exactly** true.

Once again I cannot convince my boss to remove that nasty
mutiply/divide by 1000.0 kludge since he seems to believe that solves
whatever problem they have been having. Please read my very first post
to understand why I say this -- this kludge rounded off the number to
the nearest cent higher than the floating point representation and he
seems to be happy with that. (i.e what started off with
59.889999389648437 caused problems and what made him happy after the
kludge was the fact the same number got approximated as
59.890000000000001).

Everyone posted here seems incredibly astute -- I am almost cowering in
comparison. But the good thing I found the right way to solve this
problem. Here is what I have come up with: (i am not suggesting to the
management as yet though).

int main()
{
float f = 59.89F;
double d = ff;

double d_fractional, d_integral, d_floorvalue;
d_fractional = d_integral = d_floorvalue = 0;
d_fractional = modf(d, &d_integral);
d_floorvalue = floor(d_fractional*100.0 + 0.5) / 100.0;

double result = d_integral + d_floorvalue;

return 0;
}

Am I in the right direction?

 

[Random832]

This is **exactly** true.

Once again I cannot convince my boss to remove that nasty
mutiply/divide by 1000.0 kludge since he seems to believe that solves
whatever problem they have been having. Please read my very first post
to understand why I say this -- this kludge rounded off the number to
the nearest cent higher than the floating point representation and he
seems to be happy with that. (i.e what started off with
59.889999389648437 caused problems and what made him happy after the
kludge was the fact the same number got approximated as
59.890000000000001).

Everyone posted here seems incredibly astute -- I am almost cowering in
comparison. But the good thing I found the right way to solve this
problem. Here is what I have come up with: (i am not suggesting to the
management as yet though).

int main()
{
float f = 59.89F;
double d = ff;

double d_fractional, d_integral, d_floorvalue;
d_fractional = d_integral = d_floorvalue = 0;
d_fractional = modf(d, &d_integral);
d_floorvalue = floor(d_fractional*100.0 + 0.5) / 100.0;

double result = d_integral + d_floorvalue;

return 0;
}

Am I in the right direction?

Actually, I'd say eschew floating point except where necessary

#include <math.h>
int main() {
float f=59.89f;
long cents;
#if __STDC_VERSION__ >= 199901L
cents = lrint(f*100.0)
#else
/* This code does not handle negative numbers correctly, but it's
not needed for your case. */
cents = (f*100.0+0.5)
#endif

printf("%d.%02d",cents/100,cents%100);

/*Use integral cents [or thousandths, or whatever precision you're
supposed to be using, I don't think you've told us] for all further
calculations.*/
}

This is financial data, you can't afford to be screwing around with
floating point more than is _strictly_ necessary.

Get it into your boss's head that the problem is caused by the use of
floating point, and multiplying/dividing by 0x1.f4p+9 is not going to
solve anything in the long run.

 

[William Hughes]

This is **exactly** true.

Once again I cannot convince my boss to remove that nasty
mutiply/divide by 1000.0 kludge since he seems to believe that solves
whatever problem they have been having. Please read my very first post
to understand why I say this -- this kludge rounded off the number to
the nearest cent higher than the floating point representation and he
seems to be happy with that. (i.e what started off with
59.889999389648437 caused problems and what made him happy after the
kludge was the fact the same number got approximated as
59.890000000000001).

Everyone posted here seems incredibly astute -- I am almost cowering in
comparison. But the good thing I found the right way to solve this
problem. Here is what I have come up with: (i am not suggesting to the
management as yet though).

int main()
{
float f = 59.89F;
double d = ff;

double d_fractional, d_integral, d_floorvalue;
d_fractional = d_integral = d_floorvalue = 0;
d_fractional = modf(d, &d_integral);
d_floorvalue = floor(d_fractional*100.0 + 0.5) / 100.0;

double result = d_integral + d_floorvalue;

return 0;
}

Am I in the right direction?

Yes, modulo the fact that you can achieve almost exactly the
same thing by

double result = (floor(100.0*f + 0.5))/100.0

However, the more complicated method may impress your boss
who seems to have a bad case of PHB syndrome.

- William Hughes

 

[Ernie Wright]

But the good thing I found the right way to solve this problem.

No you haven't.

If the floats you're getting are precise to one cent, you can recover
their exact values with

dollars = ( int ) f;
cents = ( int )( f * 100 + 0.5 ) % 100;

You can then assign this to a double with

d = dollars + cents / 100.0;

Or you can use your more complicated procedure, or any of the others
that have been posted.

NONE of them solve your problem. They have exactly the same voodoo
property as the original kludge. They will all work on some values and
fail on others. Instead of 58.89, try them all with 59.11. Your "right
way," my way above, and the original

f *= 1000.0;
d = (double)f / 1000.0;

kludge all produce a double with EXACTLY the same value:

59.10999999999999943156581139...

You convinced yourself before your first post that the problem was this
float-into-double business, you entitled your post with that mindset,
and after all that's been said, you still think this.

Your problem is the truncation that happens later. THAT's where you
need to do some sort of rounding. NOT HERE.

- Ernie http://home.comcast.net/~erniew

 

[Ernie Wright]

In most implementations, both float and double represent numbers as

(1 + m / 2^b) * 2^e

where m is (part of) the mantissa, b is the number of bits in m, e is
the exponent, and '^' denotes exponentiation. b is 23 for floats and 53
for doubles. Given the m, b, e of a float, you get the double with

m <<= 30
b += 30
e = e

Oops. b is 52 for doubles, not 53. So

m <<= 29
b += 29

and so on.

- Ernie http://home.comcast.net/~erniew

 

[Dilip]

No you haven't.

If the floats you're getting are precise to one cent, you can recover
their exact values with

dollars = ( int ) f;
cents = ( int )( f * 100 + 0.5 ) % 100;

You can then assign this to a double with

d = dollars + cents / 100.0;

Or you can use your more complicated procedure, or any of the others
that have been posted.

NONE of them solve your problem. They have exactly the same voodoo
property as the original kludge. They will all work on some values and
fail on others. Instead of 58.89, try them all with 59.11. Your "right
way," my way above, and the original

f *= 1000.0;
d = (double)f / 1000.0;

kludge all produce a double with EXACTLY the same value:

59.10999999999999943156581139...

You convinced yourself before your first post that the problem was this
float-into-double business, you entitled your post with that mindset,
and after all that's been said, you still think this.

Your problem is the truncation that happens later. THAT's where you
need to do some sort of rounding. NOT HERE.

I totally understand. I probably miscommunicated something. Its true
that the problem happens in another part of the system but since
changing that part of the system is simply impossible (I have no idea
why.. I am just a new guy around here), the way they solve it is by
making that double look like what they want on **my** end before that
value gets shipped off and truncated or rounded off or whatever.

See the difference?

Of course in order to make it "look like what they want" they used that
stupid kludge which as you pointed out is not going to help me in all
cases.

Oh well.. at least I have a "I told you so" moment coming up.

 

[Random832]

I totally understand. I probably miscommunicated something. Its true
that the problem happens in another part of the system but since
changing that part of the system is simply impossible (I have no idea
why.. I am just a new guy around here), the way they solve it is by
making that double look like what they want on **my** end before that
value gets shipped off and truncated or rounded off or whatever.

Well, if that's the case, just add 0.000001 - it's better if it _looks_
like a kludge, because then more knowledgeable people can fix it later.
Make sure to comment it something like "code elsewhere rounds down
improperly, so make sure we're on the right side of the real value by
adding this."

It'll also (probably) always work to do what you want.

Or just go with their way, and when it blows up, say "you should have
let me change it"

 

[Ernie Wright]

I totally understand. I probably miscommunicated something. Its true
that the problem happens in another part of the system but since
changing that part of the system is simply impossible (I have no idea
why.. I am just a new guy around here), the way they solve it is by
making that double look like what they want on **my** end before that
value gets shipped off and truncated or rounded off or whatever.

OK. In that case, I'd go with Random832's second option: leave it
alone for now.

The kludge does have an effect on the value. Using 59.11,

d = f: 59.1100006103515625
kludge: 59.10999999999999943156581139192

The kludge produces a value that differs from 59.11 by a much smaller
amount. In fact, it's the best possible double-precision representation
of 59.11. Furthermore, I'm pretty sure that'll be true for all stock
prices you're likely to encounter.

So maybe the problem isn't truncation. It's just the need for best
possible accuracy. Your safest option is to leave this alone until you
know more about the problem this is supposed to fix.

- Ernie http://home.comcast.net/~erniew

 

[Keith Thompson]

Once again I cannot convince my boss to remove that nasty
mutiply/divide by 1000.0 kludge since he seems to believe that solves
whatever problem they have been having. Please read my very first post
to understand why I say this -- this kludge rounded off the number to
the nearest cent higher than the floating point representation and he
seems to be happy with that. (i.e what started off with
59.889999389648437 caused problems and what made him happy after the
kludge was the fact the same number got approximated as
59.890000000000001).

So multiplying and dividing by 1000.0 gives you a seemingly better
result if the input happens to be 59.889999389648437 (or whatever).

Can you find a case where multiplying and dividing by 1000.0 gives you
a *worse* result, and show it to your boss?

 

[christian.bau]

I am showing exactly what is being done in our codebase. Why do you
say there is a compiler bug? If you try the example, as soon as the
1st line is executed the variable f contains 59.889999. After its
stuffed into double d, it becomes 59.889999389648437 (I know these
values vary after a certain decimal position). The
multiplication/division to 1000.0 however produces 59.890000000000001
in dd. The management somehow feels thats a more "accurate"
representation. I guess after rounding off to 2 decimal places they
expected 59.89 but a bug somewhere in the code ensured that the former
case got rounded off to 59.88. It was off by a penny and that
triggered major problems in other calculations.

Could you please post which compiler you are using so that we can avoid
it?

If the situation is as described, then I would be rather sure that your
compiler is broken, and I wouldn't go anywhere near it. If you rely on
that strange behavior of the compiler, you are skating on very thin
ice. 59.889999389648437 looks like a bona fide 32 bit floating point
number. The C Standard requires one hundred percent that f is rounded
to single precision, the multiplication by 1000.0 and division by
1000.0 are one hundred percent required to be done in double precision,
which means that getting a result of 59.890000000000001 is absolutely
impossible unless the compiler is broken.

While this particular bug worked in your favor, you cannot possibly
rely on this compiler. If it has a bug like this, you don't know what
other bugs it has. Switch compilers. Maybe switch to Java, using a Sun
certified implementation, and turn "strict floating point arithmetic"
on; that way you can rely on your results (that means if anything is
wrong, it is your code that is wrong and not the compiler).

If getting the correct results is important to you, then I recommend
hiring someone who knows how arithmetic in C works. This is not
something that you will learn on Usenet.

 

[christian.bau]

Once again I cannot convince my boss to remove that nasty
mutiply/divide by 1000.0 kludge since he seems to believe that solves
whatever problem they have been having. Please read my very first post
to understand why I say this -- this kludge rounded off the number to
the nearest cent higher than the floating point representation and he
seems to be happy with that. (i.e what started off with
59.889999389648437 caused problems and what made him happy after the
kludge was the fact the same number got approximated as
59.890000000000001).

I think your compiler problem is not the biggest problem that you have.
If your boss was a programmer where I work, he would be fired.

int main()
{
float f = 59.89F;
double d = ff;

double d_fractional, d_integral, d_floorvalue;
d_fractional = d_integral = d_floorvalue = 0;
d_fractional = modf(d, &d_integral);
d_floorvalue = floor(d_fractional*100.0 + 0.5) / 100.0;

double result = d_integral + d_floorvalue;

return 0;
}

It will give the right result for positive values, but is overly
complicated. On the other hand, according to the information that you
have given us, your compiler cannot be trusted, so both your code and
my code below cannot be guaranteed to work.

double round_to_hundredths (double x)
{
if (x >= 0) return floor (100,0 * x + 0.5) / 100.0;
else return ceil (100.0 * x - 0.5) / 100.0;
}

 

[christian.bau]

If the floats you're getting are precise to one cent, you can recover
their exact values with

dollars = ( int ) f;
cents = ( int )( f * 100 + 0.5 ) % 100;

Before you do this in real life, try what happens when f = 19.999.

 

[Ernie Wright]

Before you do this in real life, try what happens when f = 19.999.

Please note the necessary condition: IF the floats are precise to one
cent, meaning they are the float representation of a number with two
decimal places. The above works with both 19.99 and 20.00.

- Ernie http://home.comcast.net/~erniew