floating-point precision

[Enrique Cruiz]

Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111


I am not quite sure why that is the case, although I believe float do
not have enough precision, so the compiler (VC 8) gives the closest
approximation that can be had with float. Is that so? Can somebody
enlighten me on that issue?

Thanks in advance,


Enrique

 

[Ben Pfaff]

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

float is only guaranteed to have 6 decimal digits worth of
accuracy. If you need more, use double, which has at least 10
decimal digits worth of accuracy. Also, read what the C FAQ has
to say about floating point numbers.

 

[CBFalconer]

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are
ok, but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ..........] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float
do not have enough precision, so the compiler (VC 8) gives the
closest approximation that can be had with float. Is that so? Can
somebody enlighten me on that issue?

Look in <float.h> for the number of digits available. Consider
using double rather than float.


mm

 

[Keith Thompson]

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

What the heck is a "LUT", and what language are you using? ("LookUp
Table", maybe?) The above is a syntax error in C, even if you replace
the "................" with something meaningful. It's probably some
compiler-specific extension.

Aside from that, type float just doesn't provide the precision you
need. Type double or long double might. Consult your system's
documentation.

 

[William Hughes]

Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float do
not have enough precision,

Why? Float is only guaranteed to have 6 digits of precision, which
is exactly what you are seeing.

If you need more than 6 digits of precision, then, use double
(And unless you have a reason to use float, use double).

Why do you think you need this precision? (Are you trying
to check the results using the LUT against those not
using a LUT?)

Have you checked to see if using the LUT actually does
increase execution speed? (It is quite possible for
the opposite to happen). Have you checked whether
the putative incease in speed is significant?

- William Hughes

 

[CBFalconer]

.... snip ...

Have you checked to see if using the LUT actually does increase
execution speed? (It is quite possible for the opposite to
happen). Have you checked whether the putative incease in speed
is significant?

What is a LUT? A capitalized abbreviation for a lute, perhaps. If
so, who is playing it.

 

[Ben Pfaff]

What is a LUT?

Look-Up Table.