Precision and ==

  • Thread starter LAMBEAU Bernard
  • Start date
L

LAMBEAU Bernard

[Note: parts of this message were removed to make it a legal post.]

We all know that finite representation comes with some difficulties...
but...

x = 45.0*(Math::pI / 180.0)
y = 0.785398163397448
puts "#{x.class} : #{x}"
puts "#{y.class} : #{y}"
puts 45.0*(Math::pI / 180.0) == 0.785398163397448

prints

Float : 0.785398163397448
Float : 0.785398163397448
false

Does to_s show something less precise than the internal representation of
Floats?

thx
Bernard
 
J

Jesús Gabriel y Galán

We all know that finite representation comes with some difficulties...
but...

x = 45.0*(Math::pI / 180.0)
y = 0.785398163397448
puts "#{x.class} : #{x}"
puts "#{y.class} : #{y}"
puts 45.0*(Math::pI / 180.0) == 0.785398163397448

prints

Float : 0.785398163397448
Float : 0.785398163397448
false

Does to_s show something less precise than the internal representation of
Floats?

Yes, it rounds to 15 decimal positions:

irb(main):001:0> x = 45.0*(Math::pI / 180.0)
=> 0.785398163397448
irb(main):002:0> y = 0.785398163397448
=> 0.785398163397448
irb(main):003:0> puts "#{x.class} : #{"%.20f" % x}"
Float : 0.78539816339744827900
=> nil
irb(main):004:0> puts "#{y.class} : #{"%.20f" % y}"
Float : 0.78539816339744794593

Jesus.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,995
Messages
2,570,236
Members
46,825
Latest member
VernonQuy6

Latest Threads

Top