ruby numbers

[djreid]

Hi I'm new to ruby.
Started with learning-ruby.
In first_steps it looks at using ruby as a calculator=> 1.5

all this I understand BUT
=> 0.0999999999999996

Should be .1
and wants more there has only been one decimal place used so the answer
should have only 1 decimal place.

another=> 10.3359 4 decimals


Dave

 

[Robert Klemme]

Hi I'm new to ruby.
Started with learning-ruby.
In first_steps it looks at using ruby as a calculator
=> 1.5

all this I understand BUT

=> 0.0999999999999996

Should be .1
and wants more there has only been one decimal place used so the answer
should have only 1 decimal place.

another
=> 10.3359 4 decimals


Dave

This are usual numeric errors you see because a float is not represented
as a decimal number but as a binary internally.

robert

 

[Tadashi Saito]

Hi all,

On 9 Jun 2006 02:15:22 -0700

all this I understand BUT

=> 0.0999999999999996

Should be .1

Yes, I think so, too. The shortest solution is:
=> true

 

[ChrisH]

An interesting thing I noticed is that:
0.1 % 2
=> 0.1

I would think it would have the same issue? Unless % is doing
something different
when the left value is les than the right? Have to look at the code if
I get a chance...