ruby to_i problem

[Phil Cooper-king]

I'm having an issue with a float being converted to an int,

The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.

For what ever reason the first instance gives an error, and every
instance there after works as it should

This cant be right?


ruby 1.8.7
[irb]
# This works the way I would think=> 1


# This doesn't=> 1.0


# this is where its werid=> 0
[/irb]

any further 1.0.to_i works

 

[Marvin Gülker]

any further 1.0.to_i works

You're experiencing IEEE Floating point aritmetic. Look at this example:
---------------------------------------
ruby -v: ruby 1.9.1p378 (2010-01-10 revision 26273) [x86_64-linux]
irb(main):001:0> x = -1
=> -1
irb(main):002:0> y = -1
=> -1
irb(main):003:0> angle = 90 * Math:I / 180
=> 1.5707963267949
irb(main):004:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):005:0> sprintf("%.50f", n_x)
=> "0.99999999999999988897769753748434595763683319091797"
irb(main):006:0> n_x.to_i
=> 0
irb(main):007:0> n_x.round
=> 1
irb(main):008:0> sprintf("%.50f", 1.0)
=> "1.00000000000000000000000000000000000000000000000000"
irb(main):009:0>

 

[Ben Bleything]

For what ever reason the first instance gives an error, and every
instance there after works as it should

What You Think Is Happening Is Not What Is Actually Happening

ruby 1.8.7
[irb]
# This works the way I would think=> 1

Yep, and every time you say 1.0.to_i you will get 1.

# This doesn't
=> 1.0

irb is showing you an approximation. It's confusing because irb is
showing you n_x to be 1.0, when in fact:
=> "0.9999999999999998889776975"

# this is where its werid
=> 0
Yep:

=> 0

any further 1.0.to_i works

1.0.to_i and n_x.to_i (from your example) are two different things.

Search the group archives for "floating point", you're sure to find a
huge pile of other people with similar problems, and likely good
solutions.

Ben

 

[Jesús Gabriel y Galán]

I'm having an issue with a float being converted to an int,

The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.

For what ever reason the first instance gives an error, and every
instance there after works as it should

This cant be right?


ruby 1.8.7
[irb]
# This works the way I would think=> 1


# This doesn't=> 1.0


# this is where its werid=> 0
[/irb]

any further 1.0.to_i works

You are facing a problem about the inaccuracy inherent to floating
point numbers:


irb(main):001:0> 0.9999.to_i
=> 0
irb(main):002:0> x = -1
=> -1
irb(main):003:0> y = -1
=> -1
irb(main):004:0> angle = 90 * Math:I / 180
=> 1.5707963267949
irb(main):005:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):009:0> "%.20f" % n_x
=> "0.99999999999999988898"
irb(main):010:0>
irb(main):010:0> n_x
=> 1.0

n_x is very close to 1.0 but it's not. to_i rounds the number down, so
you get 0. You can check the archives for a more thorough explanation
about this, there's been plenty of discussion in the past.

Jesus.

 

[Phil Cooper-king]

thanks guys, this makes it clear