How to avoid Fixnum rounding error

[Jari Williamsson]

Which is the best approach to avoid division rounding errors when
performing integer division with a Fixnum? Is there any built-in method
for this?

Best regards,

Jari Williamsson

 

[M. Edward (Ed) Borasky]

Which is the best approach to avoid division rounding errors when
performing integer division with a Fixnum? Is there any built-in method
for this?

Best regards,

Jari Williamsson

I'm not sure what you're trying to accomplish here. Fixnums by
definition are integers. When you divide an integer by another integer,
you can get either a quotient, a remainder (the "modulo" operation), or
a list of both the quotient and remainder. For example, in irb:

irb(main):001:0> fixnum1 = 5
=> 5
irb(main):002:0> fixnum2 = 7
=> 7
irb(main):003:0> fixnum1/fixnum2
=> 0
irb(main):004:0> fixnum1%fixnum2
=> 5
irb(main):005:0> fixnum1.divmod fixnum2
=> [0, 5]

There are two other options. You can convert the Fixnums to Floats and
get their quotient as another Float:

irb(main):006:0> fixnum1.to_f/fixnum2.to_f
=> 0.714285714285714

Or, you can enter the world of Rational numbers and represent the
quotient of two integers as a Rational, like this:

irb(main):007:0> require 'mathn'
=> true
irb(main):008:0> fixnum1/fixnum2
=> 5/7


The tradeoffs between these last two options are (in most languages)
performance versus accuracy. The floating point version will usually be
faster, and the amount of memory required is fixed. But its accuracy is
limited to about 15 decimal places in most machines.

The rational version will be slower, and the numbers involved can take
up arbitrarily large amounts of memory as the computation progresses.
But it is "exact", in the sense that if the result you're seeking is a
rational number, that's what you'll get. If the result isn't a rational
number, you'll get a rational number that's close to it.

 

[Stefan Rusterholz]

Which is the best approach to avoid division rounding errors when
performing integer division with a Fixnum? Is there any built-in method
for this?

Best regards,

Jari Williamsson

Since nobody brought it up so far: 5.quo(2) # => 2.5, If you required
Rational it will be Rational(5,2).

Regards,
Stefan

 

[Mikel Lindsaar]

If you are talking about money and avoiding rounding errors on dollars
and cents, you can use the Money class (google it) or you can multiply
your dollars and cents by 100 to get an integer and then only deal
with whole cent values.

Multiplying by 100 is not a complete solution if you are really
tracking money and not just the price of an item which there might be
multiple of, but it is an easy solution to an old problem if all you
need to do is add tax and minus shipping without getting something
like a final price of $33.01 when it should really be $33.

Regards

Mikel
http://lindsaar.net/

 

[M. Edward (Ed) Borasky]

Since nobody brought it up so far: 5.quo(2) # => 2.5, If you required
Rational it will be Rational(5,2).

Regards,
Stefan

I don't really like the "Rational(5,2)" notation when "Mathn" will give
you "5/2".