BigDecimal to float error

S

Sam Kong

Hi,

I found a weird problem.

In ruby 1.8.5, I get the following result in irb.

BigDecimal.new("86.02").to_f #=> 86.2

I don't know if this is a ruby bug or machine bug.
In ruby 1.8.6 on a different machine, it's ok.

Is this a known issue?

Sam
 
S

Sam Kong

Hi,

I found a weird problem.

In ruby 1.8.5, I get the following result in irb.

BigDecimal.new("86.02").to_f #=> 86.2

I don't know if this is a ruby bug or machine bug.
In ruby 1.8.6 on a different machine, it's ok.

Is this a known issue?

Sam

I narrowed down the problem.
It has nothing to do with converting to float.

irb(main):023:0> BigDecimal.new("1.0234")
=> #<BigDecimal:b7efb05c,'0.1234E1',8(12)>

The internal representation is wrong.

Will upgrading ruby fix this problem?
 
K

Kirk Haines

Hi,

I found a weird problem.

In ruby 1.8.5, I get the following result in irb.

BigDecimal.new("86.02").to_f #=> 86.2

I don't know if this is a ruby bug or machine bug.
In ruby 1.8.6 on a different machine, it's ok.

Is this a known issue?

You don't say what patch level of 1.8.5 you are on. Using 1.8.5pl114
it works fine. However, likely it is a bug where the 0s immediately
following the decimal get eaten.

Upgrade your Ruby. Don't use 1.8.5.


Kirk Haines
 
C

Christopher Dicely

I narrowed down the problem.
It has nothing to do with converting to float.

irb(main):023:0> BigDecimal.new("1.0234")
=3D> #<BigDecimal:b7efb05c,'0.1234E1',8(12)>

The internal representation is wrong.

Will upgrading ruby fix this problem?


I suspect so:
christopher@ubuntu:~$ ruby -v -rbigdecimal -e
'd=3DBigDecimal.new("1.0234"); p d; p d.to_f'
ruby 1.8.6 (2009-08-04 patchlevel 383) [x86_64-linux]
#<BigDecimal:7ff22e270380,'0.10234E1',18(18)>
1.0234
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

Forum statistics

Threads
474,167
Messages
2,570,910
Members
47,453
Latest member
MadelinePh

Latest Threads

Top