E
Ed Howland
Hello,
I am trying to get the Division example on the FIT wiki=20
http://fit.c2.com/wiki.cgi?WhatAndHowTalk
to run under Ruby and the Fit framework. I have 2 questions. First I
get red results on last 3 tests because the precision in the HTML
table is only up to 8 digits, but Ruby is calculating 15 digits.
And second, even after I adjust the HTML to match the results, they
till don't equal. I am thinking this is because of '=3D=3D' vs. eql? on
floats.
Is there a way coerce a Ruby float to an aribitray precision via
truncate or round?
Thanks
I am trying to get the Division example on the FIT wiki=20
http://fit.c2.com/wiki.cgi?WhatAndHowTalk
to run under Ruby and the Fit framework. I have 2 questions. First I
get red results on last 3 tests because the precision in the HTML
table is only up to 8 digits, but Ruby is calculating 15 digits.
And second, even after I adjust the HTML to match the results, they
till don't equal. I am thinking this is because of '=3D=3D' vs. eql? on
floats.
Is there a way coerce a Ruby float to an aribitray precision via
truncate or round?
Thanks