D
Dale Reed
We're having a lot of problems getting visual studio to calculate even the
most basic arithmetic correctly.
For example:
Dim MyValue AS Double = 2.155 * 100
This should give me 215.5 right? Wrong. It gives me 215.499999999999999997
Change it to 1.155, it works. 3.155 it works. 2.155 it doesn't work.
We've found several values that don't seem to work correctly. I'm staggered
at this, we're trying to develop accounts based systems and yet visual
studio can't even multiply an item by 100 correctly?! It's not like I'm
doing anything complicated!
Has anyone else come across this and is there some kind of hot fix or patch
that can be applied to sort this out?
We've already been having problems with the round function in .net 1.1 and
had to write our own. The above problem occurs in 1.1 and 2.
Thanks in advance...
most basic arithmetic correctly.
For example:
Dim MyValue AS Double = 2.155 * 100
This should give me 215.5 right? Wrong. It gives me 215.499999999999999997
Change it to 1.155, it works. 3.155 it works. 2.155 it doesn't work.
We've found several values that don't seem to work correctly. I'm staggered
at this, we're trying to develop accounts based systems and yet visual
studio can't even multiply an item by 100 correctly?! It's not like I'm
doing anything complicated!
Has anyone else come across this and is there some kind of hot fix or patch
that can be applied to sort this out?
We've already been having problems with the round function in .net 1.1 and
had to write our own. The above problem occurs in 1.1 and 2.
Thanks in advance...