Closed Bug 480475 Opened 15 years ago Closed 15 years ago

JavaScript Math is producing incorrect results with floating point differences

Categories

(Core :: JavaScript Engine, defect)

x86
All
defect
Not set
normal

Tracking

()

VERIFIED DUPLICATE of bug 5856

People

(Reporter: cmtalbert, Unassigned)

Details

So, I hope you can tell me this bug is already filed or expected in some odd way.  It seems really broken to me, and I couldn't find anything in bugzilla that is still open and would explain it.  

Here's the issue (performed with build pulled & built today, 2/26 at 15:00 or so PST):
First, the original problem:
js> Math.abs(3.73205-3.73206);
0.000010000000000065512
Should be: 0.00001

Then simplifying:
js> Math.abs(1.001-1.002);
0.001000000000000112
js> Math.abs(1.01-1.02);
0.010000000000000009
js> Math.abs(1.1-1.2);
0.09999999999999987

Without abs():
js> 1.1-1.2
-0.09999999999999987
js> 1.2-1.1
0.09999999999999987
js> 

Expected: the math should be correct.  There really shouldn't need to be any rounding errors with something as simple as 1.2-1.1, but I don't know how you've implemented this.

Marking blocking ? as the ability to do simple math seems pretty critical.
Flags: in-testsuite?
Flags: blocking1.9.2?
Dammit, forgot to mention: tried this on Windows, Mac OS X and Linux with mozcentral, and got similar results across the board.  The results from the JS shell are on an intel macBook running OS X 10.5.
Another friend falls to ieee-binary!
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
Verified my friend!
Status: RESOLVED → VERIFIED
Flags: in-testsuite?
Flags: blocking1.9.2?
You need to log in before you can comment on or make changes to this bug.