Closed
Bug 480475
Opened 15 years ago
Closed 15 years ago
JavaScript Math is producing incorrect results with floating point differences
Categories
(Core :: JavaScript Engine, defect)
Tracking
()
People
(Reporter: cmtalbert, Unassigned)
Details
So, I hope you can tell me this bug is already filed or expected in some odd way. It seems really broken to me, and I couldn't find anything in bugzilla that is still open and would explain it. Here's the issue (performed with build pulled & built today, 2/26 at 15:00 or so PST): First, the original problem: js> Math.abs(3.73205-3.73206); 0.000010000000000065512 Should be: 0.00001 Then simplifying: js> Math.abs(1.001-1.002); 0.001000000000000112 js> Math.abs(1.01-1.02); 0.010000000000000009 js> Math.abs(1.1-1.2); 0.09999999999999987 Without abs(): js> 1.1-1.2 -0.09999999999999987 js> 1.2-1.1 0.09999999999999987 js> Expected: the math should be correct. There really shouldn't need to be any rounding errors with something as simple as 1.2-1.1, but I don't know how you've implemented this. Marking blocking ? as the ability to do simple math seems pretty critical.
Flags: in-testsuite?
Flags: blocking1.9.2?
Dammit, forgot to mention: tried this on Windows, Mac OS X and Linux with mozcentral, and got similar results across the board. The results from the JS shell are on an intel macBook running OS X 10.5.
Another friend falls to ieee-binary!
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
Updated•15 years ago
|
Flags: in-testsuite?
Flags: blocking1.9.2?
You need to log in
before you can comment on or make changes to this bug.
Description
•