190303 - Javascript floating point calculation

Status: VERIFIED DUPLICATE of bug 5856

(Core :: JavaScript Engine, defect)

Description

[Tom Suchanek]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2.1) Gecko/20021130
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2.1) Gecko/20021130

If Javascript have to calculate 8.2*100 it fails (returns 819.9999999999999). It
is independent if it is a script or entered from console. I suppose there is
another numbers what makes bad results.

It is strange that same bug produce NN 4.76 and MSIE 6.

Reproducible: Always

Steps to Reproduce:
1. Run Javascript console.
2. Enter command 8.2*100


Actual Results:  
819.9999999999999

Expected Results:  
820

Comment 1

[Hermann Schwab]

tested console with 4.2, 5.2, 6.2 * 100 all ok
4.2 * 100 gives 420
4.1 * 100 gives 409.99999999999994
2.04* 100 gives 204
2.05* 100 gives 204.99999999999997
2.05* 400 gives 819.9999999999999
2.05* 401 gives 822.05 ok again
2.06* 100 gives 206

Comment 2

[Phil Schwartau]

Tom, thank you for this report. This issue comes up a lot, but
I'm afraid we have to mark this invalid.

Computers do arithmetic in base 2, not base 10. Certain numbers which
have finite decimal expansions in base 10 have infinite decimal expansions
in base 2. But the computer must round off infinite expansions to finite 
ones. That produces the results above.

The same thing occurs in any that language adheres to the ANSI/IEEE
standard 754 for floating-point arithmetic, which requires this behavior.
That's why you get the same results in Mozilla, NN 4.76 and MSIE 6.
You will also get the same results in C or Java.

See the explanations in these bugs, all marked INVALID:

bug 20140   "Rounding Error?"
bug 42134   "JavaScript should be more lenient comparing (==) floats"
bug 129300  "Error in javascript internal routine : parseFloat"
bug 154103  "Arithmetic error in JavaScript Engine"
bug 160173  "Error de punto flotante al sumar en JavaScript"
bug 186563  "Number.toFixed: loss of precision"


Good web references:

http://developer.netscape.com/docs/manuals/js/core/jsref15/number.html#1200964
(see the "Description" section)

http://www.merlyn.demon.co.uk/js-maths.htm

Comment 3

[Phil Schwartau]

Marking Verified -

Comment 4

[Brendan Eich [:brendan]]

*** Bug 280544 has been marked as a duplicate of this bug. ***