1813 - Javascript has Pentium-style precision errors

Status: VERIFIED DUPLICATE of bug 5856

(Core :: JavaScript Engine, defect, P2)

Description

[Akkana Peck]

(This is probably a dup of a known bug ...)
25.4 * 3 comes out 76.1999999... instead of 76.2.  See URL for demo.
Happens on all versions of our browser up to and including 5.0, and on all
platforms we've tried so far.

Comment 1

[Mike McCabe]

Clayton, our IEEE wiz, reports:

Sigh.  Perhaps we really do need decimal or rational arithmetic.  Surely this is
the same old story.  4/10 == 2/5 isn't exactly representable in binary.
Conversion to binary and subsequent multiplication by 3 (probably) creates a
binary bit pattern whose nearest decimal representation (to 16 digits) is
1.199999999.....   Even the I18N package might fix this since rounding to fewer
digits would give the user what she expects.  It's not a universal panacea, but
delivering fewer digits would stop 99.999....% of these complaints.

= C =


The basic problem is that you're seeing machine arithmetic.  The problem isn't
Pentium-style, it's universal to binary representations of numbers.  (Which
javascript currently uses.)  We're dreaming about a decimal-arithmetic package,
but it's not currently scheduled.

Comment 2

[leger]

Javacsript component begin retired.  Moving this bug to Javascript Engine.

Comment 3

[leger]

Marking Verified/Invalid

Comment 4

[Jay Patel [:jay]]

Verified Invalid

Comment 5

[Jesse Ruderman]

*** This bug has been marked as a duplicate of 5856 ***

Comment 6

[Travis Chase]

Verifying