Bug 5856 (js-ieee-double-hurts)

javascript rounding bug

VERIFIED INVALID

Status

()

Core
JavaScript Engine
P3
normal
VERIFIED INVALID
18 years ago
2 years ago

People

(Reporter: AriB, Assigned: Norris Boyd)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(URL)

(Reporter)

Description

18 years ago
Just to give one example
14.28 x 9 should be 128.52;
your browser's answer = 128.51999999999998

This bug is also from Netscape 4.5

Updated

18 years ago
Status: NEW → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → INVALID

Comment 1

18 years ago
ECMAScript requires the use of IEEE double precision for numbers.

14.28 correctly rounded as an IEEE double is represented as

1.1100100011110101110000101000111101011100001010001111e3 in base 2, which is

exactly equal to 14.2799999999999993605115378159098327159881591796875 decimal.

Multiplying this value by 9 yields exactly

128.5199999999999942446038403431884944438934326171875.

Per the ECMA spec, this is again rounded to the nearest IEEE double, which is

1.0000000100001010001111010111000010100011110101110000e7 base 2, or

128.51999999999998181010596454143524169921875 decimal.



Why does the result print as 128.51999999999998?  Well, the ECMA spec requires

numbers to be printed with sufficient precision (and no more than that precision)

so that if the printed string were read in again and converted back to a number,

it would round to exactly the same IEEE double that was printed.  Printing 128.52

would be incorrect because the rounding 128.52 to the closest IEEE double yields

128.520000000000010231815394945442676544189453125, which is closer than

128.51999999999998181010596454143524169921875.  Thus we must print

128.51999999999998.

Every other language that uses standard IEEE double-precision arithmetic will

also give this result.  This includes Java and most implementations of C and C++.

Comment 2

18 years ago
Changing component to "Javascript Engine".  "Javascript" component is being
retired.

Updated

18 years ago
Status: RESOLVED → VERIFIED

Comment 3

18 years ago
Verified Invalid

Comment 4

13 years ago
*** Bug 251585 has been marked as a duplicate of this bug. ***
Note that ECMA-262 Edition 3 added Number.prototype.toFixed, which takes a
precision argument telling how many digits after the decimal point to show.  Use
this method well and you won't mind the disparity between finite precision base
2 and the "arbitrary" or "appropriate" precision base 10 that we use every day.

/be
*** Bug 281659 has been marked as a duplicate of this bug. ***

Comment 7

12 years ago
*** Bug 20140 has been marked as a duplicate of this bug. ***

Comment 8

12 years ago
*** Bug 1813 has been marked as a duplicate of this bug. ***
*** Bug 283303 has been marked as a duplicate of this bug. ***
*** Bug 297361 has been marked as a duplicate of this bug. ***

Comment 11

12 years ago
*** Bug 320093 has been marked as a duplicate of this bug. ***

Comment 12

11 years ago
*** Bug 356566 has been marked as a duplicate of this bug. ***

Updated

10 years ago
Duplicate of this bug: 378956
Duplicate of this bug: 393209

Updated

10 years ago
Alias: js-ieee-double-hurts

Updated

10 years ago
Duplicate of this bug: 408202

Updated

10 years ago
Duplicate of this bug: 369803

Updated

10 years ago
Duplicate of this bug: 396764
Duplicate of this bug: 409726
Duplicate of this bug: 413965

Updated

9 years ago
Duplicate of this bug: 444938
Duplicate of this bug: 452198

Updated

9 years ago
Duplicate of this bug: 460943

Updated

9 years ago
Duplicate of this bug: 463163
Duplicate of this bug: 466822

Updated

9 years ago
OS: Windows 98 → All
Hardware: PC → All
Duplicate of this bug: 480475

Updated

8 years ago
Duplicate of this bug: 516355

Updated

8 years ago
Duplicate of this bug: 520754

Updated

8 years ago
Duplicate of this bug: 190303

Updated

8 years ago
Duplicate of this bug: 280544
Duplicate of this bug: 538209
Duplicate of this bug: 557530

Updated

7 years ago
Duplicate of this bug: 582086

Updated

7 years ago
Duplicate of this bug: 618839
Duplicate of this bug: 624189

Updated

6 years ago
Duplicate of this bug: 656887
Duplicate of this bug: 699645
Duplicate of this bug: 832727
Duplicate of this bug: 839324

Comment 39

4 years ago
Duplicate of the Bug: 15* 1.33 (instead of expected 19.95 it return 19.950000000000003)

Updated

2 years ago
Duplicate of this bug: 1118287
You need to log in before you can comment on or make changes to this bug.