Closed Bug 622348 Opened 9 years ago Closed 8 years ago

JavaScript Math.round incorrect for (2^53)-1

Categories

(Core :: JavaScript Engine, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla12

People

(Reporter: allen, Assigned: evilpie)

Details

(Whiteboard: [from-beta-tester])

Attachments

(1 file, 1 obsolete file)

Math.round produces incorrect result for largest representable integer value:

Math.round(9007199254740991)
   ==> returns: 9007199254740992, should be 9007199254740991
also:
Math.round(-9007199254740991)
   ==> returns: -19007199254740990, should be -9007199254740991

Note that 9007199254740991  is the largest value - 1 of the span of continuous integers that are precisely representable using IEEE doubles. It should round to itself. Correct results for values at this boundary are important for some integer numeric algorithms.

Tested with same result on Mac using FF4b9 and 3.6.13 and on Windows using 3.6.13.

Also, this does not appear to be a numeric literal conversion bug.  The problem also occurs if the value is computed (for example: 9007199254740990+1).

Finally, note that that the ECMAScript spec. provides the identity that Math.round(x) is the same as Math.floor(x+0.5).  While that is mathematically correct it isn't necessarily correct when operating at the precision limit of floating point arithmetic.  If the Math.round is implemented using that identify, it may be the source of the problem.

Finally, note that both Safari and Chrome produce the correct answer.
Assignee: nobody → general
Component: General → JavaScript Engine
Product: Firefox → Core
QA Contact: general → general
> Finally, note that that the ECMAScript spec. provides the identity that
> Math.round(x) is the same as Math.floor(x+0.5). 

That's how we seem to implement round(), yes.  Sounds like we could use a spec change here to add to the "except" bits in that note...
And in particular:

  584 js_math_round_impl(jsdouble x)
  585 {
  586     return js_copysign(floor(x + 0.5), x);
V8 checks if exponent is greater or equal 52, and in this case just returns the number.
Assignee: general → evilpies
Attached patch v1 (obsolete) — Splinter Review
Attachment #582841 - Flags: review?(bzbarsky)
Comment on attachment 582841 [details] [diff] [review]
v1

r=me
Attachment #582841 - Flags: review?(bzbarsky) → review+
http://hg.mozilla.org/integration/mozilla-inbound/rev/6f31489f62d6 Arg this is a follow up for this bug, but I failed and somehow copied Bug 582841.
http://hg.mozilla.org/integration/mozilla-inbound/rev/504f00e1124e Backed out, because of test failures and general stupidy surrounding this landing. Going to investigate, but in theory the inlined Math.round method doesn't match anymore.
Attached patch v2Splinter Review
So well, no idea how I didn't catch that :/ But the mistake is rather obvious when you think about it, I didn't really extract the exponent.
Attachment #582841 - Attachment is obsolete: true
Attachment #584291 - Flags: review?(bzbarsky)
Comment on attachment 584291 [details] [diff] [review]
v2

This should probably get review from someone competent, since it seems my assumption about how much I know about this was wrong.
Attachment #584291 - Flags: review?(bzbarsky) → review?(jwalden+bmo)
Comment on attachment 584291 [details] [diff] [review]
v2

Review of attachment 584291 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/jit-test/tests/basic/mathRoundBig.js
@@ +2,5 @@
> + * Any copyright is dedicated to the Public Domain.
> + * http://creativecommons.org/licenses/publicdomain/
> + */
> +
> +assertEq(Math.round(9007199254740991), 9007199254740991);

Add identical versions that test the string forms of these, for extra checking that the right number gets through?  Like so:

  assertEq(Math.round("9007199254740991"), 9007199254740991);

I'd prefer if this were in js/src/tests/ecma_5/Math/round-large-number.js, but this is already written, so you don't need to change the location.

::: js/src/jsmath.cpp
@@ +580,1 @@
>  JSBool

Add a /* ES5 15.8.2.15. */ comment

@@ +588,4 @@
>      }
> +
> +    double x;
> +    if (!ToNumber(cx, args[0], &x))

It might be worth checking for integers and returning early in that case, maybe?

@@ +594,5 @@
> +    jsdpun u;
> +    u.d = x;
> +    int exponent = ((u.s.hi & JSDOUBLE_HI32_EXPMASK) >> JSDOUBLE_HI32_EXPSHIFT) - JSDOUBLE_EXPBIAS;
> +
> +    /* Some numbers are so big that adding 0.5 would give the wrong number */

Move the |int exponent = ...| below the comment, adjacent to the |if| below?
Attachment #584291 - Flags: review?(jwalden+bmo) → review+
https://hg.mozilla.org/mozilla-central/rev/37348c68ba0b
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla12
You need to log in before you can comment on or make changes to this bug.