Open
Bug 669246
Opened 10 years ago
Updated 5 years ago
toPrecision rounds ties to even, not away from zero
Categories
(Core :: JavaScript Engine, defect)
Core
JavaScript Engine
Tracking
()
NEW
People
(Reporter: bzbarsky, Unassigned)
References
(Blocks 1 open bug, )
Details
(Whiteboard: jstriageneeded)
(125).toPrecision(2) returns 120 whereas per spec it should be 130 (15.7.4.7 step 10.a says "If there are two such sets of e and n, pick the e and n for which n * 10^{e–p+1} is larger" where 'n' is in this case would be either 12 or 13). See also http://stackoverflow.com/questions/6567690/firefoxandjavascriptroundingrules : apparently everyone else gets this right.
Comment 1•10 years ago


I filed a bug in the test262 bugzilla for this, so hopefully someone will write the test and add it.
See Also: → https://bugs.ecmascript.org/show_bug.cgi?id=122
Updated•10 years ago

Whiteboard: jstriageneeded
Comment 2•8 years ago


This problem also occurs for floating point numbers, e.g. 123.445 .toPrecision(5) //> "123.44" V8 also has this issue with toPrecision for floating point numbers, however, whereas it doesn't have the original issue as it is posted. I don't think that it warrants a separate issue, though, and I realise there may be some argument against fixing it (e.g. the standard floating point arithmetic problem), but the specification requires the number to be rounded correctly.
Assignee  
Updated•7 years ago

Assignee: general → nobody
Comment 3•7 years ago


(In reply to Andy Earnshaw from comment #2) > This problem also occurs for floating point numbers, e.g. > > 123.445 .toPrecision(5) > //> "123.44" JavaScript uses radix 2 internally for the Number object, and 123.445 is not exactly representable. So, it will first be rounded (to some number that is presumably below 123.445). Here you are not in a "tie" case, and the result just depends on the rounding of the decimal number to a binary doubleprecision number (53 bits). Thus on this example, this is not a bug.
Comment 4•6 years ago


(In reply to Vincent Lefevre from comment #3) > Thus on this example, this is not a bug. I'd argue that it is a bug, since it's offspec. It might not be a tie from the point of view of the internal representation of the number, but it is still a tie and produces an unexpected result. Also, IE (out of all the mainstream browsers) has managed to produce the correct result since IE 6 and possibly older, indicating that it is absolutely achievable.
Comment 5•6 years ago


The spec http://www.ecmainternational.org/ecma262/5.1/ seems to be clear to me: * Section 15.7.4.7 "Number.prototype.toPrecision (precision)" says that toPrecision is applied on a Number value ("Return a String containing this Number value..."). * Section 4.3.19 "Number value" says "primitive value corresponding to a doubleprecision 64bit binary format IEEE 754 value". Here 123.445 is a numeric literal: Section 7.8.3 "Numeric Literals", which says "A numeric literal stands for a value of the Number type. This value is determined in two steps: first, a mathematical value (MV) is derived from the literal; second, this mathematical value is rounded as described below." The mathematical value of 123.445 is 123.445. Concerning the rounding, since there are no more than 20 significant digits, it is "the Number value for the MV (as specified in 8.5)". * Section 8.5 "The Number Type" says that "the Number value for x" corresponds to the IEEE 754 "round to nearest" mode (the paragraph in the spec gives details on what it is). Here, the rounding to nearest of 123.445 is a value below 123.445; this can be seen with the following C program (run on a conforming IEEE 754 implementation with double = IEEE 754 double precision): #include <stdio.h> int main(void) { double x = 123.445; printf ("%a %.20g\n", x, x); return 0; } which outputs: 0x1.edc7ae147ae14p+6 123.44499999999999318 (Here you have the exact binary value of the doubleprecision number in hex form, followed by a corresponding decimal approximation on 20 digits.) The toPrecision method is applied to this doubleprecision number, which is a bit below 123.445, so that on 5 digits, it shall give "123.44".
Comment 6•6 years ago


(In reply to Vincent Lefevre from comment #5) > The spec http://www.ecmainternational.org/ecma262/5.1/ seems to be clear > to me: No argument there. However, my point is that the internal representation of a number is irrelevant. Consider Number.prototype.toString(): it doesn't return "123.444999999999993179", which is the internal representation of the number, for 123.445  it returns "123.445". What is relevant is the part of the specification for `Number.prototype.toPrecision()`.
Comment 7•6 years ago


(In reply to Andy Earnshaw from comment #6) > However, my point is that the internal representation of a number is irrelevant. The internal representation is irrelevant, but the *value set* is what must be considered. I repeat: Section 4.3.19 "Number value" says "primitive value corresponding to a doubleprecision 64bit binary format IEEE 754 value". See https://en.wikipedia.org/wiki/Doubleprecision_floatingpoint_format about this format. > Consider Number.prototype.toString(): it doesn't return "123.444999999999993179", > which is the internal representation of the number, for 123.445  it returns "123.445". No, 123.444999999999993179 isn't the internal representation. As I've said, it's just a 20digit approximation (sufficient to show that the Number value is strictly less than 123.445). Moreover toString() doesn't return the exact value of the Number object. Note that 123.445 is mathematically equal to the irreducible fraction 24689/200, and since 200 is not a power of 2, it cannot be an exact doubleprecision number. I'm wondering what MSIE is doing. Could you try (123.445).toPrecision(21)? Firefox under GNU/Linux (x86_64) gives 123.444999999999993179 as expected.
Comment 8•6 years ago


(In reply to Vincent Lefevre from comment #7) > (In reply to Andy Earnshaw from comment #6) > > However, my point is that the internal representation of a number is irrelevant. > > The internal representation is irrelevant, but the *value set* is what must > be considered. I repeat: > > Section 4.3.19 "Number value" says "primitive value corresponding to a > doubleprecision 64bit binary format IEEE 754 value". Yes, I understand that, but if the steps for Number.prototype.toPrecision() are followed to the letter, is the expected output still "123.44"? I'm just a lowly front end developer, so my C++ isn't really good enough to know what's going on here (I think the code Firefox uses is at http://mxr.mozilla.org/commcentral/source/mozilla/js/src/jsdtoa.cpp#75). > I'm wondering what MSIE is doing. Could you try (123.445).toPrecision(21)? > Firefox under GNU/Linux (x86_64) gives 123.444999999999993179 as expected. The output for IE 11 is "123.445000000000000000", didn't test older versions as I was cheekily using the company's Sauce Labs time to check the result and don't want to push my luck ;).
Reporter  
Comment 9•6 years ago


> but if the steps for Number.prototype.toPrecision() are followed to the letter, is the > expected output still "123.44"? Yes. Given that x has the exact decimal value 123.44499999999999317878973670303821563720703125 (which is the exact IEEE double value closest to 123.445), in step 12 of toPrecision(5) the only valid choices for n and e are 12344 and 2. > The output for IE 11 is "123.445000000000000000" Then IE11 is buggy here, as far as I can tell. Every other JS engine I have on hand (V8, SpiderMonkey, JavaScriptCore, and Carakan) has (124.445).toPrecision(5) == 123.44 and (124.445).toPrecision(21) == 123.444999999999993179, as the spec requires. Note that it's really easy to land here with a naive implementation of toPrecision that tries multiplying the given number by 10^n for various n and comparing to integers in the range (10^(p1), 10^p) or something instead of doing what the spec actually says. You can somewhat tell that IE has a buggy toPrecision by doing this test (though you do have to be a bit careful, since none of those three numbers is exactly representable as a Number in JavaScript, so they _all_ have some rounding issues): console.log(123.445  123.44); console.log(124.45  123.445); In IE, as in every other browser, this shows: 0.0049999999999954525 0.005000000000009663 Now can we stop hijacking this bug, please? There are now 10 comments here, of which only two (and the shortest ones at that) have to do with the bug I actually filed and that needs fixing.
Comment 10•6 years ago


To come back to the original bug, note that it seems to occur only on integers, such as 125, not on numbers like 10.5, 11.5, 12.5, etc. (which are exactly representable).
Reporter  
Comment 11•6 years ago


Ah, good catch. That suggests some sort of fastpath being taken in the integer case or something that doesn't do the right thing.
Comment 12•6 years ago


toPrecision is a method that supposedly applies to decimals, and so should at least be consistent on decimals: However, for instance .00345.toPrecision(2) returns 0.0034 whereas .0345.toPrecision(2) returns 0.035 This looks buggy to me.
Reporter  
Comment 13•6 years ago


> toPrecision is a method that supposedly applies to decimals It applies to decimal representations of ES Number instances. > However, for instance Did you read comment 9? > .00345.toPrecision(2) returns 0.0034 When you type ".00345", the Number that results has the exact decimal representation 0.00344999999999999994171329120717928162775933742523193359375. > .0345.toPrecision(2) returns 0.035 When you type ".0345", the Number that results has the exact decimal representation 0.0345000000000000028865798640254070051014423370361328125. > This looks buggy to me. Please do read up on how IEEE doubleprecision floating point numbers work... I agree it's nonintuitive if you want to think of them as "real numbers", but that's because they're not "real numbers" in any meaningful sense of the word.
Comment 14•6 years ago


Yes I did read point #9 but continue to disagree with the philosophy that a method supposedly about precision of *decimal places* does not work with decimal strings. Note that (345/100000).toExponential() returns 3.45e3 as expected (it does not return "3.44999999999999994171329120717928162775933742523193359375e3 as the philosophy in #9 appears to suggest it should). Evidently, it gets represented in a rounded form somehow. So, then, why should (345/100000).toPrecision(2) not return the expected 0.0035 as well? Anyway, rather than get into a philosophical argument about decimal manipulation, I have simply written my own simple JS routine that does what one expects using string manipulation of exponential form and the fat that toExponential() does what is it supposed to do.
Reporter  
Comment 15•6 years ago


It's not a matter of philosophy. You should try (345/100000).toExponential(20) and (345/10000).toExponential(20). When toExponential is called with no arguments, I believe our behavior is actually buggy per spec, unless I'm totally misreading the spec. Per spec at http://people.mozilla.org/~jorendorff/es6draft.html#secnumber.prototype.toexponential the subtraction n*10^{ef}  x in step 12 substep b should be happening exactly on arbitraryprecision representations, but we're presumably doing it on doubleprecision floating point values. As a result, the "f is as small as possible" criterion leads us to pick the smallest f for which the difference is 0, and since we're doing the subtraction in limited precision the difference is 0 for n == 345, f = 2, e = 3. But again, that's a bug due to the limited precision in the subtraction.
Comment 16•5 years ago


This is a regression caused by http://hg.mozilla.org/mozillacentral/rev/222c29336422 (bug 384244), readding the biasUp parameter will fix this issue. I don't know if adding the biasUp parameter will introduce other bugs, though.
You need to log in
before you can comment on or make changes to this bug.
Description
•