Status
()
People
(Reporter: apeller, Unassigned)
Tracking
Firefox Tracking Flags
(Not tracked)
Details
I see this in 3.0.7 on Windows and Mac, also 3.1b2 and Minefield.
Comment 1•10 years ago


Not a valid bug as this behavior is required by the spec. See the bug I dupe this bug to for more details.
Status: NEW → RESOLVED
Last Resolved: 10 years ago
Resolution:  → DUPLICATE
Duplicate of bug: 186563
Comment 2•10 years ago


Nope, this is a bug. According to spec the result should be "2.0" if we have, in infiniteprecision math: 20 / 10  IEEEdouble(1.95) >= 19 / 10  IEEEdouble(1.95) where IEEEdouble() is the exact representation of the argument as a binary fraction. On a test program gcc sez: (gdb) pt d type = union { double d; uint64_t u; } (gdb) p d $1 = { d = 1.95, u = 4611460838446019379 } (gdb) p/x d.u $2 = 0x3fff333333333333 We have 1 sign bit, 11 exponent bits, and 52 significand bits. Decoding the above we have 1023 as exponent, so the value is the hex fraction 1.f333333333333. Expanding that into a binary fraction yields: = 1 (implicit) + (1/2+1/4+1/8+1/6) 0xf + (1/128+1/256) 0x3 + (1/2048+1/4096) 0x3 ... 0x333333333 + (3/256/(2**44)) 0x3 The first value is: = 2  IEEEdouble(1.95) = 2  (1 + 15/16 + 3/256*sum(i from 0 to 12 exclusive, 2**(4*i))) = 1  15/16  3/256*sum(i from 0 to 12 exclusive, 2**(4*i)) = 1/16  3/256*sum(i from 0 to 12 exclusive, 2**(4*i)) The second value is: = IEEEdouble(1.95)  19/10 = 1 + 15/16 + 3/256*sum(i from 0 to 12 exclusive, 2**(4*i))  19/10 = 31/16 + 3/256*sum(i from 0 to 12 exclusive, 2**(4*i))  19/10 For n and d in reduced form where n/d==sum(i from 0 to 12 exclusive, 2**(4*i)) we have: n=18764998447377 d=17592186044416 Then: 1/16  3/256*n/d ? 31/16 + 3/256*n/d  19/10 19/10  3/256*n/d ? 15/8 + 3/256*n/d 19/10  15/8 ? 3/128*n/d 1/40 ? 3/128*n/d 1/10 ? 3/32*n/d 32/10 ? 3*n/d 32*d ? 30*n Which leaves us with: 562949953421312 ? 562949953421310 Comparing those vertically: 562949953421312 > 562949953421310 So therefore: 20 / 10  IEEEdouble(1.95) > 19 / 10  IEEEdouble(1.95) and 2.0 is the correct answer. You'd better all have had as much fun reading this comment as I had running the calculations necessary to reproduce it. :P
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → 
> so the value is the hex fraction 1.f333333333333. With you up to here. However the exact value of 1.95 in hex would have the 3 infinitely repeated; this is a truncation of that. So this value is strictly less than 1.95. > 20 / 10  IEEEdouble(1.95) > 19 / 10  IEEEdouble(1.95) Agreed on that too. Going back, though: > According to spec the result should be "2.0" if we have, in infinite > precision math: > 20 / 10  IEEEdouble(1.95) >= 19 / 10  IEEEdouble(1.95) That's wrong. The inequality you give says that 1.95 is further from 2.0 than from 1.9; if that's the case the result should be 1.9. The exact spec quote: Let n be an integer for which the exact mathematical value of n/(10^f)  x is as close to zero as possible. If there are two such n, pick the larger n. Where x is IEEEdouble(1.95) and f == 1 in this case. So we want n/10 to be as close to our number as possible. The calculation above shows that 19/10x is smaller than 20/10x, so the correct value of n is 19, which gives 1.9 as the output.
Status: REOPENED → RESOLVED
Last Resolved: 10 years ago → 10 years ago
Resolution:  → DUPLICATE
Duplicate of bug: 186563
(Reporter)  
Comment 4•10 years ago


I suspected it might be a rounding problem also, but was thrown off by the fact that IE and Safari both come up with the "right" answer, at least in this case. Any thoughts on the inconsistency there?
(Reporter)  
Comment 5•10 years ago


er, I meant a floating point problem. sorry.
For what it's worth, Jeff filed a webkit bug on this.
Comment 7•10 years ago


https://bugs.webkit.org/show_bug.cgi?id=24711 /be
You need to log in
before you can comment on or make changes to this bug.
Description
•