Closed Bug 557530 Opened 14 years ago Closed 14 years ago

javascript return floating-error on enter the values (162.295 or 110.565) .toFixed(2) and .toPrecision(5);

Categories

(Core :: JavaScript Engine, defect)

1.9.2 Branch
defect
Not set
minor

Tracking

()

RESOLVED DUPLICATE of bug 5856

People

(Reporter: mateus, Unassigned)

References

()

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3

There is a javscript bug on Mozilla Firefox, the errors happen when i do:
n = 165.295;
n = n.toFixed(2);
// result = 162.29 (ERROR)
// result = 162.30 (CORRECT)

THIS toFixed function SHOULD DO THE SAME AS CALCULATOR.

On Internet Explorer works very well;

please try this link http://www.mredkj.com/javascript/nfbasic2.html on Mozilla Firefox and Internet Explorer

The same erros happen when u try toPrecision(5);

Reproducible: Always

Steps to Reproduce:
1.set a float value for variable (n = 110.565)
2.set a precision decimal (n.toFixed(2))
3.javascript return float-error value
Actual Results:  
Internet explorer works. and Mozilla Firefox not works.. this is bad..
i love mozilla.

Expected Results:  
correctly results for javascript toFixed(2) and toPrecision(5)

Please fix this error.
Keywords: APIchange
please when done contact me on mateus@sistrom.com.br
thanks.
Probably WONTFIX/WORKING AS DESIGNED.
Assignee: nobody → general
Severity: critical → minor
Component: Developer Tools → JavaScript Engine
Keywords: APIchange
OS: Windows 7 → All
Product: Firefox → Core
QA Contact: developer.tools → general
Hardware: x86_64 → All
Version: unspecified → 1.9.2 Branch
> THIS toFixed function SHOULD DO THE SAME AS CALCULATOR.

Actually, no.  It should not, depending on what your calculator does.

The observed behavior is the right behavior.  From the spec, section 15.7.4.5 (available at http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf ):

  a. Let n be an integer for which the exact mathematical value of n ÷ 10f – x
     is as close to zero as possible. If there are two such n, pick the larger
     n. 

This says that the rounding mode should be round-away-from-zero.  The problem here is that 165.295 is not representable exactly as a JavaScript Number (defined as a "double-precision 64-bit binary format IEEE 754 value" in section 4.3.19 in the same specification).  Converting 165.295 to a JavaScript Number uses the closest such double, which happens to be 165.29499999999999.  The toFixed on that returns 165.29, since that's the closest number with two decimal digits after the decimal point to 165.29499999999999.
Status: UNCONFIRMED → RESOLVED
Closed: 14 years ago
Resolution: --- → INVALID
Resolution: INVALID → DUPLICATE
You need to log in before you can comment on or make changes to this bug.