Closed
Bug 156594
Opened 22 years ago
Closed 22 years ago
rounding error in format-number() function with "#0."
Categories
(Core :: XSLT, defect)
Tracking
()
VERIFIED
FIXED
People
(Reporter: jrspm, Assigned: sicking)
Details
(Whiteboard: TX_BRIDGE_1_1_FIXED)
Attachments
(3 files)
786 bytes,
text/xml
|
Details | |
184 bytes,
text/xml
|
Details | |
3.92 KB,
patch
|
axel
:
review+
bzbarsky
:
superreview+
|
Details | Diff | Splinter Review |
Using build 2002070913 on Win2k (SP2) This bug is related to bug 155734 and its testcases... Description: It seems Mozilla also has a bug in its rounding. For instance, given this input <number>37425.6770833333</number> and this xslt statment <xsl:value-of select="format-number(number, '#0.')"/> Mozilla outputs: 37425 IE outputs: 37426 It seems to me that IE is right and Mozilla is wrong in this case since any number 5 and above after the decimal point should force the function to round up. Mozilla doesn't round at all. However, Mozilla seems to do the right thing in cases such as <xsl:value-of select="format-number(number, '#0.#')"/> where it outputs: 37425.7 just like IE Expected: The pattern of "#0." should cause 37425.6770833333 to round to 37426 Actual: The pattern of "#0." causes 37425.6770833333 to round to 37425 A testcase is coming up... Jake
Reporter | ||
Comment 1•22 years ago
|
||
Reporter | ||
Comment 2•22 years ago
|
||
Reporter | ||
Updated•22 years ago
|
Attachment #90729 -
Attachment mime type: text/xml → text/xsl
Reporter | ||
Updated•22 years ago
|
Attachment #90729 -
Attachment mime type: text/xsl → text/xml
Assignee | ||
Comment 3•22 years ago
|
||
fixes this problem, bug 155734, a buffer overrun in standalone and a buffer underrun in module
Assignee | ||
Comment 4•22 years ago
|
||
@@ -287,9 +286,15 @@: fixes a buffer overrun problem where I had forgot to allocate enough buffersize for the initial zeros in formats like '00000000.0' @@ -262,7 +262,6 @@ @@ -346,6 +351,12 @@ @@ -446,6 +459,12 @@ fixes bug 155734 by not enforcing that minIntergerSize is > 0 (i.e. allow no '0's before decimal separator). It also needs to insert a '0' if we havn't outputted any numbers while creating the number. This happens when you use a format like '##.##' to format the number 0.0001. So we will now output 0.001 formatted with '##.##' as '0' and 0.01 formatted with '##.##' as '.01' @@ -373,7 +384,7 @@ This is needed to avoid a divide-by-zero if minIntegerSize == 0 @@ -316,10 +321,10 @@ @@ -330,7 +335,7 @@ Fixes some serious brokenness in standalone. It was pretty much not working before. @@ -391,16 +402,18 @@ Fixes a buffer underrun bug in module. @@ -430,7 +443,7 @@ Fixes bug 156594. I forgot to actually add +1 when detecting that a carry existed during output of the integerpart
Comment 5•22 years ago
|
||
Are we two against foo that the 6 failing xalan tests in numberformat are both # doesn't generate leading zeros, and there is no default - prepended to the second subpattern?
Assignee: peterv → sicking
Comment 6•22 years ago
|
||
Comment on attachment 90758 [details] [diff] [review] patch to fix r=axel@pike.org (standalone has two more fails, but those are just output) "Alles wird gut". Which is evil.
Attachment #90758 -
Flags: review+
Comment 7•22 years ago
|
||
Comment on attachment 90758 [details] [diff] [review] patch to fix sr=bzbarsky
Attachment #90758 -
Flags: superreview+
Assignee | ||
Comment 9•22 years ago
|
||
fixed with the branchlanding
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → FIXED
Comment 10•22 years ago
|
||
we didn't verify for a long time. I really checked, so VERIFIED.
Status: RESOLVED → VERIFIED
You need to log in
before you can comment on or make changes to this bug.
Description
•