442094 - Rendered XML garbled particularly when server returns small chunks

Status: NEW

(Core :: XSLT, defect, P5)

Description

[Jake Garver]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

Starting with Firefox 3, we see clipped output when rendering an XML file generated by our embedded device.  I was able to reproduce the problem with a static XML file, but only if I wrapped with a "dd" that chunked the return.  The problem is very evident when using very small 5 byte chunks.

Attached is a screenshot, Truncated_XML_Rendering.jpg, so you can see what I'm talking about easily.

The page renders fine in Firefox 2 and IE7.



Reproducible: Always

Steps to Reproduce:
To reproduce, put the following 3 files in the same directory on a web server.  I used Apache on Linux.
1. x.xml.  This is a simplified version of our output, made a little longer so you'll be sure to see the problem.
2. list.xsl.  A simplified version of our stylesheet.
3. breakit.cgi.  A simple CGI script that uses "dd" to make 5 byte chunks of x.xml.

To see the problem, point Firefox 3 at "breakit.cgi".  Usually the first 10-30 lines are fine, but then you'll see the "unknown"s start to get truncated.  A refresh results in a different pattern, implying some randomness.

Actual Results:  
Usually the first 10-30 lines are fine, but then you'll see the "unknown"s start to get truncated.  A refresh results in a different pattern, implying some randomness.

Expected Results:  
Each line should be:
X/X enable unequipped unknown unknown none unknown unknown unknown unknown unknown

I suspect the problem is around the input boundaries.  If the server sends big chunks there are fewer input boundaries and you're less likely to see it.  If the server sends small chunks, more input boundaries, greater probability.

Comment 1

[Jake Garver]

Attachment: Actual output, demonstrates render problem

Comment 2

[Jake Garver]

Attachment: Simple CGI script to chunk the static file and expose the problem.

Comment 3

[Jake Garver]

Attachment: Stylesheet

Comment 4

[Jake Garver]

Attachment: Static XML file demonstrating the problem when processed through breakit.cgi

Comment 5

[Jake Garver]

The problem still exists in Minefield 3.1a2pre (Build 20080826034105).

Comment 6

[Jake Garver]

The problem still exists in Minefield 3.1b1pre

Comment 7

[Jake Garver]

It seems to be caused when text() crosses packets.  To test this theory, I wrote the following simple wrapper around my static xml file (attached).

The first fflush will flush after every packet and causes severe corruption in the rendered page.  The second fflush will guarantee that text() is always sent whole.  When the first fflush is commented and the second is uncommented, the screen renders fine.

So, I have a (ugly) workaround.

#include <stdio.h>

int main(int argc, char *argv[])
{
        FILE *src=fopen("x.xml", "r");
        int next;

        printf("Content-Type: text/xml\n\n");

        while((next=fgetc(src))!=EOF)
        {
                fputc(next, stdout);

                /* breaks */
                /* fflush(stdout); */

                /* doesn't break */
                if(next=='>') fflush(stdout);
        }
}

Comment 8

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

My guess is that this broke in the incremental xml bug. What likely is happening is that we create multiple separate textnodes, rather than appending to the existing one. We used to have code to ensure that we always appended to existing one when parsing an XSLT source file.

To work around this, try changing

<xsl:template match="*" mode="listItem">
   <!-- Finally, output the column -->
   <td class="contentTableCell">
   	<xsl:value-of select="text()"/>
   </td>
</xsl:template>

to

<xsl:template match="*" mode="listItem">
   <!-- Finally, output the column -->
   <td class="contentTableCell">
   	<xsl:value-of select="."/>
   </td>
</xsl:template>

Comment 9

[Henri Sivonen (:hsivonen)]

Perhaps fixed by the patch in bug 449219?

Comment 10

[Jake Garver]

Thanks, Jonas.

I can't always use "." because we might have text in sub-elements, but the following will also work.

<xsl:value-of select="text()[1]"/><xsl:value-of select="text()[2]"/>

That also seems to confirm that it's being broken into two (or more) textnodes.

Comment 11

[Johnny Stenback (:jst)]

Not blocking on this after all. Wanted+, and over to Jonas.

Comment 12

[Jake Garver]

I'm using the following general purpose template to work around this problem.  Anywhere that I was using text(), I call or apply this into a variable instead.  Of course, I use text() everywhere, so this work around was a bit invasive.

<xsl:template name="detailText" match="*" mode="detailText">
   <xsl:for-each select="text()">
      <xsl:value-of select="."/>
   </xsl:for-each>
</xsl:template>

The problem was worse on long screens, but I proved that I could experience it on any screen whose XML crossed multiple packets.  That's why I had to go around and wrap every use of text().

Comment 13

[Samuel Sidler (old account; do not CC)]

Doesn't really meet the "wanted" criteria (security, stability, regression from maintenance release) for 1.9.0.x. However, we'll look at a reviewed and baked patch.

Comment 14

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

This is a regression introduced in the 3.0 release. Renominating.

Comment 15

[Daniel Veditz [:dveditz]]

Marking it "wanted1.9.0.x" doesn't make a patch appear, just means we'll have a more positive attitude when presented with a branch approval request. Still needs to get fixed on trunk.

Comment 16

[Sylvestre Ledru [:Sylvestre]]

Decreasing the priority as no update for the last 2 years on this bug.
See https://github.com/mozilla/bug-handling/blob/master/policy/triage-bugzilla.md#how-do-you-triage 
about the priority meaning.