292622 - Lost packets with XMLHttpRequest in multipart/x-mixed-replace mode

Status: RESOLVED WONTFIX

(Core :: XML, defect)

Description

[Wilfred Nilsen]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3

I am using the Mozilla XMLHttpRequest in multipart/x-mixed-replace mode such
that the server can push data to the client. 

The problem that I encountered is that the onload event listener does not
trigger if the server pushes data too frequently to the browser. I presume the
new data replaces the old data before the JavaScript onload event listener gets
a chance to execute.

I am today using a hidden iframe to push data asynchronously to clients. The
idea was to replace the hidden iframe with an XMLHttpRequest in
multipart/x-mixed-replace mode.

The data can be sent at any time, thus the time between two packets can
therefore be zero. I initially send 4 packets in my test program with no time
delay between the packets. The onload event listener triggers for the first
packet only.



Reproducible: Always

Steps to Reproduce:
1. Make an XMLHttpRequest request in multipart/x-mixed-replace mode to a server.
2. The server sends messages in burst to the client.

Actual Results:  
The onload triggers only for one of the messages in the burst of messages sent.



Expected Results:  
The onload should trigger for all messages sent.


I also have some suggestions on how to improve XMLHttpRequest in
multipart/x-mixed-replace mode in anyone is interested. I can also make a small
server executable that can be used as a test case.

This BUG is related to BUG 246651 i.e. you need to setup a better test suit for
the XMLHttpRequest object.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Is this a problem with a trunk build (eg the Deer Park preview)?  If so, do you
have a URL that shows the problem?

Comment 2

[Wilfred Nilsen]

I am using standard installations of Firefox, Mozilla etc. I have the same
problem in all browsers. I have tested on Window, Linux and Mac OSX.

I do not have a link that you can use for testing, though I could produce a
small executable of our embedded web-server.

What we are trying to do is using the multipart/x-mixed-replace as the
downstream in a full duplex asynchronous communication stack we have on top of
HTTP. Asynchronous means: data can be sent at any time and this is the problem
as data is overwritten when packets with no time delay in between are received
by the browser.

Comment 3

[Boris Zbarsky [:bzbarsky]]

> I am using standard installations

Yes, but what version?

Comment 4

[Wilfred Nilsen]

Well, the Firefox version I am using right now is Firefox/1.0.4, but this
problem applies to all versions so far.

Comment 5

[Boris Zbarsky [:bzbarsky]]

I assume "all versions" includes the Deer Park preview?

If so, could you possibly attach an HTTP log (per
http://www.mozilla.org/projects/netlib/http/http-debugging.html )?

Comment 6

[Wilfred Nilsen]

Attachment: The log file

Comment 7

[Wilfred Nilsen]

Attachment: Small server (windows executable)

I have created a log as requested. I have also attached a small server that is
self contained .i.e all web-pages are contained within the executable.

Start the server and open mozilla on the same computer with URL
http://127.0.0.1:9357/SimpleDebugger/

You may have to press refresh if you get an error message. This is related to
another bug, where Mozilla have problems deflating gzip files. The JavaScript
library code is stored in the server’s executable as a gzip file.

You will see some initial text in the browser window. The browser window should
show much more but this is lost when the server pushes the data. You can send
additionally data to the client from the server by typing and pressing return
in the server’s console window. This works fine, but try pasting a number of
text lines into the console window and you will see the text is lost i.e. not
shown in the browser window.

Comment 8

[Boris Zbarsky [:bzbarsky]]

Darin, biesi, any idea what could be up here?  I can't even open zip archives
over here, so nothing useful I can do until I get back to town....

Comment 9

[Darin Fisher]

CC'ing jst since he added support for multipart/x-mixed-replace to XMLHttpRequest

Comment 10

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 11

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.

Comment 12

[Boris Zbarsky [:bzbarsky]]

Reopening. This should never have gotten expired.

Wilfred, I looked through the log and nothing jumps out at me... which exact request is the XMLHttpRequest?  Would it be possible to attach the data in question without the server?  Hard to test this bug as-is on a non-Windows system...

Comment 13

[Wilfred Nilsen]

I have tested the new Firefox 3.5.2 on Windows and it now appears to work much better. I can send data up to 100 messages a second on the Windows version of Firefox. Maybe the bug was fixed? It may be that the bug is related to other parts of the browser and that the problem still persists on other platforms.

We now have a number of demo servers for various platforms you can use to reproduce the error. Follow the following procedures if you like to verify if multipart/x-mixed-replace works.

Download a server from the following page:
http://barracudaserver.com/Barracuda_web_server_SDK.html

Start the demo by using the start script / batch file

Navigate to http://localhost:80/diy/, where localhost is the IP address of the server and 80 is replaced by the port number the server is listening on.

In the menu, click Lua tutorials, and then "Coroutines".

Scroll down to "Creating a Timer in Coroutine" Mode

Click the execute button to start the timer. The output is sent to the browser via "XMLHttpRequest in multipart/x-mixed-replace mode".

Click the next execute button, which changes the time interval to 100 milliseconds. This should still work. Change the time to say 10 milliseconds.

Comment 14

[Boris Zbarsky [:bzbarsky]]

Ria, Martijn, can you reproduce with older builds?  If so, can you figure out when the behavior changed, if it also works for you in current builds?

Comment 15

[Wilfred Nilsen]

On a second thought, it may be that I no longer see the bug since I am now using a faster computer. I think the bug is related to an internal race condition, which may not be so easy to reproduce on a faster computer.
I suggest testing on an older computer with no more than 1.5Ghz.

Comment 16

[Wilfred Nilsen]

multipart/x-mixed-replace does not work in FF 4.0

This bug was never fixed in FF 3.xx, but multipart/x-mixed-replace at least worked when not sending too much data from the server.

multipart/x-mixed-replace does not work at all in the new FF 4.0.

I now get the following error when using FF 4.0: 0x80004005 NS_ERROR_FAILURE

Comment 17

[Boris Zbarsky [:bzbarsky]]

We have automated tests for multipart/x-mixed-replace in XMLHttpRequest, and they're passing....  So whatever you see is not a uniform problem.

Can you please file a separate bug on it not working at all (since it's a different issue from this bug) and point to a testcase?  Preferably one that does not require running an untrusted binary blob on Windows to use?

Comment 18

[A. Przygienda]

so the XMLHttpRequest still has problems (at least on Windows).

I'm pushing over 3-4 parallel XMLHttpRequests (staged about 1sec apart but overlapping) small XML updates (used to be BIG, chopped them up, made line length smaller etc.) and it works fine on a big Linux box but on Windows arbitrary XML messages receive the 

"not well-formed" 

parser error at arbitrary places. The XML is 100% perfect, runs through complete schema validation before being pushed out & seems to work 100% on linux. 

What is needed is probably a stress test 

3-4 XMLHttp  pushing each couple 100s XML as quickly as possible & validating all came & were valid. 

Otherwise, until websocket is around, this is the most elegant & best to debug technology so those sanaffu's are a real pity.

Comment 19

[Boris Zbarsky [:bzbarsky]]

Note that x-mixed-replace for XHR is probably being removed.  See bug 843508.

Comment 20

[A. Przygienda]

well, yes, so before it's even stable, up and to something new. websocket is nowhere stable so it's amazing how obscure hack like comet seems to be winning the game. Just MHO.

Comment 21

[Wilfred Nilsen]

I reported this problem in 2005 and the FF team has not been able to fix it. I have given up on FF and I have instructed all of our customers to use Chrome and WebSockets. We have so far not had any problems with Chrome.

Comment 22

[Boris Zbarsky [:bzbarsky]]

Wilfred, why exactly does using WebSocket in Firefox not work?

Comment 23

[Wilfred Nilsen]

This thread is for "multipart/x-mixed-replace" not WebSockets. We simply switched to Chrome and Chrome does not implement "multipart/x-mixed-replace" so we use WebSockets.

We have not done any extensive testing using WebSockets in FF, but I recall that we run into the same problems as we did with "multipart/x-mixed-replace" when sending data at a rapid rate. We can send data chunks as fast as every 10 milliseconds and FF seems to drop data also for WebSockets. However, I cannot confirm if this is the case with newer versions.

Comment 24

[A. Przygienda]

well, back to Chrome & the drawing board then. Thanks Wilfred. Problem is that all the server code for websocket is beta quality @ best in my domain. I hope that stabilizes before the wheel gets reinvented again.

Comment 25

[Anne (:annevk)]

The feature has been removed per bug 843508 and that stuck.