Closed Bug 195193 Opened 23 years ago Closed 21 years ago

Multipart JPEG streams can wedge after first image

Categories

(Core :: Networking: HTTP, defect, P2)

defect

Tracking

()

RESOLVED WORKSFORME
Future

People

(Reporter: steve, Assigned: darin.moz)

Details

(Keywords: helpwanted)

Attachments

(1 file, 1 obsolete file)

User-Agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.3b) Gecko/20030210 Build Identifier: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.3b) Gecko/20030210 This was first noticed when trying to view the webcam output of 'motion' (V3.0.4). Netscape 4.76 could show the streams, Mozilla could not. Experiments show that the Mozilla problem relates to the multipart boundaries. *If* the boundary string is written in the same write() call that writes the final block of the JPEG, then Mozilla is happy and streams it nicely. If the boundary gets written in a different write() call though, it freezes. This is true of Mozilla certainly up to 1.3b (official build-id 2003021008). It may be more difficult to observe this over a WAN, where the MTU may be small and network fragmentation might cause the boundary to get separated from the final block of the JPEG. We get our results over ethernet (MTU 1500ish). Reproducible: Always Steps to Reproduce: 1. Get 'motion' version 3.0.4 (see motion.sourceforge.net) 2. Set it up as a webcam. 3. Try to view it with Mozilla! Actual Results: First image shows OK, nothing follows. Expected Results: A continuous video stream. This is NOT a duplicate of bug #42224. There is some discussion of it there, but I've been asked to create a new bug because 42224 originally dealt with some other issue that merely looked like this one. :-)
To darin for investigation....
Assignee: asa → darin
Component: Browser-General → Networking: HTTP
QA Contact: asa → httpqa
confirming
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Linux → All
Hardware: PC → All
I developed a version of motion based on 2.9.6 that used a separate thread for all webcam operations. By the time I actually manage to get a patch out for the main motion, it'd had changed a fair bit and I didn't have time to port it. The first thing it fixed was a newly connected client always got an image as soon as the first header was sent which seemed to make mozilla a lot happier. In the setup I was using with motion there could be up to a 5 or 10 sec delay between sending the initial header and the next image. The second thing it change was that it removed the delay between the boundry string (and it's header) and the image of each part. It doesn't combine the write()s like the original comment, but the reduced delay may fix it somehow.. ? I'd be happy to post the original patch for motion (2.9.6) that I have.
To Ashley Cambrell: obviously we don't want to clutter up this form talking about the internals of 'motion', but I'll chat with you on email about it later. Sounds like you and we have similar things. The 'motion' people might want it. To everyone: A rider to the original report: Mozilla can still lock up on the first image even with the trailing boundary in the same write() as the last image data. Hitting 'reload' a couple of times unwedges it however, and it seems that once it's going it stays going. I'd say I'm seeing this behaviour maybe 50% of the time I request a streamed JPEG. With the original 'motion' webcam, no amount of 'reload'-ing unwedged it.
It's true, this bug discusses the same symptoms as bug 42224. So, what _is_ the difference between this bug and bug 42224? Cheers Daniel Kabs Germany
bug 42224 was a different issue and was fixed. This bug arose after that was fixed, and it would be too confusing to reopen that one since the discussion there is all over the place and not focused.
Well, here you are! This is a testcase that exposes Mozilla's problems displaying a jpeg server push correctly. Please have a look at the attachment and tell me, what you observe. Cheers Daniel
just a quick note regarding the testcase: A packet trace reveals that the connection is terminated with a _client_ side TCP/IP reset (RST-flag), i.e. the browser closes the connection without emptying its TCP-buffers.
another quick note: In the testcase the streaming server does write the multipart boundary and the content-type header using _separate_ fprint() calls.
Workaround on the server side: Writing the boundary string and the content-type header in the same write() immediately after the image data solves the problem! This seems to be strong evidence that Mozilla is the culprit. Cheers Daniel
thx for the info daniel.
Severity: normal → major
Status: NEW → ASSIGNED
Priority: -- → P2
Target Milestone: --- → mozilla1.5alpha
Target Milestone: mozilla1.5alpha → Future
Hi Darin, it's a pity you had to change the target milestone from 1.5a to an indeterminate "future". The jpeg server push is such a nice thing to have in Mozilla...
daniel: with luck i'll get a chance to pull this back in, but at the moment there are way too many other bugs that need attention.
Related to current bug with extended use. If "<img src=''>" tag references a cgi/php/script application that returns a Header Redirect to the Motion/Multipart JPEG the Mozilla/Firebird application itself will lock up stop responding and crash taking 100% cpu. The script I am currently using is a PHP header redirect to the stream. Once this current issues is resolved with Multipart/JPEGs failing I would figure the issue with header redirected Multipart/JPEGs images would also be resolved. This issue doesn't happen with static gif/jpg images.
We are talking about the Content-Type: multipart/x-mixed-replace First, let's have a closer look at the anatomy of this type of content. This special kind of content may contain multiple documents of different types. In the majority of cases the document type will be the same and is of type image/* (e.g. image/jpeg). This is also known as "server push" or "server stream". The documents (or "parts") of the multipart content are separated by a multipart boundary string. This string defined in the Content-Type header of the HTTP response (sent from the server to the client). To define the content type of each "part", there has to be a Content-Type header preceding each document data. Thus a typical "server stream" may look like: ----------------------- HTTP/1.0 200 OK Content-type: multipart/x-mixed-replace;boundary=Boundary-String Boundary-String Content-Type: image/jpeg [Data] Boundary-String Content-Type: image/jpeg [Data] ... ----------------------- HTTP-Header, Boundary-String, Content-Type and Data are separated by Linefeeds (or CRLF?) Further investigation into this bug showed the following: Mozilla closes the stream, when the Boundary-String and the succeeding LF character are transmitted separately and with a time delay of more than 30 milliseconds. It seems, this is a timing issue. A network protocol of this situation looks like: (Browser runs on danielx, the web server runs on m1-daniel) No. Time Source Destination Protocol Info 266 1.253173 m1-daniel danielx HTTP Continuation 267 1.293169 danielx m1-daniel TCP 34727 > http [ACK] 268 1.293636 m1-daniel danielx HTTP Continuation 269 1.293710 danielx m1-daniel TCP 34727 > http [ACK] 270 1.295261 danielx m1-daniel TCP 34727 > http [FIN, ACK] In Frame #266 remaining data and the boundary string are sent: Hypertext Transfer Protocol, 1024 bytes Data: [...] 03e0 d3 48 2e 7f ff d9 0a 2d 2d 4d 4f 42 4f 54 49 58 .H.....--MOBOTIX 03f0 5f 46 61 73 74 5f 53 65 72 76 65 72 70 75 73 68 _Fast_Serverpush In Frame #268 the LF character and the content type header are sent: Hypertext Transfer Protocol, 27 bytes Data: 0000 0a 43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 69 .Content-type: i 0010 6d 61 67 65 2f 6a 70 65 67 0a 0a mage/jpeg.. Note, that the delay between these two packets is about 40ms. In Frame #270 the browser closes the TCP connection down. Thats the problem.
Correction to the "server stream" example: There must be no empty line between "HTTP/1.0 200 OK" and "Content-type: multipart/x-mixed-replace;boundary=Boundary-String"!
I've updated the camera which the testcase uses. Now the bug is triggered more reliably: The camera runs on 2 frames per second. After a maximum number of 16 images the stream should fail on Mozilla. Have a look at the testcase at: http://bugzilla.mozilla.org/attachment.cgi?id=118643&action=view
5000 frames and still going here... Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 Cable internet connection.
WFM - Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 T1 connection
(In reply to comment #18 and comment #19) > 5000 frames and still going here... Thanks for checking! I'll try to explain your results: As described in comment #15 the bug is triggered when the browser receives the boundary string and the following CR character separately. This is currently implemented by doing two write()-calls, separated by a usleep(50000). Because of the (intentionally induced) delay, the TCP/IP stack transfers the data in two separate packets. If the time delay also exists at the receiver, the browser will have to read() the data using two calls. But if the time delay is reduced during transfer, the browser will use only one read() and thus the bug will not be exposed. I'll work on this. :-)
If you are daring, you could try to employ a packet sniffer like ethereal (www.ethereal.com). Sniff the network traffic while the testpage is opened in your browser. Filter out packets with tcp.len == 27. These are the packets containing the LF character and the "Content-Type" header following the boundary string. Write down the frame numbers. Clear the filter to show all packets. Check the time delay between the packets you just collected and the preceding packet with the boundary string. How long is it?
The testcase now allows to set the delay from 10 ms to 1000 ms. Please check this and see if the bug is exposed when you increase the delay time.
i've increased the delay till 1000ms and after some 100 retries got Error: Stream aborted unexpectedly! Image name: video_stream Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 T1 connection
(In reply to comment #23) > i've increased the delay till 1000ms and after some 100 retries ... That's really strange. In my local network the error message duly appears with the 16th frame. Could you do a network trace of the first 40 frames and send it to me, please? Alternatively, I could try to provide a program that "simulates" a HTTP server. @ALL: Would you prefer to get the source code for WIN or UNIX?
Keywords: helpwanted
Update: I checked the testcase (the one with variable delay) with Mozilla 1.7.5 and 1.7.7. I can not reproduce the bug any more with these releases. Using Mozilla 1.6 the bug still exists (stream aborts after a approx. 16 images and a javascript alert appears). So this is a Wfm now :-)
RFC 2046 is quite clear about the syntax of the "Multipart Media Type" which a server push uses (see section 5.1 of that document). Especially about the characters terminating a header line: CRLF. Until I read the RFC I thought that LF was sufficient because Mozilla accepted the stream and displayed it properly. Now the stream format has been changed to be RFC compliant by sending CRLF as line terminating characters. Consequently, if you want to filter out the special packets "created" by the variable delay you now have to use "tcp.len == 30" (instead of 27)! Still the above said holds true: The stream aborts on Mozilla 1.6 but runs flawlessly on 1.7.5 and higher.
marking WORKSFORME based on previous comments.
Status: ASSIGNED → RESOLVED
Closed: 21 years ago
Resolution: --- → WORKSFORME
Attachment #118643 - Attachment is obsolete: true
Comment on attachment 118643 [details] Testcase for Mozilla server JPG push fails after several frames Testcase obsolete as camera URL is not available any more.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: