Last Comment Bug 137155 - POST request sent as two small packets (IIS 5 sometimes chokes)
: POST request sent as two small packets (IIS 5 sometimes chokes)
Status: RESOLVED FIXED
[rft-dl]
: helpwanted
Product: Core
Classification: Components
Component: Networking: HTTP (show other bugs)
: Trunk
: All All
: P3 enhancement with 1 vote (vote)
: mozilla1.9alpha1
Assigned To: Boris Zbarsky [:bz]
:
Mentors:
: 325327 360692 366449 (view as bug list)
Depends on: 334142 378629 383976 460816
Blocks: 277056 360692
  Show dependency treegraph
 
Reported: 2002-04-12 12:01 PDT by Jeffrey Baker
Modified: 2011-06-11 06:46 PDT (History)
15 users (show)
benjamin: blocking1.9-
reed: wanted1.9+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Ethereal dump showing a failing POST to a IIS 5.0 (2.83 KB, application/octet-stream)
2003-11-25 03:04 PST, Nicola Fankhauser
no flags Details
Ethereal human-readable hex-dump showing a failing POST to a IIS 5.0 (9.84 KB, text/plain)
2003-11-25 03:05 PST, Nicola Fankhauser
no flags Details
v1 patch (causes regressions) (2.11 KB, patch)
2006-02-03 16:38 PST, Darin Fisher
cbiesinger: review+
bzbarsky: superreview+
bzbarsky: approval‑branch‑1.8.1+
dveditz: approval1.8.0.2+
Details | Diff | Splinter Review
Merged to tip (1.85 KB, patch)
2007-08-23 15:03 PDT, Boris Zbarsky [:bz]
cbiesinger: review+
cbiesinger: superreview+
bzbarsky: approval1.9+
Details | Diff | Splinter Review

Description Jeffrey Baker 2002-04-12 12:01:40 PDT
I noticed that in Mozilla 0.9.9 on Linux, POST requests are sent as two small
when they would fit in one packet.  The first packet on a particular request I
observed had most of the headers, up to and including Referer, in 861 bytes. 
The second packet included only the Content-Type header in 117.  It would be
more efficient if these could be sent in one packet.
Comment 1 Darin Fisher 2002-04-13 10:52:57 PDT
true, but we're talking about a very small inefficiency here.
Comment 2 Brant Gurganus 2002-10-13 11:31:54 PDT
[RFE] is deprecated in favor of severity: enhancement.  They have the same meaning.
Comment 3 Nicola Fankhauser 2003-11-25 03:03:02 PST
this is in fact not merely a matter of efficiency, but actually blocks POST
requests to Microsoft IIS 5 webservers.

after the first packet IIS 5 immediately responds with a "400 Bad Request" and
sliently discards the second one.

I attached an ethereal libcap dump, as well as a human-readable hexdump of the
libcap dump, showing a POST request to a Microsoft IIS 5.0 webserver resulting
in a "400 Bad Request".

To "prove" that it's mozilla, I ran the following:
# nc mica.swissonline.ch 80 < mozilla_request.txt

which worked (mozilla_request.txt contains the two assembled packets from mozilla).
Comment 4 Nicola Fankhauser 2003-11-25 03:04:33 PST
Created attachment 136281 [details]
Ethereal dump showing a failing POST to a IIS 5.0
Comment 5 Nicola Fankhauser 2003-11-25 03:05:07 PST
Created attachment 136282 [details]
Ethereal human-readable hex-dump showing a failing POST to a IIS 5.0
Comment 6 Michael Newton 2004-11-23 16:58:35 PST
I suggest changing the severity from an RFE so it gets the attention it needs.

There is no way to submit a POST form to an IIS 5.0 server.  This just came up
on the MZ forums - http://forums.mozillazine.org/viewtopic.php?p=996972
Comment 7 Darin Fisher 2004-11-23 17:21:40 PST
If IIS 5.0 has problems with this request, then it is a major bug in that
application.  I recommend that you bring it up with Microsoft.  Afterall, it is
always possible, given the way TCP/IP works, that a HTTP request may be
arbitrarily split into multiple packets.  No matter what we do here, this
problem will never go away 100%.  Of course, by buffering the request before
sending it, we can probably avoid the problem most of the time.  But, that would
be far from a perfect solution.

Given that this causes problems in the wild, I'll elevate the severity, and try
to get it fixed for Mozilla 1.8.  But, that doesn't help Firefox 1.0 or other
existing version of Mozilla.

The server should be fixed.
Comment 8 Jeffrey Baker 2004-11-23 18:16:51 PST
I could not give a flying handshake about IIS 5, but on Linux at least you can
make POST more efficient by setting TCP_CORK on the outbound socket.
Comment 9 Jeffrey Baker 2005-03-21 20:12:44 PST
The crux of the problem seems to be that the headers Content-Length and
Content-Type for an HTTP POST are attached to the request body, instead of the
request headers.  The headers and the body are attached in that order to a
multiplex input stream, and then read in order.  The channel emits a packet for
each element in the multiplex stream, which is almost certainly another bug by
itself.

The fix may be a subtle tweak in nsMultiplexInputStream.cpp.  Otherwise
something intrusive may need to be done in netwerk/protocol/http.
Comment 10 Darin Fisher 2005-03-21 22:30:32 PST
It's not a bug that the request is split into multiple TCP segments.  That can
happen no matter what.  At most this is a performance problem or an opportunity
to improve compatability with broken web servers.
Comment 11 Michael Newton 2005-03-22 19:13:37 PST
(In reply to comment #9)
> The crux of the problem seems to be that the headers Content-Length and
> Content-Type for an HTTP POST are attached to the request body, instead of the
> request headers.

Actually, if you have a look at attachment 136281 [details] in Ethereal, you'll see that
the Content-Length and Content-Type are sent as headers, they just get dropped
into the second packet.  I expect this is what's causing IIS problems.

I agree that this is more a bug in IIS than Gecko, but just as we support
rendering bugs found in other browsers, we should try to work around bugs in
servers.  Especially when all that's needed is to stop the splitting of one packet.

(Disclaimer: I don't know C++ and understand that 'all that's needed' could mean
9 hours of work!)
Comment 12 Jeffrey Baker 2005-03-22 19:26:13 PST
If you don't understand C++ then it's not going to help you much.  But if you look in netwerk/protocol/
http you will see what I meant about the headers.  There's two datastructures, one is a list of headers, 
and the other is a stream for the request body.  The headers Content-Type and -Length are not in the 
headers data structure where you would expect, they are prepended to the body stream.

There's a boolean in the request object that indicates the presence of these extra headers.
Comment 13 Michael Newton 2005-03-22 19:50:15 PST
Yes I saw that earlier, for anyone else who might be watching this, I think it's
at 
http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/http/src/nsHttpChannel.cpp#3373
[I do program PHP and Perl, so I can make /some/ sense of this stuff :)]  If
only someone else saw it, who could fix it!  However, it doesn't look like there
are resources available at MoFo right now for this little bug.
Comment 14 Darin Fisher 2006-02-03 16:27:50 PST
*** Bug 325327 has been marked as a duplicate of this bug. ***
Comment 15 Darin Fisher 2006-02-03 16:38:36 PST
Created attachment 210652 [details] [diff] [review]
v1 patch (causes regressions)

Long overdue patch.  This is not on the critical performance path, so there's not much reason to worry about the extra buffer copies.  Indeed, this seems well worth it considering the number of times this bug has come up.
Comment 16 Jeffrey Baker 2006-02-04 10:16:50 PST
Looks good from here.  This seems worth committing now that <a ping> will be firing of POST requests constantly.
Comment 17 Darin Fisher 2006-02-04 14:58:07 PST
fixed-on-trunk
Comment 18 Darin Fisher 2006-02-08 09:39:36 PST
fixed1.8.1
Comment 19 Daniel Veditz [:dveditz] 2006-02-22 12:07:47 PST
Comment on attachment 210652 [details] [diff] [review]
v1 patch (causes regressions)

approved for 1.8.0 branch, a=dveditz for drivers
Comment 20 Darin Fisher 2006-02-22 12:35:14 PST
fixed1.8.0.2
Comment 21 Dave Liebreich [:davel] 2006-02-24 14:16:45 PST
to verify: look at network trace of small POST requests - should be one packet, not two.
Comment 22 :Gavin Sharp [email: gavin@gavinsharp.com] 2006-03-09 08:53:41 PST
Could this have caused bug 329460?
Comment 23 Darin Fisher 2006-06-22 14:54:41 PDT
This patch was reverted in an attempt to fix bug 334142.
Comment 24 Boris Zbarsky [:bz] 2006-11-27 07:52:00 PST
This is actually causing people issues with XMLHttpRequest performance, since it doubles the time each call takes, more or less.

The problem is that we'd need to figure out exactly what was going on with bug 334142... :(
Comment 25 Phil Ringnalda (:philor, back in August) 2007-01-09 11:26:08 PST
*** Bug 366449 has been marked as a duplicate of this bug. ***
Comment 26 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2007-02-01 07:11:23 PST
Not going to block, but we'd like to get this fixed.
Comment 27 Jonas Sicking (:sicking) PTO Until July 5th 2007-05-03 16:31:48 PDT
*** Bug 360692 has been marked as a duplicate of this bug. ***
Comment 28 Boris Zbarsky [:bz] 2007-06-10 16:56:43 PDT
Once bug 378629 and bug 383976 land, it might be worth retrying this...  Probably trunk-only and then ask reporter of bug 334142 to retest.
Comment 29 Darin Fisher 2007-06-11 00:39:47 PDT
-> reassign to default owner
Comment 30 Boris Zbarsky [:bz] 2007-08-23 15:03:39 PDT
Created attachment 277957 [details] [diff] [review]
Merged to tip

Per discussion with biesi, we'll try this again and hope the NSS changes in the interim will prevent bug 334142.
Comment 31 Boris Zbarsky [:bz] 2007-08-23 18:44:10 PDT
Comment on attachment 277957 [details] [diff] [review]
Merged to tip

a=me
Comment 32 Boris Zbarsky [:bz] 2007-08-23 18:45:32 PDT
Re-checked in.
Comment 33 Michal Majchrowicz 2007-10-25 07:33:23 PDT
The problem still occurs. I checked recent nightly. The only difference is that the packet has to be long enough for instance if site has 800 bytes of post data firefox will still split it into two parts without any noticable pattern :( It (in example) doesn't allow to configure Micronet switches on firefox.
Comment 34 Boris Zbarsky [:bz] 2007-10-25 08:01:43 PDT
Michal, do you have a testcase showing the problem?

In general, we limit our packet size to 4KB.  If the headers are very big (e.g. lots of cookies), I can see the first packet not having space for all the data.

Also in general, if you have more data, it'll get split into more packets.

What this bug was about was us sending small packets.  At this point all the packets except the last one should be 4KB.  If that's not happening, I'd like to know.
Comment 35 Michal Majchrowicz 2007-10-25 08:07:38 PDT
Well it's trange thing. It happens if packet is longer than 730 bytes (whole packet not only post data). I noticed one strange thing current trunk works with my switch (and sends correct packets) on windoze but not on linux. This would explain why I wasn't able to access router with ies4linux. So this is something general that affects all "linux" browser but the question remains whether it can be fixed in any way and if then how ? :(
Comment 36 Boris Zbarsky [:bz] 2007-10-25 08:13:48 PDT
It might be that your OS further breaks up TCP packets (e.g your TCP window is small or some such).  Worth checking with someone familiar with the Linux networking stack.
Comment 37 Nelson Bolyard (seldom reads bugmail) 2007-10-25 11:46:06 PDT
Boris, this may be getting OT here, but why do we still have a 4KB limit
in 2007?  4KB may have been reasonable in the era of 28kb modems and CPUs 
with 16MB of RAM, but it's so small that it triggers TCP Nagle delays, 
because it's much smaller than typical TCP window sizes, and doesn't keep 
the output queue from going empty in one round trip time on today's faster 
networks.  Could we raise it to, say, 32KB or 64KB?
Comment 38 Boris Zbarsky [:bz] 2007-10-25 12:37:28 PDT
I have no idea why this code is reading things in 4KB chunks.  You'd have to ask Darin.  Necko seems to pretty consistently use 4KB buffers for all sorts of things.

That's a discussion for a separate bug, though.  And that discussion will need some hard data on how different packet sizes affect users.
Comment 39 Michal Majchrowicz 2007-10-25 15:39:11 PDT
The problem is that the packet doesn't have even 1KB :( I don't know why it happens this way. Does anyone know any way of controling it ?
Comment 40 Michal Majchrowicz 2007-10-29 10:23:45 PDT
I have created a simple test case that shows the problem
http://xfree.republika.pl/test.html
Can someone with linux and wireshark installed confirm the existance of the bug?
Comment 41 Jeffrey Baker 2007-10-29 10:49:33 PDT
I tried your test case with Firefox 2.0.0.8, where the POST is sent as two packets, and with a Firefox trunk nightly, where the POST is sent as a single packet.  I suggest that any problems you are seeing are related to your specific computer or network.

Tested with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a9pre) Gecko/2007102904 Minefield/3.0a9pre

I recommend leaving this RESOLVED/FIXED
Comment 42 Michal Majchrowicz 2007-10-29 11:33:28 PDT
Did you check the packet size of with nightly? I noticed that wireshark not always correctly displays packets. Moreover I confirmed this bug on few machines not only mine.
Comment 43 Jeffrey Baker 2007-10-29 11:53:23 PDT
Yes, I confirmed with wireshark, tshark, and plain tcpdump.  In your test, on the trunk nightly, there are 1514 bytes in the first frame and 841 in the second.  That is the expected result.
Comment 44 Michal Majchrowicz 2007-10-29 13:47:57 PDT
So why do you thing the bug is fixed? And why do you say it is expected result. You first say it was one single packet and then that there were two frames (one 1514 and second 841). Why not one single frame? That is what happens when you run nightly on windoze.
Comment 45 Jeffrey Baker 2007-10-29 14:03:29 PDT
I don't think we should continue this conversation on this bug, as many people are receiving unnecessary bug spam.  1514 bytes is the expected frame size of ethernet.  No larger packets are expected normally, unless jumbo frames are in use.

Note You need to log in before you can comment on or make changes to this bug.