568804 - Redirect by proxy causes broken connection under OSX

Status: RESOLVED DUPLICATE of bug 452304

(Core :: Networking: HTTP, defect)

Description

[Simon Andrews]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3

We have a websense filtering system which works through a proxy server.  Some requests are redirected with a 302 response to a holding page.  Using firefox under OSX an error is produced in the browser when the connection is made.  Doing the same thing on Firefox on Windows or Linux works properly.  Other browsers on OSX also work.

Running the command in wget and copy/pasting the redirect link back into firefox also works so it's not a problem with the final page.  Trying to capture the headers with the live HTTP headers extension shows no headers sent from the request so it looks like a problem really early on.

This problem has previously been mentioned by others in the forum, but with no resolution.

http://support.mozilla.com/en-US/forum/1/92610

Reproducible: Always

Steps to Reproduce:
1.Attempt to access websense blocked site.

Actual Results:  
The connection was reset

The connection to the server was reset while the page was loading.

Expected Results:  
Browser is redirected to the websense holding page.

I'll attach tcpdump files for two connections from OSX and linux showing a failure in OSX and a corresponding success in linux.

Comment 1

[Simon Andrews]

Attachment: TCPDump of port 8080 during a failed redirect

Comment 2

[Simon Andrews]

Attachment: TCPDump of port 8080 during a successful redirect on Linux

Comment 3

[Boris Zbarsky [:bzbarsky]]

Simon, could you do a log of the failure following the instructions at https://developer.mozilla.org/en/HTTP_Logging and attach it here?

Comment 4

[Simon Andrews]

Attachment: Firefox log of failed redirect

The log shows a browser session where the browser opened to the default homepage followed by an attempt to connect to docs.google.com which should be intercepted by the proxy and redirected.

Comment 5

[Boris Zbarsky [:bzbarsky]]

Can you also do the corresponding Linux log?

This is where the failure comes in the attached log:

-1340051456[2811260]: nsHttpTransaction::HandleContent [this=1deee240 count=0 read=0 mContentRead=0 mContentLength=-1]
-1340051456[2811260]: nsSocketInputStream::Read [this=1b6c0dc0 count=4096]
-1340051456[2811260]:   calling PR_Read [count=4096]
-1340051456[2811260]:   PR_Read returned [n=-1]
-1340051456[2811260]: ErrorAccordingToNSPR [in=-5961 out=804b0014]

The "-5961" error is PR_CONNECT_RESET_ERROR.

Comment 6

[Simon Andrews]

Attachment: Firefox log of successful redirect under linux

This log also shows an attempt to connect to docs.google.com which is successfully redirected to the websense holding page.

Comment 7

[Boris Zbarsky [:bzbarsky]]

Right.  The corresponding part of this log looks like this:

-366016752[7fd2f2b547b0]: nsHttpTransaction::HandleContent [this=df589b70 count=0 read=0 mContentRead=0 mContentLength=-1]
-366016752[7fd2f2b547b0]: nsSocketInputStream::Read [this=e0345f88 count=4096]
-366016752[7fd2f2b547b0]:   calling PR_Read [count=4096]
-366016752[7fd2f2b547b0]:   PR_Read returned [n=0]

So I guess the real question is why the PR_Read (which presumably just calls read()) ends up failing on Mac.  It sounds like the OS might be randomly closing the socket out from under us or something....

Jason, any ideas?

Comment 8

[Boris Zbarsky [:bzbarsky]]

Simon, do non-proxy redirects with empty body lead to the same problem?

Comment 9

[Simon Andrews]

If I create a small CGI script* to do a normal redirect I can run it either directly or through the proxy from firefox on my Mac and they both work fine.  It's something about the redirect from websense which is odd.  I've not had any problems with any other connections.

*
#!/usr/bin/perl
use warnings;
use strict;
use CGI;

my $q = CGI->new();
print $q->redirect("http://bilin3");

Comment 10

[Boris Zbarsky [:bzbarsky]]

Simon, does that produce a redirect response with no body in your web server?

Comment 11

[Simon Andrews]

I think so.

$ ./redirect.pl 
Status: 302 Found
Location: http://bilin3


$

I don't think Apache adds anything (but I'm not entirely sure how to check)

Comment 12

[Boris Zbarsky [:bzbarsky]]

Probably the simplest way to check is something like this.  Type at your prompt:

  telnet www.example.com 80 | tee ~/log.txt

(except with your actual hostname, not 80) then type in:

GET /whatever HTTP/1.0
Host: whatever-the-hostname-is

and hit enter twice after the hostname (so as to create a blank line).  At that point log.txt should contain the server response (as well as your actual telnet communication at the top).

Given that printout from your perl script, it looks like it has a newline in the body, fwiw.

Comment 13

[Simon Andrews]

Attachment: telnet response from normal 302 redirect

I think you're right, and that there is a blank line in the redirect.  If I try to serve out a redirect without the blank line apache throws a server error.

Comment 14

[Boris Zbarsky [:bzbarsky]]

No, that has an empty body (it has the one blank line that's required after the headers, but nothing after that).

But it also has a Content-Length header, which changes the codepath that we follow.  In particular, near the end of nsHttpTransaction::HandleContent we have:

978     // check for end-of-file
979     if ((mContentRead == mContentLength) ||
980         (mChunkedDecoder && mChunkedDecoder->ReachedEOF())) {
981         // the transaction is done with a complete response.
982         mTransactionDone = PR_TRUE;

and then in both nsHttpTransaction::WritePipeSegment and nsHttpTransaction::WriteSegments we check for mTransactionDone and return NS_BASE_STREAM_CLOSED if it is, instead of calling down into read().

Let me see if I can reproduce this sort of thing locally.

Comment 15

[Boris Zbarsky [:bzbarsky]]

Apache won't let me not send a Content-Length header, sadly.  Looking for another way.

Comment 16

[Boris Zbarsky [:bzbarsky]]

OK, I found some other redirects that send empty body and no Content-Length, and those seem to work ok.

I don't know enough TCP offhand to say whether the TCP logs show anything interesting; Jason might, though.

Comment 17

[Simon Andrews]

It seems there is another bug open on this topic with a script which demonstrates the cause (an extra RST packet served with the redirect).

BZ#452304

Comment 18

[Jason Duell]

CC-ing our OSX and NSPR usual suspects, in case they've got any experience with OSX sockets "thinking different" :)

Comment 19

[Boris Zbarsky [:bzbarsky]]

> BZ#452304

Indeed.  Marking duplicate, since that bug has more useful info.