22886 - redirects with no trailing CRLF after headers don't work

Status: VERIFIED DUPLICATE of bug 48200

(Core :: Networking, defect, P1)

Description

[Cesar Eduardo Barros]

When you open www.nsa.gov, it sends a redirect to www.nsa.gov:8080 (as lynx
clearly shows). However, mozilla thinks it has gotten a document with unknown
MIME type and displays the MIME error message instead of loading the redirect
(and the way it behaves in lynx makes it look like a http redirect).

Comment 1

[Cesar Eduardo Barros]

Adding my other email to the cc:, since the default one isn't working.

Comment 2

[John Morrison]

[This applies to win95 1999123008 but it must be cross-platform].

While Lynx and Nav4.6 do follow the redirection, libwww-perl also chokes on
the headers. It appears that headers are not complete/malformed. (In the
output below, note that I'm "faking" the user-agent to be Nav4.0).

------8<--------------------------------------------------------
myunixbox% ./GET -x -e -S -d http://www.nsa.gov/
LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: GET http://www.nsa.gov/
LWP::UserAgent::_need_proxy: (http://www.nsa.gov/)
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET / HTTP/1.0
Host: www.nsa.gov
User-Agent: Mozilla/4.0 [en] (Win95; I)

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.0 302 REDIRECT
Location: http://www.nsa.gov:8080/
LWP::Protocol::http::request: HTTP/1.0 302 REDIRECT
LWP::Protocol::http::request: need more header data
LWP::UserAgent::request: Simple result: Internal Server Error
GET http://www.nsa.gov/ --> 500 unexpected EOF before all headers seen
Client-Date: Sun, 02 Jan 2000 20:27:17 GMT

myunixbox%
------8<--------------------------------------------------------

Passing to [e-mail]/Networking [... but remember, be nice to the
crypto masters :-]

Comment 3

[Cesar Eduardo Barros]

Tested with telnet. It's completely wrong. After you send the first line (GET /

HTTP/1.0), it does the redirect without waiting for the double CR/LF (empty

line), and sends the 302 Redirect and the Location line, and (the important

part), neglects to send the empty line after that. Which means Mozilla is right

and the redirector is wrong. We should present an error message in this case,

since the site is not following the standard.



I think we should also complain to the webmasters of that site, so that they add

an empty line after the Location: field, to terminate the headers properly.



The browsers that work seem to be ignoring all headers after the Location: and

redirecting anyways even with a broken HTTP reply.



I think this non-bug should not be fixed, and marked as a feature. We do not

need to comply with defective servers sending erroneus replies.

Comment 4

[Warren Harris]

Sounds like a backwards compatibility issue.

Comment 5

[Gagan]

While its true that the server is not waiting for the end of request before
sending us the response, we should still be able to handle this correctly. The
main problem is with the response portion where the Location header does not
have a trailing CRLF pair and the server shuts down the connection. In "past"
(working versions of the Lynx/Nav 4.*) we had been following "Location" headers
as soon as we saw them and that is why it all worked.

Comment 6

[John Morrison]

*** Bug 27060 has been marked as a duplicate of this bug. ***

Comment 7

[Warren Harris]

Moving post beta bugs to M18 which is now the post-beta milestone.

Comment 8

[John Morrison]

*** Bug 34913 has been marked as a duplicate of this bug. ***

Comment 9

[Sean Richardson]

Adding to Summary to include keywords for problem, making this more findable:
"(no trailing CRLF after headers)"

Comment 10

[Gagan]

It looks like the nsa.gov page changed so that its no longer redirecting... Tom 
could we have a page setup to do this internally? Or if any of you know of 
another site thats doing this... update the URL please. Thx.

Comment 11

[Gagan]

ooops... updating QA to tom. 

Comment 12

[John Morrison]

This URL from Bug 34913, http://go.compuserve.com/OnlineIssues eventually 
resolves to a Location: without a full ending to the headers (although it
goes through a couple of redirects first, so it may still be useful to 
set up a page to test the basic case). 

Comment 13

[Chris Purdum]

I did some digging and http://go.compuserve.com/OnlineIssues redirects 
eventually to http://forums.compuserve.com/vlforums/default.asp?
SRV=OnlineIssues and that is where the Location: header with the missing CRLF 
is sent.

Chris Purdum, CompuServe

Comment 14

[Jason Kersey]

If you need another testcase, check out our resources link on MozillaZine.  It 
redirects from the index file in that directory to a different file in the same 
directory.  Instead of going to that file, it redirects to 
http://www.mozillazine.org/resources/.1 instead of to 
http://www.mozillazine.org/resources/recommendations1.html .

Comment 15

[rxsherm]

http://www.orageeks.com seems to fail on its redirect as well. Will it do for
your test case?

Comment 16

[Gagan]

updating the URL and cc'ng andreas.

Comment 17

[Andreas Otte]

This is the document we get when trying to access
http://www.mozillazine.org/resources/

<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta http-equiv = "refresh" content=".1; url=./recommendations1.html">

</head>

<body bgcolor="#FFFFFF">

</body>
</html>

Seems to be another problem then the reported original one. Obviously we can't
handle this:

content=".1; url=./recommendations1.html"

The . is the problem, we do not recognize it as a .1 seconds refresh time and
take it instead as a relative uri. Take a look at nsHTMLContentSink.cpp around
line 4129.
 

Comment 18

[rpotts (gone)]

This is a sweet bug :-)

It looks like in navigator we just called atol(...) on the substring from the 
opening quote to the semicolon.

Now, we seem to verify that every character of the string is actually an 
integer.

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/layout/html/document/src/nsH
TMLContentSink.cpp#4182

It seems like this strictness breaks backward compatability :-(

-- rick

Comment 19

[Andreas Otte]

Attachment: Patch to fix the . problem

Comment 20

[Gagan]

seems like HTMLContentSink has the problem. to harishd for investigation. Am 
leaving the nomination to be assessed again based on severity/risk of the fix. 

Comment 21

[Gagan]

ah macros! :)

Comment 22

[Nisheeth Ranjan]

Marking nsbeta3+.  Harish, please check in the patch provided by Andreas after 
you've reviewed it.  Thanks.

Comment 23

[Peter "jag" Annema]

Hmmm, the http-equiv=refresh content=".X; url=URI" should've been a seperate bug
report shouldn't it?
 
I've been working on rewriting that whole parsing logic (a merge conflict
brought me here), clean it up a bit and allow some more "bad" cases... I'll see
how I can work Andreas' patch into what I have currently. See bug 45143 for
more.

Comment 24

[John Morrison]

Yes, this bug got hijacked :-/  The original problem is as described by gagan
on Jan 07, and what the Summary of the bug still says. 

Comment 25

[harishd]

I checked in Anderas patch ( slightly modified ) yesterday to fix mozillazine.

After patching http://www.mozillazine.org/resources/ got redirected properly to
http://www.mozillazine.org/resources/recommendations1.html.

Comment 26

[harishd]

Peter, since you've been working on related issues would you mind owning this 
bug? Thanx

Comment 27

[Peter "jag" Annema]

harishd, I'd gladly take over the mozillazine refresh bug (can now be found
under bug 49476) from you, I've got a fix in hand :-) Not sure on the original
bug though.

Comment 28

[harishd]

Peter, I've already checked in a fix for mozillazine refresh bug.  Are you 
saying that you have a revised patch to checkin?  

Comment 29

[Peter "jag" Annema]

"Are you saying that you have a revised patch to checkin?"

Yes and no. In fixing bug 45143 I ended up rewriting that section in
nsHTMLContentSink.cpp to which you recently checked in the fix for the
mozillazine bug, so I've had to make your checkin (Andreas' fix) part of my
rewrite. I'll be in #mozilla for the next few hours, so if I'm still unclear
perhaps we can more easily discuss in there?

Comment 30

[Gagan]

this is a dup 

*** This bug has been marked as a duplicate of 48200 ***

Comment 31

[Cesar Eduardo Barros]

Sure you mean the opposite right? 22886 is much older than 48200.

Comment 32

[Jesse Ruderman]

i think this isn't a dup of bug 48200.  bug 48200 is a recent regression.

Comment 33

[Diego]

However now that bug 48200 is fixed, I see fixed this bug too.
Linux, nightly 2000090208.

Comment 34

[Scott Brodmerkle]

Verified dupe. I also see this now working under Linux build 2000102021