80313 - Space before colon in HTTP header invalidates it

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect)

Description

[Ivo Clarysse]

On the presented URL (Belgian Railroad timetables), enter ie. From: 'Mechelen'
and To: 'Leuven' and click on 'search'.

The resulting HTTP request returns an 'HTTP/1.1 302 Object moved' response,
the subsequent request yield an HTTP object with Content-type: text/html, yet
Mozilla treats this as application/octet-stream and displays a 'Save as...' popup.

Comment 1

[Andreas M. "Clarence" Schneider]

This occures even without redirection:
http://193.121.180.17/scripts/Pcgittb.exe?langue=3&idO_r=1000810&idD_r=1000715&idV_r=0&hh=22&mn=55&j=11&m=5&a=2001&DoA=0&reseau=sncb&nombre=0

HTTP/1.1 200 OK
Date: Fri, 11 May 2001 21:16:34 GMT
Server: Microsoft-IIS/4.0
Content-Type: text/html

Confirming. OS -> all (I see this on Win NT).

Comment 2

[Andreas M. "Clarence" Schneider]

Bug 80544 is similar.

Comment 3

[Andreas M. "Clarence" Schneider]

I know now what the real problem is: The Content-Type-Header is not as listed
above, but "Content-type : text/html". Note the additional space.

AIUI, the URL will be fixed by the check-in for bug 80544, but the main problem
remains: The header is lost. Content-Type will be guessed by actual content.

I've made two testcases which send "Content-Type : text/plain":
http://clarence.de/test-cgi/nph-bug-80313.exe and
http://clarence.de/test-cgi/nph-bug-80313.gif . Besides the name they are
identical. ATM both fail; with tomorrow's builds I guess the first will work,
but the second will still fail.

Changed summary.

Comment 4

[Andreas M. "Clarence" Schneider]

Confirmed my assumption with 2001-05-16-04, Win NT.

Comment 5

[benc]

qa to me.

Isn't the " : " format a standards violation?

How common is this and should we support it?

Comment 6

[Andreas M. "Clarence" Schneider]

> Isn't the " : " format a standards violation?

No. RFC 2616 allows even "LWS" between the field name and the colon.

Content-Type
     :
             text/html

is a valid header. A single space before the colon exists in real web pages
(mostly CGIs). I do not know however how common this is. RFC 2616 recommends
a "common form" for headers (section 4.2).

I looked at the source and it seems like we aren't able to parse HTTP headers
that are not in "common form". AIUI even "folding" with LWS is not implemented
yet.

Comment 7

[Andreas M. "Clarence" Schneider]

My patch attached to bug 81008 fixes this.
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=35405

Comment 8

[Andreas M. "Clarence" Schneider]

I have split off the part fixing this bug from the patch and attach it here.
It includes better parsing for the Content-Type header, especially for the
charset parameter.

Changes to nsHttpResponseHead::ParseHeaderLine() :
- Allow whitespace between header name and colon
- Allow tabs after the colon

Changes to nsHttpResponseHead::ParseContentType() :
- Allow whitespace between mime type and semicolon
- Allow trailing whitespace other than a comment (BTW, comments are invalid
  in most headers, including Content-Type).
- Allow other parameters than "charset" (but ignore them)

CC'ing Darin.

My testcases have moved (for some reason a directory named "test-cgi" gets
cleared each day by my ISP). They are now at
http://clarence.de/testcgi/nph-bug-80313.exe and
http://clarence.de/testcgi/nph-bug-80313.gif . A more general testcase is at
http://clarence.de/tests/http-raw.html .

Comment 9

[Andreas M. "Clarence" Schneider]

Attachment: patch

Comment 10

[Darin Fisher]

the patch looks good to me.  r/sr=darin

Comment 11

[Darin Fisher]

-> me

Comment 12

[Darin Fisher]

neeti: can you review this patch when you get a chance? thx!

Comment 13

[neeti]

r=neeti

Comment 14

[Darin Fisher]

-> mozilla 0.9.1 (per approval from [e-mail])

Comment 15

[Darin Fisher]

*** Bug 82822 has been marked as a duplicate of this bug. ***

Comment 16

[Darin Fisher]

fix checked in

Comment 17

[Andreas M. "Clarence" Schneider]

*** Bug 61066 has been marked as a duplicate of this bug. ***

Comment 18

[Bradley Baetz (:bbaetz)]

VERIFIED on branch 2001060713 linux build. Leaving as resolved until the other
platforms are checked.

Comment 19

[John Unruh]

Verified on OSX and Win2000.