96519 - URL: escaping of ":"

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[ltskinol]

Linux 2001082208

The listed URL doesn't render in Mozilla at all.  I get a blank display.  Works
fine in NN4.7.

Note the ':' in the URL.  When Mozilla opens the page, this gets changed
to %3A.  Could this be the problem?

Comment 1

[timeless]

evang fodder?

Comment 2

[gavin long]

[confirming]

IE5.5 handles that URL and displayes the page, too.

A quick skim of RFC 1738 (secction 2.2) confirms (I think) that the colon ":" is
a reserved character in http URLs, and in this case, it's not being used properly.
Moz is doing the Right Thing, IMO, and IE is forgiving a pretty major mistake.

I believe that the http server should probably be able to cope with Moz sending
%3A in place of the colon (since that's an escaped colon) and serve the file anyhow.

-> Tech Evangelism

Comment 3

[Bradley Baetz (:bbaetz)]

A server MUST be able to cope with any character being escaped, IIRC, so this is
a server bug.

However, : is not reserved in the path. Hmm. Darin? Do we need to modify our
many url escaping routines?

-> default owner

Comment 4

[benc]

If the HTTP URL definitions do not reserve this character in the scheme, then we
should not encode it, based on my limited RFC 1738 only understanding of URLs.

I'm going to take this back until it is clear this escaping is the correct behavior.

Comment 5

[Gervase Markham [:gerv]]

Escaping is not optimal behaviour, but its not incorrect. I agree that we
shouldn't be doing it, though.

Comment 6

[Bradley Baetz (:bbaetz)]

Oops - that was me, using gerv's machine. Sorry.

Comment 7

[Katsuhiko Momoi]

I believe this is Mozilla's bug. RFC 1738 defines HTTP URL as follows:

; HTTP

httpurl        = "http://" hostport [ "/" hpath [ "?" search ]]
hpath          = hsegment *[ "/" hsegment ]
hsegment       = *[ uchar | ";" | ":" | "@" | "&" | "=" ]
search         = *[ uchar | ";" | ":" | "@" | "&" | "=" ]

where 

hostport       = host [ ":" port ]
uchar          = unreserved | escape
unreserved     = alpha | digit | safe | extra
escape         = "%" hex hex
reserved       = ";" | "/" | "?" | ":" | "@" | "&" | "="


So according to this RFC, reserved characters, e.g. ":" may be used 
in hpath.
And reserved characters may not be used without changing the
semantics of the URL when it is escaped. 

I have seen URL forwarding scheme used by some proxy server
take the form of:

http://www.tohoku.ac.jp:8081/=@=:www-student.ulis.ac.jp/html/virtual-library/

Nnote the use of the reserved character ":" in this proxy forwarding
scheme. This will not work if you escape it. There are probably other
uses of reserved character that is probably broken because of the 
current Mozilla behavior.

Comment 8

[benc]

Okay. If someone wants to take this up w/ excite, create a new bug and send it
to evangelism.

Comment 9

[Bradley Baetz (:bbaetz)]

Yes, but the spec also says tha the %xx form is equivalent to the 'normal' form,
when the character is not reserved. So we're not right, but they're wrong :)

Comment 10

[benc]

Sure. New bug for evanglism. I'm keeping this one b/c it's got all the good
analysis.

Comment 11

[neeti]

andreas: what do you think about this bug

Comment 12

[Andreas Otte]

According to rfc 2396 (which superceeds 1738) the colon is (although listed) not
reserved in the path component. Also it has no function in parsing the url
inside path. For that matter it can count as not reserved and is equivalent to
it's hexencoding. Excite should unescape it after parsing the path.

With the current parsing algorithm we have to escape a colon inside the filename
(which is part of the path) because otherwise we would mistake the colon in a
filename as indicator for an absolute url while resolving a relative url. We
definitly don't want that to happen. A possible solution might be to handle the
escaping of relative URLs differently than absolute URLs, but I don't think that
should happen unless absolutely necessary.

Comment 13

[Doug Turner (:dougt)]

moving neeti's futured bugs for triaging.

Comment 14

[Andreas Otte]

This will be fixed with the patch for bug 193477.

Comment 15

[Andreas Otte]

The given url is no longer valid, can't check against it, but this should be
fixed now with the checkin for bug 193477 (for 1.4a). Please reopen if this is
not correct.