84409 - URL: relative URLs break if if base URL has ";" character

Status: RESOLVED FIXED

(Core :: Networking, defect, P4)

Description

[Roshan Ail]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9) Gecko/20010505
BuildID:    2001050515

Mozilla seems to return an incorrect URL when attempting to access page
components (images etc.) which are specified by their relative locations, if the
page URL contains a ";" character.

For example, while accessing a web page located in a directory on a server where
the directory name contains a semi-colon, the requests recieved from Mozilla
seem to skip the directory name. (Please refer to the 'Steps to Reproduce' and
'Additional Information' sections.)

Steps to setup Environment:
1. In your webserver's document root directory, create a temporary directory
whose name includes a semi-colon (;). e.g 'Test;Folder'.

2. In this folder, place a simple HTML file (say 'URLTest.html') which among
other things accesses an image (I used a .gif file) in this folder. Note that
the image must be accessed with its relative URL. e.g '<IMG SRC="6jelly.gif"
HEIGHT=300 WIDTH=400>'.

3. Now load the page in Mozilla by entering the following in the address bar
'http://<hostname>/Test;Folder/URLTest.html'.


The page should show up with the image. (This seems to work with IE 5.5 and
Netscape 4.08)


Webserver Log Entries for requests:

From IE 5.5:
192.168.55.249 - - [06/Jun/2001:18:29:46 -0700] "GET /Test;Folder/URLTest.html
HTTP/1.1" 200 195
192.168.55.249 - - [06/Jun/2001:18:29:46 -0700] "GET /Test;Folder/6jelly.gif
HTTP/1.1" 200 245396

From Netscape Navigator 4.08:
192.168.55.249 - - [06/Jun/2001:18:30:34 -0700] "GET /Test;Folder/URLTest.html
HTTP/1.0" 200 195
192.168.55.249 - - [06/Jun/2001:18:30:35 -0700] "GET /Test;Folder/6jelly.gif
HTTP/1.0" 200 245396

From Mozilla M18:
192.168.55.249 - - [06/Jun/2001:18:21:15 -0700] "GET /Test;Folder/URLTest.html
HTTP/1.1" 200 195
192.168.55.249 - - [06/Jun/2001:18:21:17 -0700] "GET /6jelly.gif HTTP/1.1" 404 323

Note that the 'Test;Folder' part of the URL has been skipped in the request for
the image made by Mozilla.

Reproducible: Always
Steps to Reproduce:
1. In your webserver's document root directory, create a temporary directory
whose name includes a semi-colon (;). e.g 'Test;Folder'.

2. In this folder, place a simple HTML file (say 'URLTest.html') which among
other things accesses an image (I used a .gif file) in this folder. Note that
the image must be accessed with its relative URL. e.g '<IMG SRC="6jelly.gif"
HEIGHT=300 WIDTH=400>'.

3. Now load the page in Mozilla by entering the following in the address bar
'http://<hostname>/Test;Folder/URLTest.html'.


Actual Results:  The browser is not able to resolve the url for the image.
It just doesn't show the image.

Expected Results:  The html file should be displayed as it is designed.
i.e I would expect the image to be shown

For that matter any other relative URLS in it should be shown

Comment 1

[Andreas M. "Clarence" Schneider]

This works if you use "%3B" instead of ";" in the URL as you should if it is
part of a file/directory name and not a separator for parameters.

Nevertheless this is a bug in Mozilla.
We seem to treat ";" as a terminator of the URI if it used as a base URI.
some.gif relative to http://host/x/a;b/ becomes http://host/x/some.gif ,
same for some.gif relative to http://host/x/a;b/y/ .

Confirming (2001-06-06-04-trunk, Win NT).
-> Networking

Comment 2

[Andreas M. "Clarence" Schneider]

A quick look at nsStdURLParser makes me think our URL parsing is completely
broken with respect to parameters. We seem to assume parameters can only occure
in the final path segment and only one parameter is possible.

RFC 2396:

3.3. Path Component

   The path component contains data, specific to the authority (or the
   scheme if there is no authority component), identifying the resource
   within the scope of that scheme and authority.

      path          = [ abs_path | opaque_part ]

      path_segments = segment *( "/" segment )
      segment       = *pchar *( ";" param )
      param         = *pchar

      pchar         = unreserved | escaped |
                      ":" | "@" | "&" | "=" | "+" | "$" | ","

   The path may consist of a sequence of path segments separated by a
   single slash "/" character.  Within a path segment, the characters
   "/", ";", "=", and "?" are reserved.  Each path segment may include a
   sequence of parameters, indicated by the semicolon ";" character.
   The parameters are not significant to the parsing of relative
   references.

Comment 3

[neeti]

benc: could you set up a testcase for this bug

thanks,
neeti

Comment 4

[doctor__j]

Making a local testcase for this is simple...

1. Create a directory like C:\TEST\FOO;BAR\
2. Place an HTML in this directory, say C:\TEST\FOO;BAR\hello.html
3. Open browser and type "C:\TEST\FOO;BAR\hello.html" on the URL bar.
Actual Results: Browser cannot open this file...

But if you use "File -> Open" method to load that file, the ";" would be escaped
into "%3B" and you can open it successfully.

Comment 5

[benc]

The test case is presented pretty clearly. (Thanks rail & doctor!)

Do you mean you need this loaded on a system? Probably loading this on an NT
system would be ideal, my flagship server is Solaris, and ";" means "end of
shell command in UNIX.

If this is what you want, I'll put it on my list of things to do.

Comment 6

[doctor__j]

I'm not sure if *NIX would accept a directory name containing ";".
I can certainly create such directory in Win98.

Comment 7

[Roshan Ail]

I am using an NT system.
So I guess we would have to check it on an NT system

However, It is possible to create a folder under Solaris that contains the ; 
character by escaping it with a \

You can say mkdir test\;folder

Thanks

Comment 8

[Andreas M. "Clarence" Schneider]

I have set up small testcases at
http://clarence.de/tests/x;y/test.html and
http://clarence.de/tests/x;y/x/test.html , but I can't give you access to my
server logs. To compare the escaped URLs:
http://clarence.de/tests/x%3By/test.html and
http://clarence.de/tests/x%3By/x/test.html .

Comment 9

[Andreas Otte]

If I remember correctly it was decided to not support params inside the
directory part of the path, because nobody has ever used it that way and it is
not easy to implement.

But, maybe we have to look at the detection alogrithm that decides wether a ;
starts the param or is part of the directory name, because there are one or more
/ right of it in the string. I will look into that.

Comment 10

[Andreas M. "Clarence" Schneider]

If we parse URIs accordingly to RFC 2396 (as opposed to the obsolete RFC 1808)
we dont't need to care about parameters at all. We can treat ';' like any
other char. It's only the server's responsibility to care about parameters.

Comment 11

[Andreas Otte]

Attachment: patch to parse params differently

Comment 12

[Andreas Otte]

I've attached a patch to get these kind of urls parsed conforming to RFC 2396. I
stayed with the param component, but we now ignore ; when there is a slash after
it and it is not in the query or ref part.

Comment 13

[Darin Fisher]

-> gagan: can you review this patch?

Comment 14

[Andreas Otte]

This patch needs heavy testing on a number of test urls. I will look into my old
stack of fixed urlparser bugs to test it against some of the finest ;-) examples
of urls that use params. For example: are there params that have a / inside it,
that could be a killer ...

Comment 15

[Andreas Otte]

Attachment: more complete patch, also changes escaping of ; and / and handling of ; in CoaleseDirs

Comment 16

[Andreas Otte]

Attachment: fixed patch, tests showed it was not finding the most left ; before the first /

Comment 17

[rpotts (gone)]

hey Andreas,

I "think" the patch looks fine...  Its one of those changes where its correct 
if its been tested and doesn't break anything :-)

r/sr=rpotts

Comment 18

[Andreas Otte]

Thanks, Rick! I've gone through the old bugs, done the precheckin tests and
haven't seen any problems ( after the latest update ;-) ).

Comment 19

[Andreas Otte]

fix is checked in