12277 - URL: space encoding

Status: VERIFIED FIXED

(Core :: Networking, defect, P5)

Description

[Igor Furlan]

I will try my best to describe this "BUG" ( it is a bug...at least from my
perspective). OK here it goes:

In our company we run INTRANET, to my regret, on M$ IIS (NT4.0). We HAVE to use
MSIE5 browser. Most of the users run their application on NT workstations.
Engineering department uses Solaris ULTRA 10 workstations. Personally  I use
NS4.51 on my Solaris box. NS4.51 has a problem finding/decoding certain HTTP
addresses.
I hear you now saying: OK..  but this is NS4.51 problem and not Mozilla M8
problem!!!

NOPE !!  far away from the truth !!

It is Mozilla problem also. It behaves exactly in the same way... therefore...if
you can fix this problem in Mozilla, the  future NS5.0 and Mozilla
M9/10/11/or_whatever_number WILL BE BETTER and it can be used in company's
INTRANETs

OK. Let me try now to explain what is going on:


A) if I click on the following INTRANET URL with M$IE4/5 I DO GET requested page
on the screen.

http://10.1.1.25/strategy/New%20Product%20Launch/NPL timelines/cartesian.htm

I did notice that M$IE4/5 automagicaly replace "space" between "NPL" and
"timelines" with "%20". Who is doing this job,I  do not know. M$ browser itself
? or  M$ IIS (NT web server) recognize M$IE4/5 as a client and "automagicaly"
adds hex code for "space" ?? My guess is that browser does the job without a
help of the WEB server. The corrected URL looks like this:

http://10.1.1.25/strategy/New%20Product%20Launch/NPL%20timelines/cartesian.htm

B) if I do the same test with NS4.51 or Mozilla M8 I do not get requested page
to my screen. If I correct URL manualy then M8 and NS4.51 work OK.


One possible solution is to convince webmasters NOT to use "space" in the
file/directory structure. But, you can imagine, THIS solution will not fly.

So, better solution is convince browser to add "automagicaly"  hex code for
space (%20) into URL address BEFORE the URL is send to the server. I did try on
M$IE4/5 to insert two and more "spaces" into URL. And what was the result ?
M$IE4/5 replaced "spaces" with a string of hex codes (%20%20%20....  etc).

Can be this done in Mozilla ???

Sincerely

Igor

P.S.: I am attaching "screenshots" of M8 and M$IE5 generated pages.

Comment 1

[Igor Furlan]

Attachment: URL seen by M8 after clicking on LINK

Comment 2

[Igor Furlan]

Attachment: URL page seen by M8 after URL was corrected manualy

Comment 3

[Igor Furlan]

Attachment: URL page seen by M$IE5

Comment 4

[leger]

phillip, do you have problems with these pages on latest Linux?

Comment 5

[troy]

This is not a layout engine problem...

Comment 6

[Igor Furlan]

Attachment: additional explanation of the "bug"

Comment 7

[Gagan]

this is a simple thing to do. any volunteers?

Comment 8

[Igor Furlan]

Attachment: more details -- please read them

Comment 9

[Andreas Otte]

I think, what should happen is, that we should move to PRUnichar in
nsStdURL.cpp (URI/URL). See bug 13453. Then we need additional functions like
GetEncodedSpec and GetEncodedPath which translate the unichar-representation to
char by encoding it properly. This functions could then be used to build the
HTTP-Request. This would not only catch this bug, but also most off all the
other encoding bugs (bug 10373 and it's dups). Of course, we could do the
encoding function first and then move later to PRUnichar.

Comment 10

[Gagan]

actually all calls to create a URI should first Escape the strings being passed.
That is the only consistent way to make sure we don't get spaces in the strings.
This needs to happen in the docloader/webshell area (and whoever else is
creating URLs)

Comment 11

[Andreas Otte]

I think it would be better to do the encoding in nsStdURL while parsing the URL.

Comment 12

[leger]

Bulk move of all Necko (to be deleted component) bugs to new Networking

component.

Comment 13

[leger]

PDT+

Comment 14

[leger]

We take it back. 4.x bugs are not b1 stoppers. PDT-

Comment 15

[Andreas Otte]

Could the reporter please try again in his intranet with the latest versions of
mozilla. We did some work on urlencoding.

Comment 16

[Andreas Otte]

This should work for more than a month now, [e-mail] if you still see
this problem please reopen the bug.

Comment 17

[Igor Furlan]

Attachment: good job !! Mozilla has one bug less !!

Comment 18

[Igor Furlan]

Attachment: netscape4.61 is still confused

Comment 19

[Igor Furlan]

GREAT JOB  'mozilla guys and gals' !!

I have checked, finally, the latest and the greatest mozilla milestone (M14).
Yes, I know, Andreas asked very politely 2 times if I can verify the "FIX".
The main reason why I did not checked it is very simple. I was still running
kernel 2.0.36. Mozilla >= M10 DOES NOT run on old kernel. Today I have upgraded
my box to 2.2.12 and I can say: CONGRATULATIONS !!  This bug is not there
anymore !!!

I can use M14 to browse company INTRANET, which, unfortunately, runs on WIN NT.

in between, I did find 2 additional bugs which I plan to submit today or
tomorrow.

Sincerely

Igor

P.S.: I have attached 2 screenshots. One is Mozilla M14, which works great and
the other is the same URL seen by Netscape4.61. As you can see, Netscape4.61 is
totaly confused with that specific URL. Mozilla M14 does the job perfectly !!

Comment 20

[Tom Everingham]

verified by reporter

Comment 21

[Igor Furlan]

Yes, it is hard to believe.. but I have to RE-OPEN this OLD bug.

Here is why:

test case 1:
- please click on this URL: http://www.emulators.com/pentium4.htm
- click on link "Processor Basics"
RESULT: nothing happens

test case 2:
- Start Netscape 4.x
- click on this URL: http://www.emulators.com/pentium4.htm
- click on link "Processor Basics"
RESULT: we are able to jump to the target "Processor Basic"

test case 3:
- repeat test case 2 with MICROSOFT Internet Explorer
RESULT: it works OK

Here is the explanation:
M18 replaces every "space" with "%20". This is OK for all normal (aka regular
URLs). This was the original bug. It is fixed. No problem with this.

BUT it is not OK if the "space" is part of the "target". In this case "space"
MUST remain "space".

Example:
this URL
http://10.1.1.25/strategy/New Product Launch/NPL timelines/cartesian.htm

is corectly automaticaly converted to
http://10.1.1.25/strategy/New%20Product%20Launch/NPL%20timelines/cartesian.htm

however

this URL:
http://www.emulators.com/pentium4.htm#Processor Basics

SHOULD NOT be converted to
http://www.emulators.com/pentium4.htm#Processor%20Basics

Mozilla does it exactly this. It is WRONG.

If the "space" is part of "target", it should not be replaced with "%20".
Target starts always with "#", therefore it should not be that difficult
to fix this bug.

Sincerely

Igor

Comment 22

[Andreas Otte]

Sorry wrong! There are no spaces allowed in urls. No way around that. It seems
to me the target/ref is not properly unescaped before being used by the browser.
I suggest you close this bug and open a new one with the bug description,
because this is clearly something else.

Comment 23

[Igor Furlan]

Andreas,

I do not agree with you. Here is why :

To my (and many others too) regret, M$ started to use spaces in their URL
addresses.
They do not use spaces in URLs on INTERNET but on INTRANET only.
At least I hope so. When I filed bug 12277 many many months ago I was trying
to use M8 on our company INTRANET. We, engineers, were forced to use
M$ Internet Explorer on INTRANET. I said to my self and my colleagues:
There must be another way. Let us try Mozilla.

Unfortunatelly, Mozilla M8 was not able to show correctly INTRANET web pages
prepared/created with and for M$ IE. You like it or not, this is 
fact of the life. This is just another evidence of
hooks, created by M$ which force users to use only M$ products.

Please see: 

URL seen by M8 after clicking on LINK
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=1350

URL page seen by M8 after URL was corrected manually
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=1351

URL page seen by M$IE5
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=1352

If we want that corporations continue or, even
better, start to use Mozilla instead of MS IE, we have to
fix this bug.

So, bottom line question is:
Do we want to use Mozilla on company's INTRANETs, which
are driven by M$ products (MS IIS, IE, FRONTPAGE) or not  ?

I think that the answer is a BIG YES,

Yes, the bug was fixed in M12, if I recall correctly.
Please see the screenshot:
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=6710


M$ IE automatically replaces spaces with %20. Netscape 4.x does not
do any space replacement, therefore it is useless
on INTRANETS, where the files are created by M$ inventions.
( you know... freedom to innovate )

Mozilla does the job correctly. Mozilla (and netscape 6.0) can
be used in intranets and on internet. so, where is the problem ??

The problem is that, M$ IE and NS4.x do the job 
correctly on this simple test

- please click on this URL: 
  http://www.emulators.com/pentium4.htm
- and then click on link "Processor Basics"

Mozilla DOES NOT DO IT correctly ! Try it by your self
and compare the results. Mozilla replaces ALL
the spaces with %20, M$ IE replaces spaces with %20 only
in places where it is necessary. It does not replace space
with %20 in target/ref.

So, my assumption is that the programer, who fixed original bug,
did not test URL parsing engine with target/ref containing space.

Therefore I think that this bug is still the same bug and
there is NO NEED to open another a new one.

Comment 24

[Igor Furlan]

Mozilla 0.8 (LINUX)  works OK. 

Comment 25

[benc]

VERIFIED: Mozilla 1.1, Mac OS X.
We do this now. It isn't strictly conformant w/ RFC 2396's ideas on whitespace
in a URL, but it makes sense.

If this ever regresses again for something typed in the URL bar, file a bug in
URL bar. If this regresses on links, file a bug in Networking and include a
source code snippet.