381344 - no way to whitelist IDN for non-qualified hostname (e.g. localhost)

Status: RESOLVED WORKSFORME

(Core :: Networking, defect)

Description

[Tim]

User-Agent:       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3

IDN doesn't work on local hostnames.

Define a local hostname ø (punycoded as xn--pda):

127.0.0.1  xn--pda

When ø is entered in the address bar, FireFox correctly resolves this to "http://xn--pda/" and connects to the localhost HTTP server on port 80. However, it does not show "http://ø/" in the address bar; it shows the punycoded version.

I've tried adding "network.IDN.whitelist.localhost=true" to about:config, but this makes no difference.

Reproducible: Always

Steps to Reproduce:
1.Start a web-server on localhost:80

2.Define a local hostname ø:
In etc\hosts, add the following entry (xn--pda is punycoded ø):
127.0.0.1  xn--pda

3. Refresh DNS cache
ipconfig /flushdns

2.Enter ø in the FireFox address bar

Actual Results:  
The address bar shows the punycoded URL 'http://xn--pda/'

Expected Results:  
The address bar should show the IDN URL 'http://ø/'

Comment 1

[:Gavin Sharp [email: [e-mail]]]

The whitelist check fails if the host doesn't contain a dot:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/base/src/nsStandardURL.cpp&rev=1.97#909

Comment 2

[Tim]

Hi chaps,

A quick thought related to this issue...

Isn't it excessively paranoid to require users to add their 'localhost' domain to the safe whitelist? Why not allow IDN on the 'localhost' domain by default and use a blacklist in the rare case that the user prefers ASCII to Unicode?

Comment 3

[Tim]

This is based on the current implementation but should work with a host name that is a domain (e.g. localhost):


nsStandardURL::IsInWhitelist(const nsCSubstring &host)
{
    if (!gIDNWhitelistPrefBranch)
        return PR_TRUE;

    PRInt32 domain_pos = nsCAutoString(host).RFind(".");
    if (domain_pos == kNotFound)
        domain_pos = 0;
    else
        domain_pos += 1;

    return gIDNWhitelistPrefBranch->
        GetBoolPref(nsCAutoString(Substring(host, domain_pos)).get());
}


It ain't pretty and could really use a function to get the domain part from a host name, which must surely already exists somewhere in FireFox?

Comment 4

[Tim]

...actually, why not just this:


nsStandardURL::IsInWhitelist(const nsCSubstring &url)
{
    return gIDNWhitelistPrefBranch->GetBoolPref(url);
}

Comment 5

[:Gijs (he/him)]

This seems to be working in Firefox 31, at least...

Comment 6

[Tim]

I've just retested the exact steps I originally listed back in 2007 using Firefox Version 35.0.1916.153 m and I'm still not able to connect to a localhost webserver called 'ø' by typing 'http://ø/' in the address bar.

In fact the situation is a little worse than it was 7 years ago because the address bar seems to pass the entered text on to the registered search engine instead of first checking to see if the name can be resolved as a web-site address.

Comment 7

[:Gijs (he/him)]

(In reply to Tim from comment #6)
> I've just retested the exact steps I originally listed back in 2007 using
> Firefox Version 35.0.1916.153 m

This isn't a Firefox version number. It looks a lot like a Google Chrome version number, however...

Comment 8

[Tim]

My apologies. I retested this on the wrong browser.

Comment 9

[:Gijs (he/him)]

(In reply to Tim from comment #8)
> My apologies. I retested this on the wrong browser.

OK. But if you test on Firefox, can you confirm that it works for you, too? :-)

Comment 10

[Tim]

 I've just retested the exact steps I originally listed back in 2007 using Firefox Version 30.0 and I'm still not able to connect to a localhost webserver called 'ø' by typing 'http://ø/' in the address bar.

In fact the situation is a little worse than it was 7 years ago because the address bar seems to pass the entered text on to the registered search engine instead of first checking to see if the name can be resolved as a web-site address.

NOTE 1: If I type 'localhost' in the address bar, things work. If I type 'ø' in the address bar, I get google search results for 'ø'. If I type 'http://ø/' in the address bar, it goes off looking for 'www.ø.com'.

NOTE 2: The character 'ø' is a Danish character. I'm using this to test IDN handling. I have the following lines in my 'hosts' file:

127.0.0.1       localhost
127.0.0.1       ø
127.0.0.1       xn-pda

Comment 11

[:Gijs (he/him)]

(In reply to Tim from comment #10)
>  I've just retested the exact steps I originally listed back in 2007 using
> Firefox Version 30.0 and I'm still not able to connect to a localhost
> webserver called 'ø' by typing 'http://ø/' in the address bar.
> 
> In fact the situation is a little worse than it was 7 years ago because the
> address bar seems to pass the entered text on to the registered search
> engine instead of first checking to see if the name can be resolved as a
> web-site address.
> 
> NOTE 1: If I type 'localhost' in the address bar, things work. If I type 'ø'
> in the address bar, I get google search results for 'ø'. If I type
> 'http://ø/' in the address bar, it goes off looking for 'www.ø.com'.
> 
> NOTE 2: The character 'ø' is a Danish character. I'm using this to test IDN
> handling. I have the following lines in my 'hosts' file:
> 
> 127.0.0.1       localhost
> 127.0.0.1       ø
> 127.0.0.1       xn-pda

Hrm. I wonder if this is OS-specific now; it's fixed on OS X, but I'll need to retest on Windows. Setting needinfo so I get around this tomorrow or tonight.

Comment 12

[Tim]

NOTE 3: I should add that if I type 'xn-pda' in the address bar, which is 'ø' in punycode, it does connect to the web-server on localhost. However, if I type the IDN name 'ø' in the address bar, it doesn't seem to even try to resolve it as an address but sends it off as a google search.

Comment 13

[:Gijs (he/him)]

(In reply to Tim from comment #12)
> NOTE 3: I should add that if I type 'xn-pda' in the address bar, which is
> 'ø' in punycode, it does connect to the web-server on localhost. However, if
> I type the IDN name 'ø' in the address bar, it doesn't seem to even try to
> resolve it as an address but sends it off as a google search.

So I just retested on Windows 8 with Firefox 31 (released this week), without a server running.

I added both ø and xn-pda to my hosts file.

I then get an error that Firefox can't establish a connection to the server at ø, which is correct... I don't get a search query.

Can you reproduce this with a clean profile? ( https://support.mozilla.org/kb/profile-manager-create-and-remove-firefox-profiles )

I'm really not sure why this would be working for me but not for you...

Comment 14

[Tim]

I updated to Firefox 31 and reset the profile (as suggested). I also did a ipconfig /flushdns.

If I type xn-pda in the address bar, I get an 'Unable To Connect message'. This is good because it's trying to resolve a URL.

If I type ø in the address bar, it goes off to Google and comes back with a load of search results. Not so good because it's not trying to resolve ø as a URL. Note that ø is the Danish character, not zero.

I should say that I'm running XP, have no HTTP server running on the local machine but do have ø and xn-pda in my 'hosts' file.

I should also say that when I reset my profile, FIrefox went and re-imported history, cookies and other things.

Comment 15

[:Gijs (he/him)]

(In reply to Tim from comment #14)
> I updated to Firefox 31 and reset the profile (as suggested). I also did a
> ipconfig /flushdns.
> 
> If I type xn-pda in the address bar, I get an 'Unable To Connect message'.
> This is good because it's trying to resolve a URL.
> 
> If I type ø in the address bar, it goes off to Google and comes back with a
> load of search results. Not so good because it's not trying to resolve ø as
> a URL. Note that ø is the Danish character, not zero.
> 
> I should say that I'm running XP, have no HTTP server running on the local
> machine but do have ø and xn-pda in my 'hosts' file.
> 
> I should also say that when I reset my profile, FIrefox went and re-imported
> history, cookies and other things.

Hm. One thing I noticed was encoding - if I open my hosts file with gvim afterwards, it, errr, gets very sad. It was saved using notepad and "ansi" (Oh, Microsoft...) encoding.

Have you doublechecked that you're not using utf8 or something else that everyone and their dog would find sensible in 2014, but not MS (especially not on XP) ? :-)

Comment 16

[Tim]

I just checked the 'hosts' file. It was UTF-8 but I get the same behaviour when it is saved as UTF16-LE or ANSI.