397803 - Add test IDN TLDs to whitelist

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Gervase Markham [:gerv]]

ICANN are starting a test of IDN in the root. 
http://icann.org/announcements/announcement-2-19jun07.htm
This involves adding 11 test TLDs to the root, so software can be tested to see if it works with them.

We should add these 11 "test" TLDs to our IDN whitelist, to allow Firefox users to participate fully in the test. Once the TLDs are removed from the root at the end of the test period, we can remove them again.

The TLDs are the word "test" translated into the 11 languages in question. (I won't paste the characters in here.) Obviously, we need to add them to the whitelist in their punycode forms.

Arabic:               xn--kgbechtv 
Chinese, simplified:  xn--0zwm56d 
Chinese, traditional: xn--g6w251d 
Greek:                xn--jxalpdlp 
Devenagari:           xn--11b5bs3a9aj6g 
Japanese:             xn--zckzah 
Korean:               xn--9t4b11yi5a 
Persian:              xn--hgbk6aj7f53bba 
Russian:              xn--80akhbyknj4f 
Tamil:                xn--hlcj6aya9esc7a 
Yiddish:              xn--deba0ad

Gerv

Comment 1

[Gervase Markham [:gerv]]

Attachment: Patch v.1

Here's the patch. As this is slightly out of the ordinary, requesting rubber-stamp review from biesi.

Gerv

Comment 2

[Christian :Biesinger (don't email me, ping me on IRC)]

Hm... does the patch work? Reading some code in nsStandardURL.cpp, it seems like it doesn't really handle IDN TLDs correctly, as some code would expect the UTF-8 TLD, while other code would expect it to be in punycode.

Comment 3

[Gervase Markham [:gerv]]

Biesi: I don't deny there may be bugs in our handling of these :-) I tested this by setting the prefs in my own Firefox 2, and it looks like the URL bar works, but the status bar may not yet. Hey, part of the point of a test is testing client software, including ours :-) 

But if you are concerned, we could start with the trunk, and then decide later whether we want to push these in a 2.0.0.x update, like we do with other updates to the whitelist.

Gerv

Comment 4

[Christian :Biesinger (don't email me, ping me on IRC)]

did you enter the punycode version in the URL bar, or the non-ASCII one?

Comment 5

[Gervase Markham [:gerv]]

Test procedure:

Add the following to /etc/hosts or equivalent, all on one line:

212.58.253.71 xn--mgbh0fb.xn--kgbechtv xn--fsqu00a.xn--0zwm56d xn--fsqu00a.xn--g6w251d xn--hxajbheg2az3al.xn--jxalpdlp xn--p1b6ci4b4b3a.xn--11b5bs3a9aj6g xn--r8jz45g.xn--zckzah xn--9n2bp8q.xn--9t4b11yi5a xn--mgbh0fb.xn--hgbk6aj7f53bba xn--e1afmkfd.xn--80akhbyknj4f xn--zkc6cc5bi7f6e.xn--hlcj6aya9esc7a xn--fdbk5d8ap9b8a8d.xn--deba0ad

This maps a bunch of test URLs to www.bbc.co.uk. Then, enable IDN for the test TLDs by pasting the following into your prefs.js with Firefox closed:

user_pref("network.IDN.whitelist.xn--0zwm56d", true);
user_pref("network.IDN.whitelist.xn--11b5bs3a9aj6g", true);
user_pref("network.IDN.whitelist.xn--80akhbyknj4f", true);
user_pref("network.IDN.whitelist.xn--9t4b11yi5a", true);
user_pref("network.IDN.whitelist.xn--deba0ad", true);
user_pref("network.IDN.whitelist.xn--g6w251d", true);
user_pref("network.IDN.whitelist.xn--hgbk6aj7f53bba", true);
user_pref("network.IDN.whitelist.xn--hlcj6aya9esc7a", true);
user_pref("network.IDN.whitelist.xn--jxalpdlp", true);
user_pref("network.IDN.whitelist.xn--kgbechtv", true);
user_pref("network.IDN.whitelist.xn--zckzah", true);

Then try the links in the attachment (forthcoming).

You are right, the right hand column of links doesn't set the URL bar correctly. Bugs to fix :-)

Gerv

Comment 6

[Gervase Markham [:gerv]]

Attachment: Test HTML file of IDN links

Comment 7

[Christian :Biesinger (don't email me, ping me on IRC)]

Comment on attachment 282588 [details] [diff] [review]
Patch v.1

ok then, r=biesi, but do file a bug about making this work

Comment 8

[Gervase Markham [:gerv]]

Fixed on trunk.

Checking in modules/libpref/src/init/all.js;
/cvsroot/mozilla/modules/libpref/src/init/all.js,v  <--  all.js
new revision: 3.696; previous revision: 3.695
done

Once a nightly is available with this change, I'll do a test, file bugs on the broken bits, and put the bug numbers here.

Gerv

Comment 9

[Gervase Markham [:gerv]]

Comment on attachment 282588 [details] [diff] [review]
Patch v.1

OK, so I didn't read _all_ the checkin rules, just the state of "Open" and its definition. Sorry, my bad :-( <blush> Can I get retroactive approval for this patch?

Gerv

Comment 10

[Gervase Markham [:gerv]]

Better reopen this, just in case the approval queries miss it otherwise.

Gerv

Comment 12

[[e-mail]]

bug 409614 is about making punycode/UTF8 lookups work consistently.