452428 - Update Public Suffix List 2

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Gervase Markham [:gerv]]

This bug tracks a batch of updates to the Public Suffix List.

The flow from the initial registry mailing has died down. There is one where I got an initial email but no response to the verification - I'll post the data below so we can confirm it independently. There are also some suggestions in bug 447815, and I think we should remove the commented-out parts of the entries for .pa and .sa, because they will confuse people.

Gerv

<snip>
Hi,

I looked at the Public Suffix List and note that your entry for .AS (America
Samoa) is INCOMPLETE.

The correct list is as follows:

[---START---]
as
gov.as
[---END---]


Best Regards,

Stephen Deerhake
GDNS, LLC
AS Domain Registry

Comment 1

[Gervase Markham [:gerv]]

Attachment: Ideas for further updates

This is the output of a script I wrote which compares the Public Suffix List with this list: http://spamcheck.freeapp.net/two-level-tlds. It should give us some ideas about where to look to make further improvements in accuracy.

Gerv

Comment 2

[Pam Greene]

Where did they get their data, I wonder?

Comment 3

[Gervase Markham [:gerv]]

No idea. Someone pointed me at it on the DNSOP list. This means we can't just copy it, but we can use it for inspiration.

Gerv

Comment 4

[David Triendl]

Attachment: Patch based on Gerv's "Ideas for further updates"

Ok, thanks for the list Gerv. I just finished going through it. Most stuff is wrong, some entries are valid and I added them in my patch and some include pseudo-registries (like Internic), which I did not add. (If you want to, I can add them too). Some entries also seem to be valid but there is no way to verify them other than mailing the registry, because the registry homepage does not provide any information. I think it would be best to use a @publicsuffix.org or @mozilla.org mail address when asking them.
Gerv: I did not add the .al change yet, as I would like to use the same format as the other entries (the "Confirmed by registry ..." line).

co.ae, com.ae: The .ae domain was redelegated to .aeDA just a few weeks ago.
They reactivated the co.ae subdomain and remove pro.ae and name.ae. For a list
of possible subdomains, take a look at the "Domain Name Eligibility
Policy"[1][2]

edu.ai, gov.ai: The official registry[3] only mentions the already known
endings off.ai, com.ai, net.ai and org.ai. A google search for *.gov.ai[4] and
*.edu.ai[5] reveals no results that indicate a subdomain.

inima.al, soros.al, tirana.al, uniti.al: According to the wikipedia article[6]
these 4 domains were the only ones that were allowed at 2nd level
("grandathered", whatever that means). However, now it is possible to directly
register domains at the 2nd level.

upt.al: This is the homepage of some university, it does not seem like a
public suffix.

ac.am, co.am, com.am, edu.am, gov.am, int.am, mil.am, net.am, org.am, tld.am:
The offical registry policy[7] states, that domains with country codes or
other generic terms were not allowed until April 2007. I cannot find any
useful information on all those hostnames, but my best guess is, that they are
like Internic just some kind of reseller and don't belong the the official am
registry.

iris.arpa: According to Wikipedia[8] this 2nd level domain is used for CRISP
(the planned successor for the WHOIS protocol). As the registry confirmed the
list without the iris.arpa entry just two months ago, can you please ask them
if they can confirm iris.arpa?

belgie.be,dns.be, fgov.be: Neither the Wikipedia article[9] nor the registry
website[10] mentions these, but a google search reveals that belgie.be[11] and
fgov.be[12] have a lot of subdomains, while dns.be[13] only seems to host
www.dns.be. Can you contact the registry ([e-mail]) and ask them about it?

biz.bh, cc.bh, edu.bh, gov.bh, info.bh, net.bh, org.bh: Wikipedia[14] mentions
that registrations "are available directly at the second level, or at third
level beneath some second-level labels", but does not provide a list.
Furthermore I can't find any information about domain registrations on the
site[15] that the IANA links to.

dpn.br is not mentioned on the "dpn" list[16]. I think it is just a
misunderstanding (maybe someone used dpn.br as an example).

gc.ca is, according to Wikipedia[17] a "privately held" 2nd level domain for
the Government of Canada (Wtf? A privately held domain for the government), so
I added it to the list.

com.cd, net.cd, org.cd: I cant find any information about it on the official
registry page[18]. Can you contact them and ask them about it? They seem to be
pretty modern compared to other African registries.

com.ch, gov.ch, net.ch, org.ch don't appear anywhere on the official registry
page. Whois data reveals, that most are all registeret by different people and
not by an institution.

store.co: Does not appear like a eTLD to me. However, as the registry
submitted a list just 2 months ago, you may want to contact them again. The
official registry page is unreachable for me.

ac.com displays a VeriSign "Under Construction" page. Looks definitly not like
an eTLD.

au.com is similar to InterNic. They claim to be a registry but are just
selling subdomains. In my opinion such companies don't belong to such a list,
but if you consider the original purpose of the list (restricting cookies), it
might be worth adding it. However, I did not add it to my patch yet.

info.cu is not mentioned on the list[19] on the official registry page.

co.dk, along with biz.dk, firm.dk, store.dk and reg.dk is owned by dansk.net,
again a company similar to InterNic. I did therefore not add it to my patch.

edu.dm, gov.dm seem to be in use[20][21], but I cannot find any information
about them. I decided to add them to the patch, but the best thing would be to
contact the registry.

ass.dz is not mentioned on the official registry page[22] (list is included in
the dropdown menu in the formular). It seems to be just a type of asso.dz.

co.ee has many subdomains[23], but does not belong to the .ee registry,
therefore I did not include it in the list (it seems to be of the InterNic
type).

edu.ee, med.ee were missing on the list. I added these two and gov.ee,
riik.ee, lib.ee and aip.ee which were missing as well.[23]

aland.fi was the domain of an autonomous province of Finland and is currently
being phased out in favor of .ax. As there are still many domains under
aland.fi, I added it to the pach, with a reminder to check for updates.

barreau.fr is on the list of forbidden terms[24], but no eTLD. Looking at the
entry for fr in the list I found that fr had a duplicate entry which I removed
in my patch.

ac.gg, alderney.gg, guernsey.gg, ind.gg, ltd.gg and sark.gg may have
subdomains, but they are definitly not official, because they are not
mentioned on the registry homepage[25] and their whois records are different
than whois records for offical endings (e.g. org.gg)

asso.gp was missing in the list, as well as mobi.gp (see the dropdown menu on
the offical registry homepage[26]).

ac.hu, edu.hu, gov.hu, nui.hu are somewhat difficult: The Hungarian registry
confirmed our current list about 2 months ago, however this is only the list
of second level domains that the public can register. There is a second list
dubbed "protected names"[27]. They state, that some of the are just reserved
while others are in use for "special cases". Even worse, this list also
includes names that are registered before the introduction of the protected
names list and therefore in use. Please contact the registry about this issue,
as I don't know which protected name has which purpose.

gov.ie: According to Wikipedia[28], the Irish Government used gov.ie but is
now phasing it out. As it is still in use, I decided to add it to my patch.

lkd.co.im: I did not touch this entry yet, as there seems to be a problem in
our list: According to the source of our entry[29], only co.im, ac.im,
plc.co.im, org.im and ltd.co.im are possible, however our list has the
additional entries net.im, gov.im and nic.im.

ernet.in is not an official eTLD[30], but simple a domain used by the Indian
Educational Research Institutes (ERNET). I therefore did not add it.

gov.io, mil.io, net.io, org.io: The registry mentions that registrations at
third level are possible, but only provides com.io as an example. I can't find
a full list. I guess it would be best to contact them and ask for a list.
(see also: .sh, .vi)

ac.je, ind.je, jersey.je, ltd.je: Not offical (same reason and registry as
.gg)

gov.jp, kanazawa.jp, matsuyama.jp, net.jp, org.jp, takamatsu.jp, utsunomiya.jp
are reserved but not in use (compare whois records for gov.jp with the whois
record of an in-use label like ac.jp)

kyonggi.kr is not listed on the official registry page.[31]

mil.lb is neither in use[32] nor in the list submitted by the registry about
two months ago.

net.lc was missing, as well as co.lc.[33]

com.li, gov.li, net.li, org.li are all owned by a dubious company called
"Krypton Internet Services" and definitly not effective TLDs.

ac.lk is not listed on the official registry page.[34]

gov.lt is a separate zone similar to gc.ca.[35] I added it to my patch.

mil.lt is a simple domain registered by the Lithuanian Air Force. The only
active domain is the air force homepage[36], therefore I did not add it to my
patch

gov.lu, mil.lu, net.lu, org.lu are definitly no eTLDs. gov.lu redirects to the
government site, mil.lu is a domain kiting/tasting website, net.lu is an
internet portal (think of AOL for Luxembourg) and org.lu is a private
collection of some random organisations.

biz.ly is not listed on the official registry page.[37]

army.mil, navy.mil: Wikipedia[38] mentions that there is some kind of hierachy
with registrations at 3rd level or higher, but I can't find any info and the
DoD does not allow me to visit nic.mil.

com.mn, museum.mn: I can't find any information about .mn subdomains other
than what is listed on Wikipedia.

music.mobi, weather.mobi are just two random webpages, no eTLDs.

museum.mw is not listed on the registry homepage[39], but on Wikipedia[40].
The whois record for the existing eTLD ac.mw is exactly the same as for
museum.mw, furthermore both domains redirect to the official registry
homepage. I did not add it to my patch, but I think you should contact the
registy: [e-mail]

alt.na, cul.na, edu.na, net.na, telecom.na, unam.na: The registry homepage[41]
displays a "Under Construction" banner and provides a Registrar Agreement and
a mail address, but no information about subdomains. There is a date that
indicates that the site was last updated 2008-07-24, so I think we have good
chances to get a reply if we contact them.

com.nc, net.nc, org.nc: I don't understand a single word of French, and even
by clicking through all the links I was not able to find a list of eTLDs.
Maybe somebody who speaks French can give it a try? The site looks quite up to
date.

de.net is a domain registered by the registry of the German TLD DENIC, but no
eTLD.

sch.ng: I can't find any information on the official registry homepage[42]. As
the registry just submitted their data 2 months ago, I think we should just
ask them again (while you're at it, you can ask if they provide a whois
server).

tel.no: Could not find it on the official registry page. What I did find is,
that we definitly need to go over the list of all those cities (yes, every
city > 5.000 inhabitants gets a second-level domain) and look for updates.

fax.nr, mob.nr, mobil.nr mobile.nr, tel.nr, tlf.nr actually belong to the same
owner as tel.no. Together they form some kind of telephone number registry, I
guess similar to InterNic. -> not added

dk.org is no eTLD.

eu.org is similar to InterNic, except that you don't have to pay for a
subdomain. -> not added

lodz.pl, lublin.pl, torun.pl, za.pl: Not listed on the official registry
homepage[43].


edu.ru, gov.ru, mil.ru: Seem to be in use like normal domains, as they resolve
to an IP.

test.ru: Does not resolve and does not seem to be in use[44].

sch.sd, tv.sd: Not listed on the official registry homepage[45].

ab.se is just a random domain, no eTLD.

lanarb.se seems to be reserved, but not in use.

mil.se is a domain used by the military, but not a real eTLD.

idn.sg was used to test IDNs, according to Wikipedia[46]. It is not listed on
the official registry homepage[47].

com.sh, edu.sh, gov.sh, mil.sh, net.sh, org.sh: Seems to be serviced by the
same company as .io. No information is provided on the registry homepage[48],
I guess it is best to contact them per mail.

edu.sk, gov.sk, mil.sk: Couldn't find any information about subdomains on the
official registry homepage[49]. Maybe write a mail?

*.st: I added the list of reserved domains[50] to my patch.

*.tf: "Free subdomains" service. Irrelevant for us, because they do not offer
subdomains at DNS level, instead they just use HTTP redirects or
frames/iframes.

gov.tp: The tp domain is currently being phased out in favor of tl. It is
still active, but I can't find any information about 2nd level domains other
than Wikipedia[51] stating that registrations were also possible at 3rd level.

*.tt: Same as .tf

co.tv: As Jo Hermans already stated on bugzilla[52], the official registry
homepage[53] does not provide any information about registrations at 3rd
level. VeriSign as registry seems all about "oh gee look how great our .tv
domain is" instead of providing real information.

co.ua, in.ua: Not listed on the official registry homepage[54].

khmelnitskiy.ua, kv.ua: Were missing, added them and removed uz.ua as listed
on the official registry homepage[54].

net.uz, org.uz:  On the official registry homepage[55] there is a drop-down
menu that offers uz, co.uz and com.uz. However, I found a subpage with a
list[56] of other subdomains, but I am not sure if it is really a list of
domains under which one can register domains. By any chance, does somebody
here speak Uzbek?

vatican.va seems like a normal tld to me, although they have some subdomains
(e.g. bay.vatican.va). Furthermore, as they don't even offer registrations to
the public, I think the danger of Cross-Site-Cookie attacks is not given.

co.vi, net.vi: I can't find any information on the registry homepage[57].
However I think it is the same company as on nic.sh and nic.io.

ch.vu, de.vu, fr.vu: same as *.tf and *.tt

com.vu, edu.vu, net.vu, org.vu: They seem to belong to the official registry
(TVL), but there is no information on the registry homepage[58]. We should
contact them via mail.

-----
1: http://www.aeda.ae/eng/aepolicy.php
2: http://www.aeda.ae/eng/policies/AEDA-POL-007_Domain_Name_Eligibility_Policy.pdf
3: http://nic.com.ai/
4: http://www.google.at/search?hl=de&q=site%3A*.gov.ai&btnG=Google-Suche&meta=
5: http://www.google.at/search?hl=de&q=site%3A*.edu.ai&btnG=Google-Suche&meta=
6: http://en.wikipedia.org/w/index.php?title=.al&oldid=226600383
7: https://www.amnic.net/policy/
8: http://en.wikipedia.org/w/index.php?title=.arpa&oldid=234249946
9: http://en.wikipedia.org/w/index.php?title=.be&oldid=233808442
10: http://www.dns.be/en/home.php?n=43.001
11: http://www.google.at/search?hl=de&q=site%3A*.belgie.be&btnG=Google-Suche&meta=
12: http://www.google.at/search?hl=de&q=site%3A*.fgov.be&btnG=Google-Suche&meta=
13: http://www.google.at/search?hl=de&q=site%3A*.dns.be+-belgium&btnG=Suche&meta=
14: http://en.wikipedia.org/w/index.php?title=.bh&oldid=233809357
15: http://www.inet.com.bh/
16: http://registro.br/info/dpn.html
17: http://en.wikipedia.org/w/index.php?title=.gc.ca&oldid=213815201
18: https://www.nic.cd/
19: http://www.nic.cu/normasparaelcentro.php#cladom
20: http://www.google.at/search?hl=de&q=site%3A*.edu.dm&btnG=Google-Suche&meta=
21: http://www.google.at/search?hl=de&q=site%3A*.gov.dm&btnG=Google-Suche&meta=
22: http://www.nic.dz/index.php
23: http://www.eenet.ee/EENet/dom_reeglid.html#lisa_B
24: http://www.afnic.fr/obtenir/chartes/fondamentaux
25: http://www.channelisles.net/applic/avextn.shtml
26: http://www.nic.gp/index.php?lang=en
27: http://www.domain.hu/domain/English/szabalyzat/specnev.html
28: http://en.wikipedia.org/w/index.php?title=.ie&oldid=233660375
29: https://www.nic.im/pdfs/imfaqs.pdf
30: http://www.inregistry.in/policies/
31: http://domain.nida.or.kr/eng/registration.jsp
32: http://www.google.at/search?hl=de&q=site%3A*.mil.lb&btnG=Google-Suche&meta=
33: http://www.nic.lc/rules.htm
34: http://www.nic.lk/seclevpr.html
35: http://www.gov.lt/index_en.php
36: http://af.mil.lt/
37: http://www.nic.ly/regulations.php
38: http://en.wikipedia.org/w/index.php?title=.mil&oldid=234412779
39: http://www.registrar.mw/
40: http://en.wikipedia.org/w/index.php?title=.mw&oldid=234091045
41: https://swakop.omadhina.co.na/
42: http://psg.com/dns/ng/
43: http://www.dns.pl/english/domain.html
44: http://www.google.at/search?hl=de&q=site%3A*.test.ru&btnG=Google-Suche&meta=
45: http://www.isoc.sd/sudanic.isoc.sd/billing_pricing.htm
46: http://en.wikipedia.org/w/index.php?title=.sg&oldid=234095607
47: http://nic.net.sg/sub_policies_agreement/2ld.html
48: http://www.nic.sh
49: https://www.sk-nic.sk/
50: http://www.nic.st/html/policyrules/
51: http://en.wikipedia.org/w/index.php?title=.tp&oldid=233339267
52: https://bugzilla.mozilla.org/show_bug.cgi?id=447815#c12
53: http://www.tv/
54: http://www.nic.net.ua/
55: http://www.cctld.uz/reg/
56: http://cctld.uz/specdomain/
57: http://nic.vi/
58: http://www.vunic.vu/

Comment 5

[Jo Hermans]

belgie.be and fgov.be are used by the government (it's the portal), but it's just a website, not a eTLD

dns.be is just the website of the country registrar

Comment 6

[Gervase Markham [:gerv]]

David: wow, that's an incredible piece of work! Fantastic.

No, please don't add pseudo-registries. We need explicit consent for that.

> I did not add the .al change yet, as I would like to use the same format
> as the other entries (the "Confirmed by registry ..." line).

Your patch includes an addition of "al". Do you mean "as", from comment 0? If so, don't add "Confirmed by registry", because the entire point is that it hasn't been confirmed like all the others. (I got no replies to my email seeking confirmation.)

> ("grandathered", whatever that means)

It's a typo - they meant "grandfathered". 
http://en.wikipedia.org/wiki/Grandfathered

> Can you contact the registry ([e-mail]) and ask them about it?

For registries which sent me an official list, such as be and co, I think we should leave things alone for now. And for ones which haven't, they may well not respond to a further email anyway. :-| So I think the thing to do is to do the thing you think is best, being cautious about what you add to the list. (Because in most cases, adding something is a restriction which could potentially break something.)

> however our list has the additional entries net.im, gov.im and nic.im.

Probably because the source says "the public"; in the UK also, the public can't get a net.uk or gov.uk or nic.uk address. 

Gerv

Comment 7

[Gervase Markham [:gerv]]

+// Unconfirmed: iris.arpa 2008-08-27

If they didn't send it to us, we shouldn't add it IMO. At least not without more evidence that they made a mistake.

+// Please note, that nic.in is not an offical eTLD, but used by most
+// government institutions.

Are you sure you mean nic.in?

One small nit: can you make sure that blank lines only appear between different top-level labels or different registries? So e.g. lt and gov.lt should not have a line between.

Gerv

Comment 8

[David Triendl]

> No, please don't add pseudo-registries. We need explicit consent for that.
Shall we remove priv.at and Centralnic then as well (btw above I sometimes wrote Internic, but I was talking about Centralnic)

> Your patch includes an addition of "al". Do you mean "as", from comment 0? If
> so, don't add "Confirmed by registry", because the entire point is that it
> hasn't been confirmed like all the others. (I got no replies to my email
> seeking confirmation.)
Yes, I meant "as". Was a bit late yesterday, sorry ;-) I added it to the new patch.

> > ("grandathered", whatever that means)
> 
> It's a typo - they meant "grandfathered". 
> http://en.wikipedia.org/wiki/Grandfathered
I made the typo ;-) Wikipedia (http://en.wikipedia.org/wiki/.al) said the domain was "grandfathered" and I did not understand what it meant (and my EN-DE dictionary was not helpful either). Thanks for the link.

> > Can you contact the registry ([e-mail]) and ask them about it?
> 
> For registries which sent me an official list, such as be and co, I think we
> should leave things alone for now. And for ones which haven't, they may well
> not respond to a further email anyway. :-| So I think the thing to do is to do
> the thing you think is best, being cautious about what you add to the list.
> (Because in most cases, adding something is a restriction which could
> potentially break something.)
My guess is that some registries would respond to a direct mail, even if they don't answer requests coming from a mailing list.

> > however our list has the additional entries net.im, gov.im and nic.im.
> 
> Probably because the source says "the public"; in the UK also, the public can't
> get a net.uk or gov.uk or nic.uk address. 
Okay. Do you think that lkd.co.im (the one listed in your attachement) is an eTLD too?

> +// Unconfirmed: iris.arpa 2008-08-27
> 
> If they didn't send it to us, we shouldn't add it IMO. At least not without
> more evidence that they made a mistake.
Okay, I'll remove it.

> Are you sure you mean nic.in?
Yes, the Wikipedia article for .in states that nic.in became some kind of government domain.

> One small nit: can you make sure that blank lines only appear between different
> top-level labels or different registries? So e.g. lt and gov.lt should not have
> a line between.
gov.lt and gc.ca are both different registrars than lt and ca respectively, that's why I added a blank line, but I removed it in the new patch.

Greetings,
David

Comment 9

[David Triendl]

Attachment: My first patch + changes suggested by Gerv

Comment 10

[Gervase Markham [:gerv]]

(In reply to comment #8)
> > No, please don't add pseudo-registries. We need explicit consent for that.
> Shall we remove priv.at and Centralnic then as well (btw above I sometimes
> wrote Internic, but I was talking about Centralnic)

Ones that are in there are in there because we do have explicit consent :-) So don't remove them.

> My guess is that some registries would respond to a direct mail, even if they
> don't answer requests coming from a mailing list.

They got direct mail the first time. Personalized direct mail too :-)

> Okay. Do you think that lkd.co.im (the one listed in your attachement) is an
> eTLD too?

If we can't find anything official saying it is, I'd leave it out for now.

Gerv

Comment 11

[Gervase Markham [:gerv]]

Fixed.
changeset:   18583:8dccd2cf32d2

Gerv