712640 - Split PSL into "registry" and "owner-requested" sections

Status: RESOLVED FIXED

(Core Graveyard :: Networking: Domain Lists, defect)

Description

[Gervase Markham [:gerv]]

https://wiki.mozilla.org/Public_Suffix_List/Use_Cases suggests that there is significant value for many PSL use cases in a simple split between what I will (for simplicity) call "registry" Public Suffixes - i.e. those like "co.uk" and "k12.sch.us", and "owner-requested" Public Suffixes like "appspot.com" or "operaunite.com". 

The key point here is that we can do such a split simply by moving the owner-requested suffixes to the bottom of the list, and putting a parseable marker-comment in between - this means we don't have to change the file format or disrupt existing clients.

If it turns out more is needed, we can cross that bridge when we come to it.

Gerv

Comment 1

[Gervase Markham [:gerv]]

Jothan: is this something you can take on? I don't want to invalidate a load of patches, so perhaps it would be a good thing to do after the current queue is processed.

Gerv

Comment 2

[Jothan Frakes]

Yes I can do this.  I think it is helpful to keep uk.com or other sub-domained registries in the "registry".

Comment 4

[Gervase Markham [:gerv]]

Attachment: Patch v.1

Here's an initial patch. It moves all the entries (as far as I can tell) that are private domains into a section at the bottom, and adds some delimiter comments so people can distinguish the two parts of the list. 

I'd like feedback on:

- Whether I got all of the private domains, or if I missed some
- Whether the way of doing the delimiter comments is OK
- Anything else you think of

Gerv

Comment 5

[Peter Kasting]

Comment on attachment 599222 [details] [diff] [review]
Patch v.1

Of the TLDs you made "private", four do not seem to be directly navigable:

priv.at
co.ca
za.net
za.org

Since the original request that started this whole snowball was to mark "TLD"s that were navigable, maybe it makes sense to leave these in the main section?

Comment 6

[Gervase Markham [:gerv]]

What Chrome is looking for is "TLDs that are navigable"; however, that's not right for all the use cases. :-) For example, people doing certificate issuance are looking for a distinction between ICANN and private. The clear distinction, which doesn't change regularly, is ICANN suffixes vs. private suffixes. So that's the split I've used.

priv.at could make their domain navigable tomorrow with no reference to anyone except themselves and no warning. If .uk plans to open up a foo.uk, then there's going to be a lot more consultation, time taken, and publicity.

If I type "priv.at" into Chrome's URL bar, I would expect to be taken to http://priv.at (and given an error if it didn't exist). I wouldn't expect a search to be done. And I suspect the owners of priv.at would expect the same.

Gerv

Comment 7

[Peter Kasting]

OK.  I thought this wasn't what our security folks wanted for wildcard certs but I talked to them and that's fine.

If you want Firefox and Chrome to be in sync on any behaviors, here's what we intend to do once the list is split into these two sections:

(1) For navigation purposes, we will ignore the "private" section entirely.  This will cause all "eTLDs" listed there to become navigable, whether or not they truly resolve.
(2) For SSL Cert purposes, we will ignore the "private" section entirely.  This will allow us to accept a certificate for "*.appspot.com".
(3) For cookie purposes, the behavior will depend on the origin of the request.  A subdomain of one of the "private" eTLDs will behave as if the "private" section of the list is fully in-play; that is, foo.appspot.com will not be allowed to set a domain cookie on appspot.com, and will ignore any domain cookies that do exist on appspot.com.  If the user navigates to the eTLD itself, we will behave as if the "private" rules do not exist, so appspot.com would be allowed to set both host and domain cookies on appspot.com (but not a domain cookie on .com).

Comment 8

[Gervase Markham [:gerv]]

Those behaviours seem very sensible. I don't think we do (2) or (1) in the sense you do. We do the first part of 3, but not the special exception you make at the end. That is a change in what domain owners might expect from browser behaviour using the PSL; in the past, we have told them that whole-domain cookies will not work if they get their domain added to the PSL. What makes you want to add the exception? Is there some Google or appspot.com requirement for it, or is it just because it's what you think people would like?

Do you think my labels for the sections ("ICANN DOMAINS" and "PRIVATE DOMAINS") are correct? I'm a bit worried about them. "ICANN" is probably right in some sense, because you could imagine other people adding a section for other root zone operators. But in another sense, they are all below the ICANN root.

ICANN-AFFILIATED REGISTRY SUFFIXES
PRIVATE REGISTRY SUFFIXES

?

Gerv

Comment 9

[Peter Kasting]

So to clarify, the Gecko cookie behavior is that "appspot.com" itself will simply not be allowed to read or write any cookies at all?

The reason I wrote that exception description is twofold:
(1) It allows websites hosted on one of these "fake TLDs" to continue working normally after the entry is added to the PSL, without introducing the possibility of session fixation attacks.  It seems better to not break sites unless we need to.
(2) It should be slightly easier for us to code.  With the exception, every navigable hostname can set host cookies on itself, and can set domain cookies on at least itself, and possibly some number of parents.  Without it, hostnames that are themselves in the PSL behave very differently, and thus the cookie code needs to be more careful.

As for labels, I don't have a strong feeling.  "Recognized root domains" and "Sub-let domains" is lousy, as is "Real TLDs" and "Fake TLDs", so I'm out of ideas :)

Comment 10

[Jothan Frakes]

Comment on attachment 599222 [details] [diff] [review]
Patch v.1

I'd consolidate the CentralNic entries into one, but otherwise it matches what was proposed by some of the community users.

Comment 11

[Peter Kasting]

Comment on attachment 599222 [details] [diff] [review]
Patch v.1

r=me but the CentralNic and DynDNS domain handling is inconsistent.  The former is split to keep the domains in roughly the same order as in the main TLD list; the latter is all grouped in one big block.

I don't have a strong feeling which is better, but we should pick one and be consistent.

Comment 12

[Gervase Markham [:gerv]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/91e535989963

Gerv

Comment 13

[Marco Bonardo [:mak]]

https://hg.mozilla.org/mozilla-central/rev/91e535989963