621708 - Update .au entry in PSL

Status: RESOLVED FIXED

(Core Graveyard :: Networking: Domain Lists, defect)

Description

[Gervase Markham [:gerv]]

Hi,

You have left some domains out of your list by mistake:

    csiro.au
    asn.au
    id.au

...

The comment "// au geographical names (vic.au etc... are covered
above)" needs to be removed - the exhaustive list of CGDNs is already
included below there.

...

Kind Regards,
Chris Drake

Comment 1

[Gervase Markham [:gerv]]

The current rules are correct here - *.au covers csiro.au, asn.au etc.

The comment was correct but the list of CGDNs was unnecessary added. :-| Bug 560885 was probably a mistake. Ah, well. I'll fix the comment in another bug - it's hardly worth making a separate patch.

Gerv

Comment 2

[Peter Kasting]

We should remove *.au and add:
.com.au
.net.au
.org.au
.edu.au
.gov.au
.csiro.au
.asn.au
.id.au
.info.au
.conf.au
.oz.au

(Verified that these are open for registration or have existing sites via Google "site:" search)

At least for Chrome, we want as much granularity as we can get.

Comment 3

[Peter Kasting]

Attachment: patch v1

Comment 4

[Jothan Frakes]

That list looks correct.  I have a confirmation in to operations at the registry.  

I think I read the patch application correctly, and it won't, but please confirm this change won't remove the CGDN subdomains of .au. 

They have a very responsible means of allocation for regions in place that would be bad to roll up cookies for.  http://www.cgdn.org.au

Comment 5

[Gervase Markham [:gerv]]

The trouble with switching from *.au to the explicit list is that the PSL gets more fragile - it breaks if the .au registry opens up some more 2nd level domains, whereas the *.au version doesn't break in that circumstance. (Both break if .au starts accepting first-level registrations, of course.)

If a TLD has first-level registrations and also some special second-level domains, then obviously you have to have an explicit list. But that's not the case here.

pkasting: can you elaborate more on why Chrome's uses prefer the more fragile version?

Gerv

Comment 6

[Peter Kasting]

(In reply to comment #5)
> The trouble with switching from *.au to the explicit list is that the PSL gets
> more fragile - it breaks if the .au registry opens up some more 2nd level
> domains, whereas the *.au version doesn't break in that circumstance. (Both
> break if .au starts accepting first-level registrations, of course.)

For a very long time a number of us have tried to make the list as explicit as possible, avoiding wildcards unless they're truly necessary; both pamg and I have put in patches in that vein.  This doesn't represent a new point of view on how the PSL should work.

For example, just scanning the file quickly, I see that basically all of the initial entries use explicit lists of 2LDs where in theory they could use "*.tld", even when (as with .aero) the number of these is fairly large.

I also don't think there is a significant practical cost incurred here.  While you are correct as to how this can incur future inaccuracy, practically speaking, the overall number of domains in the file is extremely large and we already change these "as someone notices", with the cost of an individual change being very low.  So the incremental cost over what we're doing now is effectively zero.

> pkasting: can you elaborate more on why Chrome's uses prefer the more fragile
> version?

We prefer the accurate version because this list drives a number of different security and user-facing policies in Chrome, including among other things the functioning of the address bar, and it's important to us to catch invalid URLs ahead of time as much as possible.

Finally, I know you also received this mail, but for the benefit of anyone else reading, audo.org.au has confirmed via email that the proposed list here is accurate.

Comment 7

[Jo Hermans]

(In reply to comment #6)
> For a very long time a number of us have tried to make the list as explicit as
> possible, avoiding wildcards unless they're truly necessary; both pamg and I
> have put in patches in that vein.  This doesn't represent a new point of view
> on how the PSL should work.
> 
> For example, just scanning the file quickly, I see that basically all of the
> initial entries use explicit lists of 2LDs where in theory they could use
> "*.tld", even when (as with .aero) the number of these is fairly large.

As the original author, I have to disagree. I did exactly the opposite. The problem was that far too many TLD's accepted entries at the second level, so a * could not be used (unless you could compile a list with ! exceptions).

The original Australian entry can be found in attachment 234865 [details] ; it uses *.au

Comment 8

[Gervase Markham [:gerv]]

Would it work to put both the list of TLDs and also *.au into the PSL? Then, you would get the future-proofing properties, but Chrome could adopt a rule that if specific entries are given, the * is ignored (and take the possible forward compatibility hit). That allows different groups using the list to make different trade-offs between forward compatibility and exact precision.

pkasting: what do you think?

Gerv

Comment 9

[Gervase Markham [:gerv]]

pkasting: ping? What do you think about comment 8?

Gerv

Comment 10

[Peter Kasting]

Sorry, missed the pings here somehow; I think gmail has been spamboxing bugzilla mail.

I don't want to code a rule like in comment 8 into Chrome's parser, partly because it's effectively a fork of the PSL but maintained by subtle code changes instead of an obvious public fork, and partly because I think the rule is confusing to readers and not terribly beneficial.

I don't find the forward-compat args terribly compelling for a couple reasons:
  * We have orders of magnitude more forward compat problems today with entirely new TLDs appearing from ICANN (both for IDN TLDs and the recent anybody-can-make-a-TLD rules), and when we have to spend lots of constant effort to stay current on these, it just doesn't seem like a huge deal to say that we'll occasionally need to take a registry update for an existing TLD.
  * More importantly, Firefox takes a minimal forward-compat here compared to Chrome.  In Firefox the possible consequence is that a malicious site in a new 2LD can access cookies of another site in the new 2LD, which seems comparatively unlikely, whereas in Chrome the site cannot easily be navigated to at all.  This means that we (Chrome) are accepting most of the risk here, and it also means that we're very likely to catch omissions quickly as they result in obvious problems rather than subtle ones.

Given that, I'd still like to land the patch on this bug.

Comment 11

[Gervase Markham [:gerv]]

Comment on attachment 526796 [details] [diff] [review]
patch v1

Review of attachment 526796 [details] [diff] [review]:
-----------------------------------------------------------------

It doesn't seem like this has been the thin end of the wedge; in the interests of keeping the PSL unified, I'm going to land this.

Gerv

Comment 12

[Gervase Markham [:gerv]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bb68ca933ea8

Gerv

Comment 13

[Peter Kasting]

Should this be ->FIXED then?

Comment 14

[Gervase Markham [:gerv]]

pkasting: new checkin process. It'll get closed when mozilla-inbound gets merged to mozilla-central (which happens once daily or so).

Gerv

Comment 15

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/bb68ca933ea8

Comment 16

[Gervase Markham [:gerv]]

Comment on attachment 526796 [details] [diff] [review]
patch v1

[Approval Request Comment]

This is a representative request for approval for checkin for all of the recent batch of PSL changes, as listed here:
https://bugzilla.mozilla.org/buglist.cgi?bug_id=621708%2C%20660215%2C%20660387%2C%20661509%2C%20666500%2C%20676359%2C%20685781%2C%20687459%2C%20688908%2C%20694030;bug_id_type=anyexact

The approval information is the same for all of them. I can do a roll-up patch if it would help.

User impact if declined: domain highlighting and cookie setting policies and behaviour are less accurate.
Testing completed (on m-c, etc.): Policy rather than code change; but patches have baked over the weekend on m-c with no ill effects.
Risk to taking this patch (and alternatives if risky): Very low.
String changes made by this patch: None.

Gerv

Comment 17

[Alex Keybl [:akeybl]]

Comment on attachment 526796 [details] [diff] [review]
patch v1

[Triage Comment]
Assuming this has gone through all the proper processes, approving for Aurora 12 and Beta 11. We do not expect to see any regressions caused by this patch which, as Gerv notes, is really just policy change.

Comment 18

[Gervase Markham [:gerv]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/6362ff66c1a4
https://hg.mozilla.org/releases/mozilla-beta/rev/53fd188c0841

Gerv

Comment 19

[u279076]

What can QA do to verify this fix? go to sites as per comment 3?

Comment 20

[tclugg]

Commit 8ffe71e1ce092f446ae96d7c291822698c581bac has caused a regression: the csiro.au domain is not a 2LD and should not have been added at line 266.

Comment 21

[Gervase Markham [:gerv]]

tclugg: what makes you say that? Can you tell us more about this domain and what it's used for? Are you connected with its administration? http://en.wikipedia.org/wiki/.au lists .csiro.au.

Gerv

Comment 22

[tclugg]

Gerv: Apologies, my comment was misleading and I should clarify. The "csiro.au" domain is not like the other .au 2LD domains, as it is the only .au 2LD that is for *one* *specific* organisation.

www.csiro.au should be able to set cookies for the entire csiro.au domain.    Removing the "+csiro.au" rule from line 266 allows this to happen without affecting other domains.

I worked for CSIRO IT for 5 years from 2001 through 2006.

http://en.wikipedia.org/wiki/Commonwealth_Scientific_and_Industrial_Research_Organisation#Domain_name says the following:
> CSIRO was the first Australian organisation to start using the internet,
> and as such was free to register the second-level domain csiro.au (as
> opposed to csiro.org.au or csiro.com.au). Guidelines were introduced in
> 1996 to regulate the use of the .au domain.

Tyson.

Comment 23

[Gervase Markham [:gerv]]

Tyson: apart from conf.au ;-) Seriously, if CSIRO want their 2LD to be removed from the list, I'd be happy to do that - but I would really need a request from someone who still works there. Perhaps you know someone you could encourage to file a bug about it?

Gerv

Comment 24

[tclugg]

Gerv: Why do we need CSIRO representative to authorise the proposed change?  Did such a person authorise the addition of line 266?  Is there a documented Mozilla policy that you are following?

I guess this relates to https://bugzilla.mozilla.org/show_bug.cgi?id=712640 - which gives good insight into where domains such as csiro.au should fit in the PSL.

Can we at least mark the csiro.au domain as "private" in context of #712640?  It isn't administered by a public body, IMHO it shouldn't be listed in the public section.

Comment 25

[Gervase Markham [:gerv]]

Tyson: there is no documented Mozilla policy on this question; we are an organization which documents policy only reluctantly :-) Changes to the PSL which have an effect on clients need some sort of validation. The easiest way is to get the admins of the TLD or 2LD in question to tell us that they want the change - then it's easy. This is particularly true for additions, which can break things in unexpected ways.

If we can't manage that, then we look for verifiable sources such as the web pages of the registry. I've now found a list on auda.org.au:
http://www.auda.org.au/domains/au-domains/

Based on that, I agree it's reasonable to remove csiro.au. Please can you file a new bug for this?

Gerv

Comment 26

[Gervase Markham [:gerv]]

https://bugzilla.mozilla.org/show_bug.cgi?id=791770 for CSIRO.

Gerv