Last Comment Bug 669792 - gov.uk is not considered to be a public suffix
: gov.uk is not considered to be a public suffix
Status: VERIFIED FIXED
[qa-]
: regression, verified-aurora, verified-beta, verified1.9.2
Product: Core
Classification: Components
Component: Networking: Domain Lists (show other bugs)
: Trunk
: All All
: -- normal with 2 votes (vote)
: mozilla9
Assigned To: Gervase Markham [:gerv]
:
: Patrick McManus [:mcmanus]
Mentors:
: 670947 677017 (view as bug list)
Depends on:
Blocks: 658965
  Show dependency treegraph
 
Reported: 2011-07-06 16:46 PDT by Alex Bishop
Modified: 2011-09-09 15:24 PDT (History)
22 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
.2-fixed
fixed
fixed
.22-fixed


Attachments
Patch v.1 from Nominet (243 bytes, patch)
2011-08-26 02:01 PDT, Gervase Markham [:gerv]
christian: approval‑mozilla‑aurora+
christian: approval‑mozilla‑beta+
christian: approval‑mozilla‑release+
Details | Diff | Splinter Review
gov.uk Problem - Testing with Firefox 6.01 and 6.02 (395.09 KB, application/octet-stream)
2011-09-05 05:50 PDT, charles.fuller
no flags Details

Description Alex Bishop 2011-07-06 16:46:59 PDT
Bug 658965 updated the public suffix list such that gov.uk is no longer considered to be a public suffix.

This does not seem right to me. The gov.uk domain name acts like a public suffix (similar to co.uk or org.uk). It definitely doesn't act like a single registered domain name (as bl.uk and jet.uk do). It doesn't act like both (as nhs.uk and police.uk do).

The most notable effect is that if one visits a gov.uk site, like http://www.direct.gov.uk/, http://www.tfl.gov.uk/ or http://home.scotland.gov.uk/, only the "gov.uk" part gets highlighted in the location bar.

The relevant rule that makes gov.uk not a public suffix was added at the request of Nominet, so it has some authority behind it (even though Nominet do not manage gov.uk; the Central Office of Information do, with day-to-day operations delegated to JANET). However, I still think it should be reconsidered. I can't see the need for the Northern Ireland Executive and Transport for London to share cookies.
Comment 1 Jo Hermans 2011-07-12 11:48:21 PDT
*** Bug 670947 has been marked as a duplicate of this bug. ***
Comment 2 Gervase Markham [:gerv] 2011-07-13 03:58:49 PDT
(In reply to comment #0)
> The relevant rule that makes gov.uk not a public suffix was added at the
> request of Nominet, so it has some authority behind it 

This is why I approved the change.

> (even though Nominet
> do not manage gov.uk; the Central Office of Information do, with day-to-day
> operations delegated to JANET). 

That's interesting. If it's so, I would take a counter-opinion from them and change it back, if they provided one.

> However, I still think it should be
> reconsidered. I can't see the need for the Northern Ireland Executive and
> Transport for London to share cookies.

However, this sort of consideration can't be a factor. The PSL is not defined by what we think should or shouldn't be the case, it's defined by what the domain owners tell us they want - taking into account all the factors, such as UI presentation, cookie sharing and whatever. We can't make that call for them.

Gerv
Comment 3 Alex Bishop 2011-07-13 16:42:24 PDT
(In reply to comment #2)
> (In reply to comment #0)
> > (even though Nominet
> > do not manage gov.uk; the Central Office of Information do, with day-to-day
> > operations delegated to JANET). 
> 
> That's interesting. If it's so, I would take a counter-opinion from them and
> change it back, if they provided one.

It is so. From http://coi.gov.uk/guidance.php?page=192#section1a:

".gov.uk - the Digital Policy team at COI through the Naming and Approvals Committee is responsible for maintaining this guidance which acts as the policy governing the SLD. JANET(UK) administers the SLD on behalf of COI, providing the name submission, name modification, approval and registration systems."

See also: http://www.nic.uk/registrants/aboutdomainnames/sld/delegated/

Of course, neither the COI nor JANET have ever expressed an opinion on whether gov.uk is a public suffix or not (only Nominet have, though of course their decision may have been informed by either of the other two organisations).

> > However, I still think it should be
> > reconsidered. I can't see the need for the Northern Ireland Executive and
> > Transport for London to share cookies.
> 
> However, this sort of consideration can't be a factor. The PSL is not
> defined by what we think should or shouldn't be the case, it's defined by
> what the domain owners tell us they want - taking into account all the
> factors, such as UI presentation, cookie sharing and whatever. We can't make
> that call for them.

I see things have changed since bug 460776. While the maintainers of the list cannot make the call for a particular domain name, they do control the guidelines that the domain owners use to determine what is and what is not a public suffix.

I feel that in this case the guidelines have been wrongly interpreted and that the COI should be urged to re-examine their decision (or make a decision in the first instance, depending upon how much involvement they had with Nominet's initial request). It was never my intention to suggest that the public suffix list entry for gov.uk should be changed unilaterally; the relevant authorities must be involved.
Comment 4 Gervase Markham [:gerv] 2011-07-15 09:37:40 PDT
(In reply to comment #3)
> I see things have changed since bug 460776.

Hmm. Perhaps. Although I think it's more reasonable to regard the government as a single entity than it is a collection of NHS trusts which have no shared organizational structure.

> I feel that in this case the guidelines have been wrongly interpreted and
> that the COI should be urged to re-examine their decision (or make a
> decision in the first instance, depending upon how much involvement they had
> with Nominet's initial request). It was never my intention to suggest that
> the public suffix list entry for gov.uk should be changed unilaterally; the
> relevant authorities must be involved.

I'm not sure I have time to chase this up, but would happily hear from whichever official representatives anyone else is able to make contact with.

Gerv
Comment 5 think-cell support team 2011-07-20 01:38:29 PDT
We use the public suffix list to provide a default assignment of new customer email addresses to existing accounts in our online shop. It usually works very well in this regard, but a rule change such as !gov.uk screws it up.

We know that such a use is not "officially supported" in any way, but it may be interesting to know that the PSL _is_ used in other ways and there are other effects of erroneous rules besides a wrong highlighting in the location bar.

Michael
Comment 6 William Warby 2011-08-19 00:04:41 PDT
I work for a London Borough council and came here to report this same "bug". I don't have an opinion on the complex issue of ownership and management of the ".gov.uk" suffix and whether or not it is a public suffix, but from an end user perspective, this *looks* like a bug, plain and simple. Our domain is walthamforest.gov.uk - not gov.uk. Internet Explorer has the same highlighting feature, and it recognises our domain as walthamforest.gov.uk.
Comment 7 Dão Gottwald [:dao] 2011-08-19 00:05:40 PDT
*** Bug 677017 has been marked as a duplicate of this bug. ***
Comment 8 Gervase Markham [:gerv] 2011-08-19 01:41:10 PDT
If someone wants to contact the Nominet guy who originally submitted the change, or the COI, and ask them to request this to be changed, that would be fine :-) But as it stands, Nominet want it this way.

Gerv
Comment 9 Ian Nartowicz 2011-08-22 14:13:56 PDT
Did this just land or did I just get it with an upgrade?  What a nightmare.  It is wrong and all sorts of stuff is broke with this.  Who do I have to contact to get this clarified?
Comment 10 Anand Kumria 2011-08-22 16:41:53 PDT
(In reply to Ian Nartowicz from comment #9)
> Did this just land or did I just get it with an upgrade?  What a nightmare. 
> It is wrong and all sorts of stuff is broke with this.  Who do I have to
> contact to get this clarified?

It landed as part of Firefox 6.0, alas.

I've been sending an email to the COI everyday as soon as I was made aware of this.

naming@coi.gsi.gov.uk - which has not responded yet - and after some searching in whois, I'm now starting to email gsi@ogcbs.gsi.gov.uk as well. So I can find out who needs to make a decision on this.

(In reply to Gervase Markham [:gerv] from comment #4)
> (In reply to comment #3)
> > I see things have changed since bug 460776.
> 
> Hmm. Perhaps. Although I think it's more reasonable to regard the government
> as a single entity than it is a collection of NHS trusts which have no
> shared organizational structure.
> 
> > I feel that in this case the guidelines have been wrongly interpreted and
> > that the COI should be urged to re-examine their decision (or make a
> > decision in the first instance, depending upon how much involvement they had
> > with Nominet's initial request). It was never my intention to suggest that
> > the public suffix list entry for gov.uk should be changed unilaterally; the
> > relevant authorities must be involved.
> 
> I'm not sure I have time to chase this up, but would happily hear from
> whichever official representatives anyone else is able to make contact with.
> 
> Gerv


I've no idea why Nominet decided to make this change.

Unfortunately Bug 658965, does not include the email address of the originator.

Nor, indeed the actual email. 

Frankly it seems unusual that they were able to bypass utilising bugzilla themselves. It, the bug, and patch went via Gerv. 

Anand
Comment 11 Jothan Frakes 2011-08-23 10:27:19 PDT
Bug 658965 has the patch request that modified the entry.

In any regard, I am more interested in solving this than pointing fingers.

This seems like it would be solved if the entry were to be changed from !gov.uk to gov.uk in a request from Nominet.

I am reaching out to Nominet on this to have them submit a patch request.
Comment 12 charles.fuller 2011-08-25 08:27:36 PDT
Here is another side-effect of this problem: the NoScript extension uses this to propose the scripting rules. This means that you must manually edit the "Whitelist" entries to allow a site to work - no one in their right minds would allow all *.gov.uk websites to run JavaScript etc. without limit.

For example, go to http://cityark.medway.gov.uk/ (I've intentionally picked an obscure site unless you happen to be a genealogist so that NoScript will require a rule to be configured) and NoScript will ask whether to allow the "site" gov.uk
Comment 13 Doug Thorndyke 2011-08-25 08:29:10 PDT
Hello, 

We have spoken with JANET to ask what they would prefer to do regarding this matter.

If they confirm they would like a patch request submitted to reverse the change, Nominet will do this.
Comment 14 Gervase Markham [:gerv] 2011-08-25 08:32:34 PDT
Doug: that's good to hear :-) Let us know if we can provide further technical input, advice or explanation.

Gerv
Comment 15 Doug Thorndyke 2011-08-25 09:28:01 PDT
Hi Gerv,

JANET have confirmed that they would like a patch request submitted.

You should get one in the next few minutes.

-Doug
Comment 16 Gervase Markham [:gerv] 2011-08-26 02:01:35 PDT
Created attachment 555979 [details] [diff] [review]
Patch v.1 from Nominet

Here's the patch.

Gerv
Comment 17 Gervase Markham [:gerv] 2011-08-26 02:21:44 PDT
Firefox 7 releases on 27th September; in order to get this in, we need to:

- get it reviewed
- check it in on the trunk
- let it bake for a day or two
- apply for approval to check in to branches
- check it in, in good time

Gerv
Comment 18 Doug Thorndyke 2011-08-26 02:36:13 PDT
Gerv,

Can I just clarify:

This patch will not resolve the issue in Firefox 6 and the behaviour it currently exhibits for .gov.uk will remain always?
Comment 19 Gervase Markham [:gerv] 2011-08-26 03:01:37 PDT
Doug: the new Mozilla rapid release process means that we release a new version every 6 weeks, and that Firefox 7 is the successor to Firefox 6 for all purposes - just as 6 was for 5, and 5 was for 4. There won't be a separate "security release" branch for Firefox 6, or anything like that, unless there's a security firedrill before September 27th.

However, Mozilla now does silent updates, which means that a few months after 7 is released, we hope very few people will be running 6.

Gerv
Comment 20 Anand Kumria 2011-08-26 04:39:37 PDT
(In reply to Gervase Markham [:gerv] from comment #19)
> Doug: the new Mozilla rapid release process means that we release a new
> version every 6 weeks

That said, they - Mozilla - released 5.0.1 to fix a severe problem on a specific platform.

With things as they are, Firefox 6 allows all .gov.uk sites to impact each other.

Apart from the privacy implications of leaking cookies from one site to another there are security implications for users too.

I would hope that this lands fairly quickly (I assume you are able to do r=, gerv?) and land it on central (and then aurora, beta) and release?

If not you (since you landed the original patch, and this is just backing it out), who would do the review?

> separate "security release" branch for Firefox 6, or anything like that,
> unless there's a security firedrill before September 27th.

As I indicated above, I believe that this is a privacy and security firedrill.

And that a 6.0.1 is warranted.

Regards,
Anand
Comment 21 Gervase Markham [:gerv] 2011-08-26 06:33:50 PDT
Anand: this is way, way below firedrill level. I think you misunderstand the implications.

If foo.gov.uk sets a cookies for foo.gov.uk, then it will be sent to foo.gov.uk - not to bar.gov.uk. bar.gov.uk cannot read those cookies.

The difference is that foo.gov.uk is currently _capable_ of setting cookies for all of gov.uk, and if they do, they will be sent to every gov.uk domain. (This would allow, for example, single signon across all of the UK government.) This will change when we check in this patch. But the fact that foo.gov.uk may do that does not mean that it actually does do it. It's certainly not recommended practice. And if it did, perhaps by accident, the results would most likely be harmless.

In some circumstances, it might be easier for a malicious UK government site to attack another UK government site - but that seems to me to be highly unlikely. Are you aware of any malicious sites in the .gov.uk domain?

I would note that the current situation is exactly what was the case before the PSL was introduced, and therefore what was the case for a decade or more as the web developed. It is not a firedrill-worthy thing.

Gerv
Comment 22 Adam Bailin 2011-09-01 02:36:12 PDT
Hi, I am a Digital Policy Manager at COI and I manage the .gov.uk domain. I don't know how the Nominet request originated but I certainly don't want individual government websites to be able to set cookies for the whole of .gov.uk. Only the owner of .gov.uk (root) should be able to do that.

I understand that the list will be updated following a request from Nominet and approval by JANET, who operate the .gov.uk DNS on our behalf. My concern now is that this leaves us open until 27 September which is way past a reasonable timeframe for sorting this out.

Whilst I don't know of any malicious .gov.uk sites, I don't want to take any security or privacy risks. From a government perspective this problem erodes user trust that we have worked hard to build over the years. Therefore, can I request that this is firedrilled and fixed in a version 6.0.1 as soon as possible?
Comment 23 Gervase Markham [:gerv] 2011-09-01 09:22:00 PDT
Comment on attachment 555979 [details] [diff] [review]
Patch v.1 from Nominet

I can confirm that this patch applies cleanly to mozilla-central, mozilla-aurora, mozilla-beta and mozilla-release. Making it apply to other branches, if any modifications are required, is left as an exercise for the reader ;-)

Gerv
Comment 24 Daniel Veditz [:dveditz] 2011-09-01 12:11:39 PDT
We'll want this fixed in 3.6.x too, right?
Comment 25 Gervase Markham [:gerv] 2011-09-01 14:17:40 PDT
Yep.

Gerv
Comment 27 Geo Mealer [:geo] -- This account is inactive after 2015-07-07 2011-09-04 18:16:09 PDT
If anyone has some testing hints on how to verify this fix in a black-box way for desktop and Fennec builds (particularly the ones w/o URL bar highlighting) it would be very useful. 

Right now, based on current knowledge, this bug is at risk for not being verified.
Comment 28 charles.fuller 2011-09-04 23:11:45 PDT
Yes, see comment [12] above - the NoScript add-on relies on this functionality, so when you go to an xxx.gov.uk website NoScript proposes to whitelist/blacklist "gov.uk" instead of the actual domain. Before release 6.0, NoScript picked out the domain correctly. If using NoScript is an acceptable method of testing, I can provide a list of xxx.gov.uk pages (for Record Offices) and some notes on what should/shouldn't happen.
Comment 29 Geo Mealer [:geo] -- This account is inactive after 2015-07-07 2011-09-04 23:23:49 PDT
It's better than nothing, thanks! If anyone else has other suggestions, we could use them. I'm fine with indirect methods like this.
Comment 30 charles.fuller 2011-09-04 23:46:24 PDT
(In reply to Geo Mealer [:geo] from comment #29)
> It's better than nothing, thanks! If anyone else has other suggestions, we
> could use them. I'm fine with indirect methods like this.

I've just tested using NoScript 2.1.2.6 with Firefox 6.01 under Windows XP by visiting an xxx.nhs.uk and an xxx.gov.uk web-site (for which I didn't have rules in NoScript), and NoScript only picked up the "gov.uk"/"nhs.uk" parts of the domain names.

If it helps (and someone can guide me on where to get/how to install the patched version under openSUSE 11.4), I can do some testing for you.
Comment 31 Geo Mealer [:geo] -- This account is inactive after 2015-07-07 2011-09-04 23:53:14 PDT
Charles,

That'd be great! I can't guide you on OpenSUSE, though, unfortunately, and I'm guessing our default Linux builds don't work for you. That's really what we'd want tested most urgently, since this is a pre-release test of the generated binaries.

If they do work for you, though, they'd be here:

ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.22-candidates/build2/
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/6.0.2-candidates/build2/
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/7.0b4-candidates/build2

Otherwise, if you're able to get builds on those branches you can test and don't mind doing so, we'd appreciate any help. Just post in the bug what you've done and what you saw, just like you did in c30.
Comment 32 Jo Hermans 2011-09-05 01:35:48 PDT
(In reply to Geo Mealer [:geo] from comment #27)
> If anyone has some testing hints on how to verify this fix in a black-box
> way for desktop and Fennec builds (particularly the ones w/o URL bar
> highlighting) it would be very useful. 
> 
> Right now, based on current knowledge, this bug is at risk for not being
> verified.

Actually, you don't need NoScript to check this. Just go to a site like http://www.direct.gov.uk/ : in Firefox 6, you'll see that "gov.uk" is bold in the location bar. In a Nightly build, you'll see that "direct.gov.uk" is bold.

This has nothing to do with cookies ofcourse, but it follows the same rules.
Comment 33 Geo Mealer [:geo] -- This account is inactive after 2015-07-07 2011-09-05 03:31:38 PDT
Jo, excellent point! The challenges there are 3.6.22 and Fennec, neither of which have that behavior, unfortunately, and which are being qualified for this upcoming release.
Comment 34 charles.fuller 2011-09-05 05:03:46 PDT
I'm about to test Firefox 6.01 --> 6.02 under Windows XP, saving screen-shots in a LibreOffice document. (I couldn't install the 6.02 nightly-build under openSUSE 11.4 because of a dependency problem with libc.6.so and I don't yet know enough about gcc under Linux to solve this quickly.)

I'm not sure what to look for/test under Firefox 3.6.22 - I have the downloads ready to install - I last used 3.6 quite some time ago.
Comment 35 charles.fuller 2011-09-05 05:50:47 PDT
Created attachment 558268 [details]
gov.uk Problem - Testing with Firefox 6.01 and 6.02

Successful test with Firefox 6.02 nightly build under Windows XP (compared with 6.01 which has problems).
Comment 36 Henrik Skupin (:whimboo) 2011-09-05 07:13:55 PDT
Verified fixed by checking the location bar highlight and trying to add exceptions with NoScript in the following builds:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0a1) Gecko/20110904 Firefox/9.0a1

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a2) Gecko/20110904 Firefox/8.0a2

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0) Gecko/20100101 Firefox/7.0 (build #2)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0.2) Gecko/20100101 Firefox/6.0.2

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.22) Gecko/20110902 Firefox/3.6.22

Not sure how to mark this bug as verified fixed for Firefox 6. Looks like there is no keyword for it around.
Comment 37 Geo Mealer [:geo] -- This account is inactive after 2015-07-07 2011-09-05 11:40:09 PDT
Charles,

Thanks tons for your help, and for tipping us off on a good way to verify (and sorry I missed your c12 where you did it the first time. ;) ) We'd have been stuck here in QA without you.
Comment 38 charles.fuller 2011-09-05 12:10:28 PDT
(In reply to Geo Mealer [:geo] from comment #37)
> Charles,
> 
> Thanks tons for your help, and for tipping us off on a good way to verify
> (and sorry I missed your c12 where you did it the first time. ;) ) We'd have
> been stuck here in QA without you.

Is testing version 3.6.22 nightly build still outstanding? If yes, what should I look for when I compare it with version 3.6.21? (I have both sets of software ready to install side-by-side under Windows XP if this is required - testing should take about 10 minutes.)
Comment 39 juan becerra [:juanb] 2011-09-05 12:18:47 PDT
Charles, I believe Henrik has already verified this on 3.6.22 per comment #36. Thanks for your help.
Comment 40 gsakljgla 2011-09-06 06:23:05 PDT
If you wish to test this on 1.9.2 without using an extension, an easy way to do so is to go to an https page:
https://www.brent.gov.uk/
If gov.uk is a domain name, "gov.uk" is written with white letters on a blue background. On the other hand, if the domain name is brent.gov.uk, the white letters read "brent.gov.uk".
Comment 41 juan becerra [:juanb] 2011-09-09 15:24:58 PDT
QA no longer needs to track this.

Note You need to log in before you can comment on or make changes to this bug.