Lotus Notes webmail (at Hawaii DOE) broken by eTLD list update




Networking: Domain Lists
7 years ago
7 years ago


(Reporter: beltzner, Assigned: gerv)



Dependency tree / graph

Firefox Tracking Flags

(blocking2.0 final+, blocking1.9.2 .13+, status1.9.2 .13-fixed, blocking1.9.1 .16+, status1.9.1 .16-fixed)



(3 attachments)

Original report from Hendrix:


Aloha. The latest update of Firefox prevents hundreds of employees of
the Hawaii Department of Education (DOE) from accessing any DOE
web-based applications (i.e. email, student databases, professional
development registration, etc.). The DOE's technical support
branch--the Centralized Service Desk, (xxx) xxx-xxxx -- is advising all
employees to use an alternate browser.  Firefox is my preferred browser.
Is there anyway that you can rectify this matter since so many users
are being affected?


Cheng dug in with a username and password given by the contact, and said:

"I have headers (unfortunately they have username/password in plaintext so I won't attach).  Looks like it's trying to do a redirect to itself but failing to set a cookie (?)."

He then found this regression range:


in which I strongly suspect this as cause:

   bug 595300 - Update to NSS_3_12_8_RTM

Which was fixed in 3.6.11 and 3.5.14

Cheng can provide a test login/password to people who can debug what's happening here.
blocking1.9.1: --- → ?
blocking1.9.2: --- → ?
blocking2.0: --- → ?
Blocks: 595300
Assignee: nobody → nobody
Component: Security → Libraries
OS: Mac OS X → All
Product: Core → NSS
QA Contact: toolkit → libraries
Hardware: x86 → All
Ever confirmed: false
Assignee: nobody → bsmith


7 years ago
blocking1.9.1: ? → .17+
blocking1.9.2: ? → .14+
This was actually caused by http://hg.mozilla.org/releases/mozilla-1.9.2/rev/5d8b8a4a366a (confirmed via local bisect).
Assignee: bsmith → nobody
Component: Libraries → Networking
Product: NSS → Core
QA Contact: libraries → networking
In particular, this addition:

(the site in question is a k12.hi.us site)
Summary: Update to NSS_3_12_8_RTM breaks Lotus Notes webmail (at Hawaii DOE) → Lotus Notes webmail (at Hawaii DOE) broken by eTLD list update
Ever confirmed: true
Blocks: 576508
No longer blocks: 595300
The site is at http://lotusnotes.k12.hi.us/ .

In builds that work, the login process adds a cookie with the following attributes:

Name: LtpaToken
Domain: .k12.hi.us
Path: /

In builds that don't work, that cookie cannot be set (presumably because we now consider that domain to be "top level"), so the login fails.

Comment 4

7 years ago
Gerv, any idea what we should do here? I'm really not sure if they actually mean to set a cookie for .k12.hi.us; if that domain is used as a US-wide k12 single signon portal or something, then we should fix on our end; if not, we should get them to fix theirs.
Blocks: 428944

Comment 5

7 years ago
(I guess what I'm really asking is whether someone wants to get in touch with them to work through this.)
I think it actually probably is intended as a *.K12.hi.us login, since presumably all of the infra under that domain is under the control of the Hawaii DOE (comment 0 suggests that it affects multiple systems).

Comment 7

7 years ago
Oh, right, I misread as k12.us. So I think we probably want to remove all the k12.*.us rules...
Component: Networking → Networking: Domain Lists
QA Contact: networking → networking.domain-lists

Comment 8

7 years ago
Ok, I followed up with the tech support desk there.  They would like us to remove k12.hi.us from the list of eTLDs.  I was wondering if we could get that in for 3.6.13 since it's really really low risk?

Comment 9

7 years ago
(FWIW, it may be that we need to look at this on a state-by-state basis as it seems different states expect different behavior)

Comment 10

7 years ago
(In reply to comment #9)
> (FWIW, it may be that we need to look at this on a state-by-state basis as it
> seems different states expect different behavior)

Sounds fragile to me... the argument for keeping them all in there isn't fantasically strong, since different K12 schools in a given state aren't likely to try to mess with each other's logins. I'd just drop them all.

Thanks for doing the legwork Cheng!
presumptively assigning to gerv as the official Keeper of the Lists.

If we re-spin 3.5.16/3.6.13 it'd be nice to pick this up. Renominating to keep on the radar.
Assignee: nobody → gerv
blocking1.9.1: .17+ → ?
blocking1.9.2: .14+ → ?
Keywords: regression

Comment 12

7 years ago
I actually want to pick this up for 3.6.13 and 3.5.16. I'll send an email with the plan, but this is what I want:

* Keep the current builds live on the beta channel
* Have Toronto RelEng create build #2 of both with the .edu stuff backed out (not sure if this is possible currently)
* Push the build #2s to the beta audience on Monday or Tuesday
blocking1.9.1: ? → .16+
blocking1.9.2: ? → .13+

Comment 13

7 years ago
Just catching up here: I'm very happy to remove them, even pending further investigation. Removing them is pretty low risk, as you say. So do that, and I can think about the larger issues at leisure :-)


Comment 14

7 years ago
Sounds good. Can we get a patch added to the bug so I can approve and we can land on both the relbranches and default?

Comment 15

7 years ago
Created attachment 493313 [details] [diff] [review]
1.9.2 patch
Attachment #493313 - Flags: review?(gerv)

Comment 16

7 years ago
Created attachment 493314 [details] [diff] [review]
1.9.1 patch
Attachment #493314 - Flags: review?(gerv)


7 years ago
These patches essentially back out bug 428944. We'll fix Hawaii but re-break Indiana and other states (although the two breakages were not equal).

Is there going to be a trunk patch? Maybe the trunk could delete only HI and wait for feedback on the other states before removing them.

HI doesn't seem to have any cc.hi.us or lib.hi.us sites (according to a Google inurl: search). The states that do seem to have very local sites.

> since different K12 schools in a given state aren't likely
> to try to mess with each other's logins. I'd just drop them all.

The adults might not, but schools are full of students. I can easily imagine parts of a school website being created by students much the way the yearbook and weekly paper is. If messing with those pages could potentially gather credentials to a "student database" (grades?) that might be very very tempting.
Comment on attachment 493313 [details] [diff] [review]
1.9.2 patch

Attachment #493313 - Flags: review?(gerv) → review+
Comment on attachment 493314 [details] [diff] [review]
1.9.1 patch

Attachment #493314 - Flags: review?(gerv) → review+
For the branches just back out the whole thing. For trunk we should consider only backing out k12.HI.us and leave the rest until we get feedback.
1.9.2.x:  http://hg.mozilla.org/releases/mozilla-1.9.2/rev/69e253891ca3 http://hg.mozilla.org/releases/mozilla-1.9.2/rev/426f09f86038
1.9.1.x:  http://hg.mozilla.org/releases/mozilla-1.9.1/rev/26016fb65870 http://hg.mozilla.org/releases/mozilla-1.9.1/rev/612ad84a4004
status1.9.1: --- → .16-fixed
status1.9.2: --- → .13-fixed
Created attachment 493828 [details] [diff] [review]
trunk patch, only removes k12.hi.us

trunk patch to remove Hawaii from the k12 list.
Attachment #493828 - Flags: review?
Attachment #493828 - Flags: review? → review?(gerv)

Comment 23

7 years ago
OK, so the tension here is between the two different uses of the PSL. There's cookie setting, for which Hawaii wants to be able to set a cookie for hi.k12.us, and there's SSL indicator display, for which Indiana want to display foo.in.k12.us rather than in.k12.us.

In general, I have more sympathy with the latter view, and I think states any larger than Hawaii are less likely to have state-wide single signon across schools, libraries or community colleges. But the compat issues are smaller with the SSL indicator than with cookies.

So my view is that on the branches, we should back out the patch (better compat), and on trunk we should take this on a case-by-case basis for now. Let's back out hi.k12.us and see if we get other complaints as well. (Note bug 608362, where the company fixed their over-expansive cookie-setting.)

So basically, we should do what we just did :-)


Comment 24

7 years ago
Is it really the job of the Firefox team, the IE team, the Chrome team, the Opera team, and so on to debate the cookie scope for every possible domain? Shouldn't that be a decision for each domain owner?

How about something in the DNS records for hi.k12.us that says "cookies can be set for this domain" and something in "in.k12.us" that says "cookies can not be set for this domain, only subdomains"? Browsers would keep current rules, and gradually strip them back as the domains took over the work.

Comment 25

7 years ago
B.J.: there are internet-drafts (from Opera) for that sort of idea, but they haven't been ratified. And even if they were, they would need to be implemented on a wide enough scale that we could drop the PSL. If that happens, great. In the mean time, we need a list.



7 years ago
blocking2.0: ? → final+


7 years ago
Attachment #493828 - Flags: approval2.0+
Comment on attachment 493828 [details] [diff] [review]
trunk patch, only removes k12.hi.us

I'm interpreting comment 23 as an r=gerv, clegnitto agreed.
Attachment #493828 - Flags: review?(gerv) → review+
Last Resolved: 7 years ago
Resolution: --- → FIXED

Comment 28

7 years ago
Yep, r=gerv.

You need to log in before you can comment on or make changes to this bug.