1578597 - Firefox 69 has stopped storing cookies for 3rd level domains

Status: RESOLVED INVALID

(Core :: Networking: Cookies, defect)

Description

[hhhh7777]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Our domain is name.edu.au
We have several apps within this domain, for example: app1.name.edu.au and app2.name.edu.au
To allow single sign on to function across all apps within our domain we set cookies at the name.edu.au level
This has worked across Firefox, Chrome, IE and Edge browsers for over a decade
Firefox 69 is no longer storing the cookies and our applications fail to login successfully because the required cookie is not being stored and sent back in subsequent requests.
Chrome, IE & Edge continue to work as expected.

I have tried disabling Enhanced Tracking Protection, added the app url app1.name.edu.au to the Enhanced Tracking Protection exceptions list. Also added name.edu.au, .name.edu.au and app1.name.edu.au as "Allow" to the "Exceptions - Cookies & Site Data" list.

Actual results:

Cookie being set by the web server is not being stored in Firefox and is not being sent back to the server on subsequent requests.
F12 Developer tools > Storage panel > Cookies never lists the name.edu.au cookie.

Expected results:

Cookie stored in cookie list for the app's URL and cookie returned to the server on subsequent requests.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Hi Nhi, can you please help find someone to investigate this new regression in 69?

Comment 2

[Nhi Nguyen (:nhi)]

Honza, could you take a look? Is this necko or content security?

Comment 3

[(no longer active)]

hhhh7777, thanks for your bug report.

May I ask what name is in name.edu.au? (If this is information that you are not comfortable sharing publicly I would appreciate if you could please email me about it privately.)

Comment 4

[Honza Bambas (:mayhemer)]

A life test case would be best to check with. Few more questions: what path is the cookie set for? is it secure only? is it http only? is it samesite? How exactly the storage inspector looks like? Thanks.

Comment 5

[Florencia Di Ciocco, NI to vbalducci]

Hi all,
I was able to reproduce this bug on Windows 10. As it was mentioned before, this does not happen on nightly (71.0a1 (2019-09-04) (64-bit)) so I will do the regression manually. On the other hand, I will leave you the steps to reproduce this with a workaround.

1. Log in on https://store.churchofjesuschrist.org/arg/es and save the user and pwd. Anyone can create a user as "visitor" so it's cool to test.
2. log out
3. Log in on https://www.churchofjesuschrist.org/?lang=eng.
   On Nightly I can use the same credentials from step 1 to log in on step 3. On Release (68.0, 68.02 or 69.0) I have to type in the credentials again. I'll attach a picture with my credentials (duplicated) from release.

As soon as I'm able to do the regression I'll post another message.

Regards, Flor.

Comment 6

[Florencia Di Ciocco, NI to vbalducci]

Attachment: RELEASE.png

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 8

[Florencia Di Ciocco, NI to vbalducci]

Hi again,

So I've done the regression, though it wasn't so much a regression as an improvement. I noticed that this hasn't been working on Nightly either until recently:

71.0a1 (2019-09-04) -> WORKS
69.0a1 (2019-06-01) -> WORKS
68.0a1 (2019-03-26) -> DOESN'T WORK
67.0a1 (2019-02-01) -> DOESN'T WORK
59.0a1 (2018-01-02) -> DOESN'T WORK

Here's the pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=af54b2de7028db03f42207598f7a0b4ba81e262f&tochange=90f17d7076dff559c70f81af55cd7612256e3e3e.

Best regards, Flor.

Comment 9

[hhhh7777]

(In reply to :ehsan akhgari from comment #3)

hhhh7777, thanks for your bug report.

May I ask what name is in name.edu.au? (If this is information that you are not comfortable sharing publicly I would appreciate if you could please email me about it privately.)

"name" is the name of our educational institution within the .edu.au namespace.

.edu.au is a public namespace (not sure if I'm using the correct terminology) similar to .com and I can understand why cookies set for .edu.au would be ignored as this would allow tracking between sites belonging to different institutions, again similar to how cookies set for a domain of .com should ignored.

However "name.edu.au" belongs to our institution and we should be able to (and have previously been able to) set cookies at this level for use across multiple apps within our domain namespace. Firefox prior to 69 allowed setting these cookies, and Chrome, Edge and IE still allow cookies to be set at this level.

PS I am able to replicate this issue on Ff Dev Edition (70.0b3).

thx

Comment 10

[hhhh7777]

(In reply to Honza Bambas (:mayhemer) from comment #4)

A life test case would be best to check with. Few more questions: what path is the cookie set for? is it secure only? is it http only? is it samesite? How exactly the storage inspector looks like? Thanks.

Sorry, you cannot see the bug on our site unless you have a login and we have strict student protection laws that prevent us from creating logins for people not associated with our institution.

As for the cookieattributes the Set-Cookie header returned by the web server (with values redacted) is:
set-cookie: cookie.name=value; Path=/; Domain=name.edu.au; Secure; HttpOnly

In the storage inspector, the cookie simply does not show up. Other cookies set at the app.name.edu.au level show up as expected.

thx

Comment 11

[Ryan VanderMeulen [:RyanVM]]

(In reply to Florencia Marina Di Ciocco from comment #5)

1. Log in on https://store.churchofjesuschrist.org/arg/es and save the user and pwd. Anyone can create a user as "visitor" so it's cool to test.
2. log out
3. Log in on https://www.churchofjesuschrist.org/?lang=eng.
   On Nightly I can use the same credentials from step 1 to log in on step 3. On Release (68.0, 68.02 or 69.0) I have to type in the credentials again. I'll attach a picture with my credentials (duplicated) from release.

That's not the issue this bug is reporting. What you're seeing is bug 589628 where saved login information can be used on subdomains. This bug is about not losing login sessions across subdomains (i.e. no Step 2 in the STR in this comment).

Comment 12

[Honza Bambas (:mayhemer)]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #11)

Thanks, Ryan.

Comment 13

[Honza Bambas (:mayhemer)]

Can reproduce with an emulated subdomain (4th level) on act.edu.au, insecure with a PHP:

setcookie("TEST", "VALUE", 0, "/", "act.edu.au", FALSE, TRUE);

works (cookie is stored) when accessed on act.edu.au

going to look.

Comment 14

[Honza Bambas (:mayhemer)]

2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie ===== COOKIE NOT ACCEPTED =====
2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie request URL: http://sub1.act.edu.au/bugs/1578597/play1.php
2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie cookie string: TEST=1100675750; path=/; domain=act.edu.au; HttpOnly
2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie current time: Thu Sep 05 14:07:39 2019 GMT
2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie rejected because failed the domain tests
2019-09-05 14:07:39.686000 UTC - [Parent 27924: Main Thread]: W/cookie 

Comment 15

[Honza Bambas (:mayhemer)]

Before I jumped on regression bisection, I tested in 68.0.2. My test case fails as well there, so I'm not sure this is a recent regression or in what release this has actually broke.

Hmmm...

Comment 16

[Ryan VanderMeulen [:RyanVM]]

There was an eTLD update which landed on Fx69 which looks possibly-relevant here:
https://hg.mozilla.org/releases/mozilla-beta/diff/0869bbb117034bf98f15d540763e117507d1b27f/netwerk/dns/effective_tld_names.dat

Reporter, can you maybe confirm if your affected domain is one of the green lines shown at the top of that link?

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Said changes came in from this upstream Pull Request:
https://github.com/publicsuffix/list/pull/825

Comment 18

[Honza Bambas (:mayhemer)]

Based on Ryan's comments (thanks, Ryan!) and that I see no difference with my test case between 65, 68 and 69 in behavior for act.edu.au (is public) and xxx.edu.au (non-public, arbitrary), we can conclude that the name from your report is probably one of the added public suffix domains.

This bug is then likely INVALID, waiting for reply from hhhh7777.

Comment 19

[hhhh7777]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #16)

There was an eTLD update which landed on Fx69 which looks possibly-relevant here:
https://hg.mozilla.org/releases/mozilla-beta/diff/0869bbb117034bf98f15d540763e117507d1b27f/netwerk/dns/effective_tld_names.dat

Reporter, can you maybe confirm if your affected domain is one of the green lines shown at the top of that link?

Yes our domain is one of those green lines. That explains what's going on.
I guess we need to talk to Education Services Australia to talk about why they added our domain to the public suffixes list??
I'm guessing this change will make its way into Chrome and Edge at some stage as well?

thx

Comment 20

[Ryan VanderMeulen [:RyanVM]]

(In reply to hhhh7777 from comment #19)

Yes our domain is one of those green lines. That explains what's going on.

Glad we were able to get to the bottom of it!

I guess we need to talk to Education Services Australia to talk about why they added our domain to the public suffixes list??

Sounds like it. FWIW, we sync with the upstream PSL twice a week all the way through the end of the Beta cycle for any given Firefox release, so any changes made there should make their way to a shipping release in relatively short order :)

I'm guessing this change will make its way into Chrome and Edge at some stage as well?

I would assume so, but I have no idea how often they do so in practice.