1604786 - Level 2 Trackers are not being blocked on Firefox 72 beta

Status: RESOLVED INVALID

(Firefox :: Protections UI, defect)

Description

[Simona Badau, Desktop QA]

Mozilla/5.0 (X11; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0
Build ID: 20191218033533

Affected versions:

• Firefox 72 beta 7
• Firefox 72 beta 8

Affected platforms:

• Ubuntu 18.04 x64
• Windows 10
• Mac OS X 10.14

Steps to reproduce:

1. Open Firefox
2. Navigate to: https://senglehardt.com/test/trackingprotection/test_pages/tracking_protection.html
3. Look at the "Level 2 (Strict) List" section at the bottom of the page.

Actual results:
For Level 2 (Strict) cookies, the message "Cookies not blocked" is displayed.

Expected results:
For Level 2 (Strict) cookies, the displayed message should be "Cookies BLOCKED".

This issue is not reproducible on Firefox 72 beta 6 and neither is on the latest Nightly 73.0a1.

Comment 1

[Liz Henry (:lizzard) (relman/hg->git project)]

Erica, might this kind of test be automatable?

Comment 2

[Erica Wright [:ewright]]

passing this ni to Steven

Comment 3

[Steven Englehardt [:englehardt]]

The strict list is only enabled in Early beta or earlier (see: https://searchfox.org/mozilla-central/rev/6305f6935f496b3a302c7afcc579399a4217729c/modules/libpref/init/StaticPrefList.yaml#7190-7195). I'm actually not sure which version number beta moves from "early" to "late", but I suspect it's between 6 and 7 in this case. If so, then we can close this as INVALID.

(In reply to Liz Henry (:lizzard) from comment #1)

Erica, might this kind of test be automatable?

Yes, this test can be automated. It's possible to move these test pages in tree and automatically verify the results.

We might actually already have coverage for this though, so I'm going to further redirect this to Ehsan for his input on whether we'd see any additional test coverage by moving this in tree.

Comment 4

[Steven Englehardt [:englehardt]]

Liz: is 72b6 considered early beta and 72b7 and beyond considered late beta?

Comment 5

[Liz Henry (:lizzard) (relman/hg->git project)]

For this cycle, early beta flag was flipped on Dec. 13: https://hg.mozilla.org/releases/mozilla-beta/rev/d4dcd61db3a6f285e694301ecc84c80353a9fac4 but the next build and release wasn't until Monday/Tues, so, the first late beta release was beta 7.

Comment 6

[Steven Englehardt [:englehardt]]

(In reply to Liz Henry (:lizzard) from comment #5)

For this cycle, early beta flag was flipped on Dec. 13: https://hg.mozilla.org/releases/mozilla-beta/rev/d4dcd61db3a6f285e694301ecc84c80353a9fac4 but the next build and release wasn't until Monday/Tues, so, the first late beta release was beta 7.

Thanks! And now I know where to look.

Thus the strict list being disabled in beta 7 onwards for Firefox 72 is expected.

Comment 7

[Hani Yacoub, Desktop QA]

This bug was logged while we were testing "ETP: Strict list in standard privacy protections" feature (PI-304) in order to send a pre-release sign-off.
In this case, how should we continue testing in order to send a pre-release sign-off?
What's are the plans for release, is it going to be enabled or disabled?
Thanks.

Comment 8

[Julien Cristau [:jcristau]]

As far as I can tell from https://trello.com/c/Af9P95sq/814-etp-move-strict-list-into-standard-category-of-privacy-protections this is not expected to ship enabled in 72 but you can continue testing by flipping the pref back on to increase confidence in upcoming studies.

Comment 9

[Steven Englehardt [:englehardt]]

(In reply to Julien Cristau [:jcristau] from comment #8)

As far as I can tell from https://trello.com/c/Af9P95sq/814-etp-move-strict-list-into-standard-category-of-privacy-protections this is not expected to ship enabled in 72 but you can continue testing by flipping the pref back on to increase confidence in upcoming studies.

That's correct. We do not plan to ship this is 72, but will run a shield study during the 72 cycle (hence our desire to test it now).

To be specific, you can toggle privacy.annotate_channels.strict_list.enabled to true to continue testing.

Comment 10

[(no longer active)]

(In reply to Steven Englehardt [:englehardt] from comment #3)

We might actually already have coverage for this though, so I'm going to further redirect this to Ehsan for his input on whether we'd see any additional test coverage by moving this in tree.

We don't have any automated tests using the Disconnect list, since our test automation CI cannot access the remote network so it is unable to fetch the Disconnect block lists. As a subset of Disconnect list, we also don't have any automated tests using the "content" category of the Disconnect list (aka the strict list).