1350599 - Rewrite test_sts_preloadlist_perwindowpb.js and friends to not depend on bugzilla.mozilla.org or any other domain that can ever be removed from the HSTS preload list

Status: RESOLVED FIXED

(Core :: Security: PSM, defect, P1)

Description

[Phil Ringnalda (:philor)]

As https://hg.mozilla.org/mozilla-central/annotate/e03e0c60462c775c7558a1dc9d5cf2076c3cd1f9/security/manager/ssl/tests/unit/test_sts_preloadlist_perwindowpb.js#l1 says, we have a set of tests that depend on having bugzilla.mozilla.org, login.persona.org, and www.torproject.org included in the preload list.

Surprisingly, persona was not the one of those to break first (the "service has shut down" page apparently still pointlessly includes the hsts header), but because bugzilla.mozilla.org is currently mitigating an attack by sending strangers to hardhat.cdn.whatevertherestofthatis, the automatic updates of the preload list have removed bugzilla.mozilla.org, resulting in permaorange in test_sts_preloadlist_perwindowpb.js, test_sts_preloadlist_selfdestruct.js, test_sss_readstate.js, and test_sss_readstate_empty.js.

That's a great big party foul, having your test depend on hitting an external site over the network, even if the external site access is done in a separate task from the actual test run, so I'm disabling those tests on esr45/esr52/aurora/mozilla-central (even though they haven't yet broken on mozilla-central since updates are separately broken there).

Not sure what the clean solution is, since the only things guaranteed to remain on the list are "pins": "google", which leaves you at the mercy of Google not deciding to remove glass.google.com or something less imaginable like losing a trademark battle and losing control of www.gmail.com.

Comment 1

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/2ecf610d3185
disable hsts tests which depend on bugzilla.mozilla.org always being in the preload list, a=bustage

Comment 2

[Phil Ringnalda (:philor)]

And also disabled in

https://hg.mozilla.org/releases/mozilla-aurora/rev/997b98cce1aebf76641e14c79ac250cdb147041c
https://hg.mozilla.org/releases/mozilla-esr52/rev/f6a978b2fcecf67baa9b9bdd0dab0adbf93b8282
https://hg.mozilla.org/releases/mozilla-esr45/rev/9b2d1cbecb1dbe5cb14166ce0be029f67172c78b

Comment 3

[:Cykesiopka]

Indeed, making these assumptions was not the best idea.

Anyways, a solution that will work is to have the preload script always insert some test entries that we can then use in tests. This is what the HPKP script already does.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f9ef7edd12bae839ac0675646d90209c076c7caf

Comment 4

[:Cykesiopka]

Attachment: Bug 1350599 - Use guaranteed preloaded test domains instead of real domains in HSTS tests.

Some of our tests currently assume that certain real domains are HSTS preloaded.
While most of the time these domains are in fact preloaded, this may change
during periods of maintenance or other events.

To avoid this, the changes here perform the following renames:
bugzilla.mozilla.org -> includesubdomains.preloaded.test
login.persona.org -> includesubdomains2.preloaded.test
www.torproject.org -> noincludesubdomains.preloaded.test

In addition, some tests that refer to mozilla.com (but don't depend on it being
preloaded) are made to refer to example.com instead to avoid referring to real
domains in tests.

Review commit: https://reviewboard.mozilla.org/r/123884/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/123884/

Comment 5

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Comment on attachment 8851623 [details]
Bug 1350599 - Use guaranteed preloaded test domains instead of real domains in HSTS tests.

https://reviewboard.mozilla.org/r/123884/#review126494

Great - thanks.

Comment 6

[:Cykesiopka]

Thanks!

https://treeherder.mozilla.org/#/jobs?repo=try&revision=340c6e05b684cdbf967c3cf2110a931d7c1b9ea0
(As a gentle reminder, the changes in Bug 1350868 need to land first.)

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9cd5529dcc38
Use guaranteed preloaded test domains instead of real domains in HSTS tests. r=keeler

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/9cd5529dcc38

Comment 9

[:Cykesiopka]

Attachment: bug1350599_fix-hsts-tests_esr52v1.patch

This is the ESR 52 version of the patch.

Comment 10

[:Cykesiopka]

https://bugzilla.mozilla.org/attachment.cgi?id=8851623 is test only and should be uplifted to Aurora.
https://bugzilla.mozilla.org/attachment.cgi?id=8854954 is test only and should be uplifted to ESR 52.

Thanks!

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/198292931177

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr52/rev/0af73defbea2