All xcpshell jobs on all trees are going orange: https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=xpcshell 08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_perwindowpb.js | test failed (with xpcshell return code: 0), see following log: 08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_perwindowpb.js | false == true - See following stack: 08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/head.js | 2147500036 - See following stack: 08:50:20 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_selfdestruct.js | test failed (with xpcshell return code: 0), see following log: 08:50:20 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_selfdestruct.js | false == true - See following stack: 09:21:20 ERROR - Return code: 1 There are speculations that it could be related to external sites' re-directions but it could also be a red herring: https://tbpl.mozilla.org/php/getParsedLog.php?id=20225800&tree=Mozilla-Inbound&full=1 I don't have more info than that. We mainly wanted a placeholder to figure this out.
CCing people who've touched this test (and it's parent test). It appears it's hitting the network?
All trees are currently closed for this issue.
Severity: normal → blocker
Summary: xpcshell issues → xpcshell failures on all platforms and trees due to failures in test_sts_preloadlist_perwindowpb.js
I noticed that the test was failing on an STS check for factor.cc and that https://factor.cc/ is down with some sort of redirect loop. However, I think the redirect part is a red herring because the test also fails for the next host (arivo.com.br) and possibly others. If this test depends on external network resources, we should disable it on all trees.
https://mxr.mozilla.org/mozilla-central/source/security/manager/boot/src/nsSTSPreloadList.inc#11 : const PRTime gPreloadListExpirationTime = INT64_C(1362156597190000); It's now about 1362167542000000. This test is failing because we've gone too long without updating the preload list. Basically, we need bug 836097 to be finished (it's almost there). Options as I see them: 1. Change just the expiration time until bug 836097 is done 2. Disable the test until the same 3. Check in an entirely new preload list (i.e. manually update it) Anyway, we'll need to update that test because factor.cc is going to be removed from the preload list, it looks like.
I can't get ahold of keeler on IRC, so I'm going to disable the test for now.
If it is actually hitting the network and depending on hitting those sites, and it has to hit them rather than be rewritten to hit something locally proxied, then please do immediately disable it on every branch, and file a bug for turning it back on that depends on a bug for adding a new test suite called something like NetworkCausedFlakiness, where we will know that test failures usually mean nothing.
Wound up being unclear from the midair, what I meant was "now that RyanVM is disabling it everywhere, if you really are hitting the network then please don't reenable it until you're run in a new separate suite which..."
Tests disabled: https://hg.mozilla.org/integration/mozilla-inbound/rev/98cb2dc01041 https://hg.mozilla.org/releases/mozilla-aurora/rev/7705597bb9cb https://hg.mozilla.org/releases/mozilla-beta/rev/d2b1b94590f6 https://hg.mozilla.org/releases/mozilla-b2g18/rev/614f3551dc58 https://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/35c772794719 https://hg.mozilla.org/releases/mozilla-esr17/rev/54bf819db766 https://hg.mozilla.org/releases/mozilla-release/rev/011ac6cdddad
Assignee: nobody → dkeeler
Depends on: 836097
Whiteboard: [leave open]
I can say with near certainty that these tests do not hit the network. I'll double-check when I get them working again after bug 836097 is done.
Somebody could also just review bug 844527, and we could land that. :/
Now that the HSTS preload list updating is automatic (bug 836097), we can re-enable these tests. I did have to replace one host with another because it no longer sets includeSubdomains. I also made sure that the test does not hit the network at all.
Attachment #743191 - Flags: review?(honzab.moz)
Attachment #743191 - Flags: review?(honzab.moz) → review+
https://hg.mozilla.org/releases/mozilla-aurora/rev/c305e40ffe33 This is now fixed in central and aurora. Since I don't think we'll be uplifting the HSTS preload list updates required to re-enable this on beta/release/etc., I think we can call this fixed.
You need to log in before you can comment on or make changes to this bug.