xpcshell failures on all platforms and trees due to failures in test_sts_preloadlist_perwindowpb.js

RESOLVED FIXED in Firefox 22

Status

()

defect
--
blocker
RESOLVED FIXED
7 years ago
6 years ago

People

(Reporter: armenzg, Assigned: keeler)

Tracking

unspecified
mozilla23
x86
macOS
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox22 fixed, firefox23 fixed)

Details

Attachments

(1 attachment)

All xcpshell jobs on all trees are going orange:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=xpcshell

08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_perwindowpb.js | test failed (with xpcshell return code: 0), see following log:
08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_perwindowpb.js | false == true - See following stack:
08:50:19 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/head.js | 2147500036 - See following stack:
08:50:20 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_selfdestruct.js | test failed (with xpcshell return code: 0), see following log:
08:50:20 WARNING - TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/xpcshell/tests/security/manager/ssl/tests/unit/test_sts_preloadlist_selfdestruct.js | false == true - See following stack:
09:21:20 ERROR - Return code: 1

There are speculations that it could be related to external sites' re-directions but it could also be a red herring:
https://tbpl.mozilla.org/php/getParsedLog.php?id=20225800&tree=Mozilla-Inbound&full=1

I don't have more info than that. We mainly wanted a placeholder to figure this out.
CCing people who've touched this test (and it's parent test). It appears it's hitting the network?
All trees are currently closed for this issue.
Severity: normal → blocker
Summary: xpcshell issues → xpcshell failures on all platforms and trees due to failures in test_sts_preloadlist_perwindowpb.js
Component: General → Security
Product: Testing → Core
I noticed that the test was failing on an STS check for factor.cc and that https://factor.cc/ is down with some sort of redirect loop.  However, I think the redirect part is a red herring because the test also fails for the next host (arivo.com.br) and possibly others.

If this test depends on external network resources, we should disable it on all trees.
https://mxr.mozilla.org/mozilla-central/source/security/manager/boot/src/nsSTSPreloadList.inc#11 :

const PRTime gPreloadListExpirationTime = INT64_C(1362156597190000);

It's now about 1362167542000000. This test is failing because we've gone too long without updating the preload list. Basically, we need bug 836097 to be finished (it's almost there).

Options as I see them:
1. Change just the expiration time until bug 836097 is done
2. Disable the test until the same
3. Check in an entirely new preload list (i.e. manually update it)

Anyway, we'll need to update that test because factor.cc is going to be removed from the preload list, it looks like.
I can't get ahold of keeler on IRC, so I'm going to disable the test for now.
If it is actually hitting the network and depending on hitting those sites, and it has to hit them rather than be rewritten to hit something locally proxied, then please do immediately disable it on every branch, and file a bug for turning it back on that depends on a bug for adding a new test suite called something like NetworkCausedFlakiness, where we will know that test failures usually mean nothing.
Wound up being unclear from the midair, what I meant was "now that RyanVM is disabling it everywhere, if you really are hitting the network then please don't reenable it until you're run in a new separate suite which..."
I can say with near certainty that these tests do not hit the network. I'll double-check when I get them working again after bug 836097 is done.
Somebody could also just review bug 844527, and we could land that. :/
Whiteboard: [leave open] → [test disabled][leave open]
Now that the HSTS preload list updating is automatic (bug 836097), we can re-enable these tests. I did have to replace one host with another because it no longer sets includeSubdomains. I also made sure that the test does not hit the network at all.
Attachment #743191 - Flags: review?(honzab.moz)
Attachment #743191 - Flags: review?(honzab.moz) → review+
https://hg.mozilla.org/releases/mozilla-aurora/rev/c305e40ffe33

This is now fixed in central and aurora. Since I don't think we'll be uplifting the HSTS preload list updates required to re-enable this on beta/release/etc., I think we can call this fixed.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Whiteboard: [test disabled][leave open]
Target Milestone: --- → mozilla23
You need to log in before you can comment on or make changes to this bug.