1409910 - browser_certViewer.js fails intermittently under test-verify

Status: NEW

(Core :: Security: PSM, defect, P2)

Description

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

browser_certViewer.js fails intermittently under test-verify (to reproduce, make a meaningless change to the test file and run a task like test-linux64/debug-test-verify-e10s). See e.g. https://treeherder.mozilla.org/#/jobs?repo=try&revision=19fed7f6d649ca4429bdfb8737077376e346433e
This is concerning.

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

I think I'm hitting this in bug 1552714. According to Gijs, this test is only run when something in the test changes, which makes it easy to fall into "intermittent, but not hit often enough to disable" as evidenced by the number of intermittent failures filled against this test with INCOMPLETE resolution: https://bugzilla.mozilla.org/buglist.cgi?list_id=14750742&short_desc_type=allwordssubstr&short_desc=browser_certViewer.js&resolution=---&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=INACTIVE&resolution=DUPLICATE&resolution=WORKSFORME&resolution=INCOMPLETE&resolution=SUPPORT&resolution=EXPIRED&resolution=MOVED&classification=Client%20Software&classification=Developer%20Infrastructure&classification=Components&classification=Server%20Software&classification=Other&classification=Graveyard&query_format=advanced&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED

Is it enough to justify disabling this test?

Comment 2

[:Gijs (he/him)]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #1)

I think I'm hitting this in bug 1552714. According to Gijs, this test is only run when something in the test changes,

Test-verify only runs a test when that test changes. This test runs normally in a browser-chrome (bc) chunk.

Ideally, we should investigate why. I don't see the point of disabling the test for TV given the test doesn't change frequently anyway.

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

Here's my try attempt with a lot of console logging around - https://treeherder.mozilla.org/#/jobs?repo=try&revision=145a6dcc8c96120f6c452ceb1a34b022c2a703ff

Comment 4

[Zibi Braniecki [:zbraniecki][:gandalf]]

it seems to me that for some reason, in the failure in the try above, displayUsages not someSuccess is triggered where the test expects someSuccess to be true.

Comment 5

[Intermittent Failures Robot]

4 failures in 5197 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• macosx1014-64: 1
• macosx1014-64-shippable: 1
• windows10-64-shippable: 1
• linux64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1409910&startday=2019-06-03&endday=2019-06-09&tree=all