1584280 - Intermittent toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Subject Name must be the selected item - "" == "Subject Name" -

Status: RESOLVED FIXED

(Firefox :: Security, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nerli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=268593806&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/ER3KEGF6Q6KvuaAiR__pWA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-09-26T16:05:37.400Z] 16:05:37 INFO - TEST-INFO | started process screenshot
[task 2019-09-26T16:05:37.491Z] 16:05:37 INFO - TEST-INFO | screenshot: exit 0
[task 2019-09-26T16:05:37.491Z] 16:05:37 INFO - Buffered messages logged at 16:05:34
[task 2019-09-26T16:05:37.491Z] 16:05:37 INFO - Entering test bound runTests
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - Buffered messages logged at 16:05:35
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Error section was not found - null == null -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - Buffered messages logged at 16:05:36
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tabs section found - {} == true -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | There must be 2 tabs - 2 == 2 -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab name must be equal to expected tab name - "www.mozilla.org" == "www.mozilla.org" -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | First tab must be selected - true == true -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab name must be equal to expected tab name - "github.com" == "github.com" -
[task 2019-09-26T16:05:37.492Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Just the first tab must be selected - false == false -
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - Buffered messages logged at 16:05:37
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab at index 0 found - {} == true -
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab name should be www.mozilla.org - "www.mozilla.org" == "www.mozilla.org" -
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | infoGroup found - {} == true -
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - Buffered messages finished
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Subject Name must be the selected item - "" == "Subject Name" -
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - Stack trace:
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - resource://testing-common/content-task.js line 62 > eval:checkSelectedTab:31
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - resource://testing-common/content-task.js line 62 > eval:null:47
[task 2019-09-26T16:05:37.493Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | sectionItems must be the same length - 6 == 6 -
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab at index 1 found - {} == true -
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Tab name should be github.com - "github.com" == "github.com" -
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | infoGroup found - {} == true -
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - Not taking screenshot here: see the one that was previously logged
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | Subject Name must be the selected item - "" == "Subject Name" -
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - Stack trace:
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - resource://testing-common/content-task.js line 62 > eval:checkSelectedTab:31
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - resource://testing-common/content-task.js line 62 > eval:null:47
[task 2019-09-26T16:05:37.494Z] 16:05:37 INFO - TEST-PASS | toolkit/components/certviewer/tests/browser/browser_handleMultipleCertsURL.js | sectionItems must be the same length - 9 == 9 -
[task 2019-09-26T16:05:37.495Z] 16:05:37 INFO - GECKO(9920) | ++DOCSHELL 0000024470760000 == 2 [pid = 4560] [id = {7c1452bc-f585-4d39-a221-a27fbd6cd3c0}]
[task 2019-09-26T16:05:37.495Z] 16:05:37 INFO - GECKO(9920) | ++DOMWINDOW == 11 (0000024473A14F20) [pid = 4560] [serial = 17] [outer = 0000000000000000]
[task 2019-09-26T16:05:37.495Z] 16:05:37 INFO - GECKO(9920) | ++DOMWINDOW == 12 (0000024470C71800) [pid = 4560] [serial = 18] [outer = 0000024473A14F20]
[task 2019-09-26T16:05:37.495Z] 16:05:37 INFO - GECKO(9920) | ++DOMWINDOW == 13 (00000244743ED000) [pid = 4560] [serial = 19] [outer = 0000024473A14F20]
[task 2019-09-26T16:05:37.495Z] 16:05:37 INFO - GECKO(9920) | [Child 3968, Main Thread] WARNING: NS_ENSURE_TRUE(mDocShell) failed: file z:/build/build/src/toolkit/components/browser/nsWebBrowser.cpp, line 617
[task 2019-09-26T16:05:37.504Z] 16:05:37 INFO - GECKO(9920) | --DOCSHELL 000002AD5C510800 == 6 [pid = 3968] [id = {227f57cb-f5a0-4146-b60c-93b6ce5a843e}] [url = about:certificate?cert=MIIIIDCCBwigAwIBAgIQGk0sGNQUuaOL9Rii2XQ4yjANBgkqhkiG9w0BAQsFADBUMQswCQYDVQQGEwJVUzEeMBwGA1UEChMVR29vZ2xlIFRydXN0IFNlcnZpY2VzMSUwIwYDVQQDExxHb29nbGUgSW50ZXJuZXQgQXV0aG9yaXR5IEczMB4XDTE5MDYxODA4MjE1OFoXDTE5MDkxMDA4MTUwMFowZjELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDU1vdW50YWluIFZpZXcxEzARBgNVBAoMCkdvb2dsZSBMTEMxFTATBgNVBAMMDCouZ29vZ2xlLmNvbTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABMRScn8kk6qy3LHVktWZWxm%2FMq4kowlEdxQH40wijThZ%2B%2F5Jrqh6UlWnWuiulNorHH2DEW4OkSKreFoYze7w8O6jggWlMIIFoTATBgNVHSUEDDAKBggrBgEFBQcDATAOBgNVHQ8BAf8EBAMCB4AwggRqBgNVHREEggRhMIIEXYIMKi5nb29nbGUuY29tgg0qLmFuZHJvaWQuY29tghYqLmFwcGVuZ2luZS5nb29nbGUuY29tghIqLmNsb3VkLmdvb2dsZS5jb22CGCouY3Jvd2Rzb3VyY2UuZ29vZ2xlLmNvbYIGKi5nLmNvgg4qLmdjcC5ndnQyLmNvbYIRKi5nY3BjZG4uZ3Z0MS5jb22CCiouZ2dwaHQuY26CFiouZ29vZ2xlLWFuYWx5dGljcy5jb22CCyouZ29vZ2xlLmNhggsqLmdvb2dsZS5jbIIOKi5nb29nbGUuY28uaW6CDiouZ29vZ2xlLmNvLmpwgg4qLmdvb2dsZS5jby51a4IPKi5nb29nbGUuY29tLmFygg8qLmdvb2dsZS5jb20uYXWCDyouZ29vZ2xlLmNvbS5icoIPKi5]

Comment 1

[Intermittent Failures Robot]

1 failures in 4530 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 2

[Intermittent Failures Robot]

1 failures in 3801 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-10-07&endday=2019-10-13&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 4314 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-10-28&endday=2019-11-03&tree=all

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 5

[Andrei Ciure[:aciure]]

New occurrence:

https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=b26a70a0fe8f22ee4a5118c7c563ab98a115e692&selectedJob=280803757&searchStr=OS%2CX%2C10.14%2Cdebug%2CMochitests%2Ctest-macosx1014-64%2Fdebug-mochitest-browser-chrome-e10s-2%2CM%28bc2%29

Comment 6

[Intermittent Failures Robot]

2 failures in 4867 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1

Platform breakdown:

• linux64: 1
• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-12-09&endday=2019-12-15&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 4582 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-12-16&endday=2019-12-22&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 1862 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2019-12-30&endday=2020-01-05&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 3773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-01-13&endday=2020-01-19&tree=all

Comment 10

[Intermittent Failures Robot]

1 failures in 4004 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-01-20&endday=2020-01-26&tree=all

Comment 11

[Intermittent Failures Robot]

4 failures in 3364 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2
• autoland: 2

Platform breakdown:

• linux64: 2
• macosx1014-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-02-03&endday=2020-02-09&tree=all

Comment 12

[Intermittent Failures Robot]

4 failures in 4690 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• linux64: 3
• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-02-10&endday=2020-02-16&tree=all

Comment 13

[Intermittent Failures Robot]

3 failures in 4045 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• linux64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 14

[Intermittent Failures Robot]

1 failures in 5233 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform breakdown:

• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 15

[Intermittent Failures Robot]

1 failures in 5483 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 16

[Intermittent Failures Robot]

4 failures in 5336 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• mozilla-central: 1

Platform breakdown:

• macosx1014-64: 2
• linux1804-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 17

[Intermittent Failures Robot]

5 failures in 4955 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5

Platform breakdown:

• macosx1014-64: 4
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-03-16&endday=2020-03-22&tree=all

Comment 18

[Intermittent Failures Robot]

5 failures in 4773 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 4

Platform breakdown:

• linux1804-64: 2
• macosx1014-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-03-23&endday=2020-03-29&tree=all

Comment 19

[Zibi Braniecki [:zbraniecki][:gandalf]]

My guess is that I changed this to perma-fail with bug 1617669 - which means that the code (likely related to some web component) was intermittent before because it doesn't properly hook Fluent into shadowDOM, and relied on a racy condition of the template being accidentally localized.

This code should be fixed.

Comment 20

[Zibi Braniecki [:zbraniecki][:gandalf]]

Attachment: Bug 1584280 - Hook Fluent into the certviewer web components on connected callback. r?johannh

Comment 21

[Natalia Csoregi [:nataliaCs]]

Attachment: Bug 1584280 - Disable browser_handleMultipleCertsURL.js for turning into permafail r=fix

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11e18648ff9f
Disable browser_handleMultipleCertsURL.js for turning into permafail r=fix CLOSED TREE

Comment 23

[Johann Hofmann [:johannh]]

Things I don't understand:

• Bug 1617669 hasn't landed. How did it make this test permafail?
• Why are we disabling the test instead of backing out (or rather, preventing from landing) the offending patch?
• How, why, and by which authority did we decide that we can just land commits with r=fix without notifying the component owner, triage owner or test author? It would be great if there was public communication around this.

Thank you.

Comment 24

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/11e18648ff9f

Comment 25

[Zibi Braniecki [:zbraniecki][:gandalf]]

Hi Johann,

(In reply to Johann Hofmann [:johannh] from comment #23)

Things I don't understand:

• Bug 1617669 hasn't landed. How did it make this test permafail?

It did land last night.

• Why are we disabling the test instead of backing out (or rather, preventing from landing) the offending patch?

The fact that the test was intermittent for 6 months without any reaction indicates that the test is not of the highest importance.

Code that turns intermittent tests into perma-fails usually simplifies debugging and often indicates a bug in the test, rather than in the original patch. In this particular case, the test also does something tests shouldn't be doing - https://firefox-source-docs.mozilla.org/l10n/fluent/tutorial.html#testing

• How, why, and by which authority did we decide that we can just land commits with r=fix without notifying the component owner, triage owner or test author? It would be great if there was public communication around this.

I take blame for this.
Seeing the above I decided to ask for a temporary disabling of the intermittent test, rather than backing out of the patch that fixes a broken behavior.
I don't think this is the best reaction, and I will be leaning toward reverting even a good patch and investigating such regression further next time.
I'm sorry.

Comment 26

[Johann Hofmann [:johannh]]

Ok, thanks for the explanation and the chat on Matrix and no worries, this wasn't much trouble for me (you fixed everything after all), I just wanted to understand what was going on :)

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5599381de53c
Hook Fluent into the certviewer web components on connected callback. r=johannh

Comment 28

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/5599381de53c

Comment 29

[Intermittent Failures Robot]

10 failures in 5015 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 10

Platform breakdown:

• linux1804-64: 3
• macosx1014-64: 2
• windows10-64: 1
• windows10-64-qr: 2
• windows7-32: 1
• linux1804-64-asan: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1584280&startday=2020-03-30&endday=2020-04-05&tree=all