1117731 - Test failure 'Notification popup state has been opened' in testSecurity/testSSLStatusAfterRestart.js

Status: RESOLVED FIXED

Description

[Teodor Druta]

Module:    testDisplayCertificateStatusAfterRestart
Test:      testSecurity/testSSLStatusAfterRestart.js 
Failure:   Notification popup state has been closed
Branches:  mozilla-aurora
Platforms: Linux x86
Locale:    All 
Report:    http://mozmill-daily.blargon7.com/#/remote/report/5c7e7a22f9984b46554bc3b9fc44c734

Fails intermittently on Ubuntu 14.10, 14.04 x86.

Comment 1

[Teodor Druta]

Just a small note I have mistakenly reported the failure message wrong, it should be:
"Notification popup state has been opened"

Comment 2

[Teodor Druta]

This failed on Nightly, in the testDisplayCertificateStatus module, most probably it's the same issue.
http://mozmill-daily.blargon7.com/#/remote/report/04e54d608fc589a28217304fe674870e

Comment 3

[Mihaela Velimiroviciu (:mihaelav)]

Failed again on 36.0b1:
http://mozmill-release.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2015-01-06&test=%2FtestSecurity%2FtestSSLStatusAfterRestart.js&func=testDisplayCertificateStatusAfterRestart

Comment 4

[Andreea Matei [:AndreeaMatei]]

Failed about 20 times since friday on all platforms, aurora and nightly.

On beta I see it's only ubuntu 14.10 affected, but nightly and aurora has all:
http://mozmill-daily.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2015-01-16&to=&test=%2FtestSecurity%2FtestSSLStatusAfterRestart.js&func=testDisplayCertificateStatus

Comment 5

[Daniela Domnici]

Failed again on beta 36.0b5

Reports: http://mozmill-release.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2015-01-29&to=2015-01-30&test=%2FtestSecurity%2FtestSSLStatusAfterRestart.js&func=testDisplayCertificateStatus

Comment 6

[Daniela Domnici]

Created attachment 8558377 [details] [diff] [review]
skip patch for beta

Created the skip patch for beta branch.

Comment 7

[Andreea Matei [:AndreeaMatei]]

https://hg.mozilla.org/qa/mozmill-tests/rev/3385ed64bddb (beta)

Comment 8

[Daniela Domnici]

I`ve investigated today this bug and I fixed it, tomorrow I `ll run  more testruns on ubuntu 14.04 production machine.Today I`ve did ~30 testruns on ubuntu 14.04 staging machine and ~20 testruns on ubuntu 14.04 production machine.
Here are the results: http://mozmill-crowd.blargon7.com/#/remote/reports?app=All&branch=36&platform=Linux&from=2015-02-03&to=2015-02-03

Comment 9

[Daniela Domnici]

Created attachment 8559115 [details] [diff] [review]
fix testSSLStatusAfterRestart.js

Created a fix patch for this test.This patch also fix the bug 1122541. I`ve did ~150 testruns on ubuntu 14.04-32-2 production machine.

Reports:http://mozmill-crowd.blargon7.com/#/remote/reports?app=All&branch=All&platform=All&from=2015-02-04&to=2015-02-04

Comment 10

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8559115 [details] [diff] [review]
fix testSSLStatusAfterRestart.js

Review of attachment 8559115 [details] [diff] [review]:
-----------------------------------------------------------------

::: firefox/tests/remote/testSecurity/testSSLStatusAfterRestart.js
@@ +51,5 @@
>  function teardownTest(aModule) {
>    if (persisted.nextTest) {
>      controller.restartApplication(persisted.nextTest);
>    }
> +  aModule.tabBrowser.closeAllTabs();

Why do we need to close tabs after each test?

Comment 11

[Mihaela Velimiroviciu (:mihaelav)]

Comment on attachment 8559115 [details] [diff] [review]
fix testSSLStatusAfterRestart.js

Review of attachment 8559115 [details] [diff] [review]:
-----------------------------------------------------------------

::: firefox/tests/remote/testSecurity/testSSLStatusAfterRestart.js
@@ +51,5 @@
>  function teardownTest(aModule) {
>    if (persisted.nextTest) {
>      controller.restartApplication(persisted.nextTest);
>    }
> +  aModule.tabBrowser.closeAllTabs();

This line of code is not reached

@@ -55,4 @@
>  }
>  
>  function teardownModule(aModule) {
> -  aModule.tabBrowser.closeAllTabs();

Please add this line back

Comment 12

[Daniela Domnici]

Please ignore the patch above.It seems that Cosmin`s closeTabs refactor solve this problem. Today, I`ve did 100 remote - testruns using 14.04-32-2 production machine and it didn`t fail at all.

Here are the results: 
http://mozmill-crowd.blargon7.com/#/remote/reports?app=All&branch=36&platform=All&from=2015-02-05&to=2015-02-05.

Comment 13

[Andreea Matei [:AndreeaMatei]]

Great. Then we can mark this as fixed for now. Hopefully won't fail again :)
https://hg.mozilla.org/qa/mozmill-tests/rev/17fa1493b539 (beta)