Closed Bug 922967 Opened 7 years ago Closed 7 years ago

Test failure "Identity is unknown - 'verifiedIdentity' should equal 'unknownIdentity'" in /testSecurity/testSecurityNotification.js

Categories

(Mozilla QA Graveyard :: Mozmill Tests, defect, P2)

defect

Tracking

(firefox27 fixed, firefox28 fixed, firefox29 fixed, firefox30 fixed, firefox-esr17 wontfix, firefox-esr24 fixed)

RESOLVED FIXED
Tracking Status
firefox27 --- fixed
firefox28 --- fixed
firefox29 --- fixed
firefox30 --- fixed
firefox-esr17 --- wontfix
firefox-esr24 --- fixed

People

(Reporter: mario.garbi, Assigned: cosmin-malutan)

References

()

Details

(Whiteboard: [mozmill-test-failure])

Attachments

(3 files)

I also tried to reproduce it on 'mm-win-8-32-2' running both single tests using mozmill -t and testrun_remote.py script. I was not able to reproduce it as can be seen in the reports:

http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750bde5bb
http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be0151
http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be285e
What fails in detail given by the stack information from the tests?
The value for "identityBox.getNode().className" did not changed after loading the "unsecure HTTP site" at line 35 and so remained "verifiedIdentity". This caused a failure at line 38 that can be seen in the link below:
 
http://hg.mozilla.org/qa/mozmill-tests/file/541b0cb3957ff33b7b7991e0ccde4b6e80e77af7/tests/remote/testSecurity/testSecurityNotification.js#l38

I suspect this was caused by a 404 when loading the page at line 35 because the waitForPageLoad() was not triggered. The fact that this is not reproducible now indicates a possible network issue that got fixed in the meanwhile. I will look over the failing machine as soon as it reproduces again to better understand what happens.
With the upgrade to Mozmill 2.0 we should not fetch those broken page loads. Lets see if it happens again and we get more information.
Happened again today on Linux Ubuntu 13.04 (x86) with Firefox 26.0a2:
http://mozmill-daily.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750eb2aac
We should skip this if it fails that much on Beta especially.
Assignee: nobody → cosmin.malutan
Status: NEW → ASSIGNED
Attached patch skip.patchSplinter Review
Here is the skip patch for beta.

I tried to reproduce this failure on affected machines yesterday and today again, without success.
http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a696d
http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a5bf6

Most likely it fails because waitforpagetoload passes before the page loads at line 
http://hg.mozilla.org/qa/mozmill-tests/file/b3f578c0111d/tests/remote/testSecurity/testSecurityNotification.js#l36
As Henrik said on comment 4 we shouldn't see those failures again once we have mozmill-2.0 on ci
Attachment #814340 - Flags: review?(andrei.eftimie)
Attachment #814340 - Flags: review?(andreea.matei)
Comment on attachment 814340 [details] [diff] [review]
skip.patch

Review of attachment 814340 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled: 
http://hg.mozilla.org/qa/mozmill-tests/rev/b5a65140e3dc (beta)

We still want to disable on all affected branches as I saw a failure on esr24 as well today.

Also, please try to reproduce on the remote machines or watch a few runs there. Today reproduced quite often, it might be an issue with the page, but we cannot know for sure until we catch it.
Attachment #814340 - Flags: review?(andrei.eftimie)
Attachment #814340 - Flags: review?(andreea.matei)
Attachment #814340 - Flags: review+
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure][mozmill-test-skipped]
Attached patch skip_esr17.patchSplinter Review
Here is the skip patch for esr17, the previous patch applies cleanly on  aurora, beta, release, esr24
Attachment #814347 - Flags: review?(andrei.eftimie)
Attachment #814347 - Flags: review?(andreea.matei)
Comment on attachment 814347 [details] [diff] [review]
skip_esr17.patch

Review of attachment 814347 [details] [diff] [review]:
-----------------------------------------------------------------

Landed as noted above.

I see that the skip in the manifest file for bug 708491 should have been taken out a good while ago.
We should when unskipping as a simple backout would again disable the test.

We should do these things separately to avoid confusion.
Attachment #814347 - Flags: review?(andrei.eftimie)
Attachment #814347 - Flags: review?(andreea.matei)
Attachment #814347 - Flags: review+
Attachment #814347 - Flags: checkin+
Attached patch patch_v1.0.patchSplinter Review
The event-listener that updates this element 
https://developer.mozilla.org/en-US/docs/XPCOM_Interface_Reference/nsIWebProgressListener#onSecurityChange()
it's  called after a security transition and after a "load" event. Also the UI get's updated asynchronous.
Waiting for the UI to change will be more safe than only checking it after a pageload.

  
Reports:
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b757f9
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b7c1ea
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b76032
Attachment #820946 - Flags: review?(andrei.eftimie)
Attachment #820946 - Flags: review?(andreea.matei)
Comment on attachment 820946 [details] [diff] [review]
patch_v1.0.patch

Review of attachment 820946 [details] [diff] [review]:
-----------------------------------------------------------------

Looks like a good fix.
Lets prove that with our daily testruns before enabling on all branches.

http://hg.mozilla.org/qa/mozmill-tests/rev/e99ceebcfe34 (default)
Attachment #820946 - Flags: review?(andrei.eftimie)
Attachment #820946 - Flags: review?(andreea.matei)
Attachment #820946 - Flags: review+
Lets not skip it yet if it only failed 4 times today.

If it hasn't failed since we landed the fix, and we can't reproduce it manually/locally, I'd say to leave it in and monitor failures closely.

Please update the bug if/when this fails again, ATM we don't have a TOP failures for remote tests, so it is harder to see them all in the dashboard.
I have managed to reproduce this locally on a Win8 VM while testing the environment for 1.5.24.1
http://mozmill-crowd.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2784f6a2

Will try to see if its further reproducile
This failed a lot this week so I suggest we backout the last patch as it did't fixed the problem.
Ok lets get this skipped again, fails roughly 10 times per day.
An update here, I had ran this test for 600 times today and I couldn't reproduce it, we might want to re-enable it.
Re-enabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/69a2e8fafb45 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/1832161a6a5e (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/9841ecc9e6da (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/7bf8efb8f799 (esr24)

Hopefully now it works fine with 2.0.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Whiteboard: [mozmill-test-failure][mozmill-test-skipped] → [mozmill-test-failure]
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.