If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Test failure "Identity is unknown - 'verifiedIdentity' should equal 'unknownIdentity'" in /testSecurity/testSecurityNotification.js

RESOLVED FIXED

Status

Mozilla QA
Mozmill Tests
P2
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: mario garbi, Assigned: cosmin)

Tracking

unspecified

Firefox Tracking Flags

(firefox27 fixed, firefox28 fixed, firefox29 fixed, firefox30 fixed, firefox-esr17 wontfix, firefox-esr24 fixed)

Details

(Whiteboard: [mozmill-test-failure], URL)

Attachments

(3 attachments)

(Reporter)

Description

4 years ago
This started happening recently on all platforms with different localizations of Beta 25 as well as en-US.

I tried to reproduce it locally but was not able.

Failure reports:
http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750770d89
http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a75098c5b2
http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a7509d388c
http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a7509e3ae2
(Reporter)

Updated

4 years ago
status-firefox24: --- → ?
status-firefox25: --- → affected
status-firefox26: --- → ?
status-firefox27: --- → ?
status-firefox-esr17: --- → ?
status-firefox-esr24: --- → ?
(Reporter)

Comment 1

4 years ago
I also tried to reproduce it on 'mm-win-8-32-2' running both single tests using mozmill -t and testrun_remote.py script. I was not able to reproduce it as can be seen in the reports:

http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750bde5bb
http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be0151
http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be285e
What fails in detail given by the stack information from the tests?
(Reporter)

Comment 3

4 years ago
The value for "identityBox.getNode().className" did not changed after loading the "unsecure HTTP site" at line 35 and so remained "verifiedIdentity". This caused a failure at line 38 that can be seen in the link below:
 
http://hg.mozilla.org/qa/mozmill-tests/file/541b0cb3957ff33b7b7991e0ccde4b6e80e77af7/tests/remote/testSecurity/testSecurityNotification.js#l38

I suspect this was caused by a 404 when loading the page at line 35 because the waitForPageLoad() was not triggered. The fact that this is not reproducible now indicates a possible network issue that got fixed in the meanwhile. I will look over the failing machine as soon as it reproduces again to better understand what happens.
With the upgrade to Mozmill 2.0 we should not fetch those broken page loads. Lets see if it happens again and we get more information.
(Reporter)

Comment 5

4 years ago
Happened again today on Linux Ubuntu 13.04 (x86) with Firefox 26.0a2:
http://mozmill-daily.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750eb2aac
status-firefox26: ? → affected
(Assignee)

Comment 6

4 years ago
Happend mac, with Esr24 to
http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea109c6f
status-firefox-esr24: ? → affected
We should skip this if it fails that much on Beta especially.
Assignee: nobody → cosmin.malutan
Status: NEW → ASSIGNED
(Assignee)

Comment 8

4 years ago
Created attachment 814340 [details] [diff] [review]
skip.patch

Here is the skip patch for beta.

I tried to reproduce this failure on affected machines yesterday and today again, without success.
http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a696d
http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a5bf6

Most likely it fails because waitforpagetoload passes before the page loads at line 
http://hg.mozilla.org/qa/mozmill-tests/file/b3f578c0111d/tests/remote/testSecurity/testSecurityNotification.js#l36
As Henrik said on comment 4 we shouldn't see those failures again once we have mozmill-2.0 on ci
Attachment #814340 - Flags: review?(andrei.eftimie)
Attachment #814340 - Flags: review?(andreea.matei)
Comment on attachment 814340 [details] [diff] [review]
skip.patch

Review of attachment 814340 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled: 
http://hg.mozilla.org/qa/mozmill-tests/rev/b5a65140e3dc (beta)

We still want to disable on all affected branches as I saw a failure on esr24 as well today.

Also, please try to reproduce on the remote machines or watch a few runs there. Today reproduced quite often, it might be an issue with the page, but we cannot know for sure until we catch it.
Attachment #814340 - Flags: review?(andrei.eftimie)
Attachment #814340 - Flags: review?(andreea.matei)
Attachment #814340 - Flags: review+
status-firefox25: affected → disabled
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure][mozmill-test-skipped]
(Assignee)

Comment 10

4 years ago
Created attachment 814347 [details] [diff] [review]
skip_esr17.patch

Here is the skip patch for esr17, the previous patch applies cleanly on  aurora, beta, release, esr24
Attachment #814347 - Flags: review?(andrei.eftimie)
Attachment #814347 - Flags: review?(andreea.matei)
(Assignee)

Comment 11

4 years ago
Nightly is affected too 
http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea329953
Esr 17 is affected too
http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea27b11a
Esr 24 is affected too 
http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4f7515
status-firefox27: ? → affected
status-firefox-esr17: ? → affected

Comment 12

4 years ago
Ok, lets skip this until we have mozmill 2.0 running in CI:

Disabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/15c8233978b8 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/3a5f2c543803 (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/2da3a2a39254 (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/e8aa8bc990cb (mozilla-esr24)
http://hg.mozilla.org/qa/mozmill-tests/rev/9899ffda642d (mozilla-esr17)
status-firefox24: ? → disabled
status-firefox26: affected → disabled
status-firefox27: affected → disabled
status-firefox-esr17: affected → disabled
status-firefox-esr24: affected → disabled

Comment 13

4 years ago
Comment on attachment 814347 [details] [diff] [review]
skip_esr17.patch

Review of attachment 814347 [details] [diff] [review]:
-----------------------------------------------------------------

Landed as noted above.

I see that the skip in the manifest file for bug 708491 should have been taken out a good while ago.
We should when unskipping as a simple backout would again disable the test.

We should do these things separately to avoid confusion.
Attachment #814347 - Flags: review?(andrei.eftimie)
Attachment #814347 - Flags: review?(andreea.matei)
Attachment #814347 - Flags: review+
Attachment #814347 - Flags: checkin+
(Assignee)

Comment 14

4 years ago
Created attachment 820946 [details] [diff] [review]
patch_v1.0.patch

The event-listener that updates this element 
https://developer.mozilla.org/en-US/docs/XPCOM_Interface_Reference/nsIWebProgressListener#onSecurityChange()
it's  called after a security transition and after a "load" event. Also the UI get's updated asynchronous.
Waiting for the UI to change will be more safe than only checking it after a pageload.

  
Reports:
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b757f9
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b7c1ea
http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b76032
Attachment #820946 - Flags: review?(andrei.eftimie)
Attachment #820946 - Flags: review?(andreea.matei)
Comment on attachment 820946 [details] [diff] [review]
patch_v1.0.patch

Review of attachment 820946 [details] [diff] [review]:
-----------------------------------------------------------------

Looks like a good fix.
Lets prove that with our daily testruns before enabling on all branches.

http://hg.mozilla.org/qa/mozmill-tests/rev/e99ceebcfe34 (default)
Attachment #820946 - Flags: review?(andrei.eftimie)
Attachment #820946 - Flags: review?(andreea.matei)
Attachment #820946 - Flags: review+
status-firefox24: disabled → ---
status-firefox28: --- → fixed

Updated

4 years ago
status-firefox27: disabled → fixed
(Assignee)

Comment 16

4 years ago
This is still failing, it failed today 4 times on Aurora, I could't reproduce it. I would say to disable this on Nightly again, until we get a better understanding of why is failing. 

http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa276fa987
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2772b3ad
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27727f2b
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2772df08
status-firefox27: fixed → affected
status-firefox28: fixed → affected

Comment 17

4 years ago
Lets not skip it yet if it only failed 4 times today.

If it hasn't failed since we landed the fix, and we can't reproduce it manually/locally, I'd say to leave it in and monitor failures closely.

Please update the bug if/when this fails again, ATM we don't have a TOP failures for remote tests, so it is harder to see them all in the dashboard.

Comment 18

4 years ago
I have managed to reproduce this locally on a Win8 VM while testing the environment for 1.5.24.1
http://mozmill-crowd.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2784f6a2

Will try to see if its further reproducile
(Assignee)

Comment 19

4 years ago
This failed a lot this week so I suggest we backout the last patch as it did't fixed the problem.

Comment 20

4 years ago
Ok lets get this skipped again, fails roughly 10 times per day.

Comment 21

4 years ago
Backed out:

http://hg.mozilla.org/qa/mozmill-tests/rev/17b2cd9bd872 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/8afbeba6b6f4 (mozilla-aurora)
status-firefox27: affected → disabled
status-firefox28: affected → disabled
(Assignee)

Comment 22

4 years ago
An update here, I had ran this test for 600 times today and I couldn't reproduce it, we might want to re-enable it.
Reenabled on default, if all goes well will do the rest tomorrow:
http://hg.mozilla.org/qa/mozmill-tests/rev/8f1bd29492eb
status-firefox25: disabled → ---
status-firefox26: disabled → ---
status-firefox29: --- → disabled
status-firefox30: --- → unaffected
Re-enabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/69a2e8fafb45 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/1832161a6a5e (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/9841ecc9e6da (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/7bf8efb8f799 (esr24)

Hopefully now it works fine with 2.0.
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
status-firefox27: disabled → fixed
status-firefox28: disabled → fixed
status-firefox29: disabled → fixed
status-firefox30: unaffected → fixed
status-firefox-esr17: disabled → wontfix
status-firefox-esr24: disabled → fixed
Resolution: --- → FIXED
Whiteboard: [mozmill-test-failure][mozmill-test-skipped] → [mozmill-test-failure]
You need to log in before you can comment on or make changes to this bug.