1.94 KB, patch
|Details | Diff | Splinter Review|
2.17 KB, patch
Andrei Eftimie: review+
Andrei Eftimie: checkin+
|Details | Diff | Splinter Review|
3.06 KB, patch
|Details | Diff | Splinter Review|
This started happening recently on all platforms with different localizations of Beta 25 as well as en-US. I tried to reproduce it locally but was not able. Failure reports: http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750770d89 http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a75098c5b2 http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a7509d388c http://mozmill-release.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a7509e3ae2
I also tried to reproduce it on 'mm-win-8-32-2' running both single tests using mozmill -t and testrun_remote.py script. I was not able to reproduce it as can be seen in the reports: http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750bde5bb http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be0151 http://mozmill-crowd.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750be285e
What fails in detail given by the stack information from the tests?
The value for "identityBox.getNode().className" did not changed after loading the "unsecure HTTP site" at line 35 and so remained "verifiedIdentity". This caused a failure at line 38 that can be seen in the link below: http://hg.mozilla.org/qa/mozmill-tests/file/541b0cb3957ff33b7b7991e0ccde4b6e80e77af7/tests/remote/testSecurity/testSecurityNotification.js#l38 I suspect this was caused by a 404 when loading the page at line 35 because the waitForPageLoad() was not triggered. The fact that this is not reproducible now indicates a possible network issue that got fixed in the meanwhile. I will look over the failing machine as soon as it reproduces again to better understand what happens.
With the upgrade to Mozmill 2.0 we should not fetch those broken page loads. Lets see if it happens again and we get more information.
Happened again today on Linux Ubuntu 13.04 (x86) with Firefox 26.0a2: http://mozmill-daily.blargon7.com/#/remote/report/6ec6776efe900da3fd2b64a750eb2aac
Happend mac, with Esr24 to http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea109c6f
We should skip this if it fails that much on Beta especially.
Created attachment 814340 [details] [diff] [review] skip.patch Here is the skip patch for beta. I tried to reproduce this failure on affected machines yesterday and today again, without success. http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a696d http://mozmill-crowd.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4a5bf6 Most likely it fails because waitforpagetoload passes before the page loads at line http://hg.mozilla.org/qa/mozmill-tests/file/b3f578c0111d/tests/remote/testSecurity/testSecurityNotification.js#l36 As Henrik said on comment 4 we shouldn't see those failures again once we have mozmill-2.0 on ci
Comment on attachment 814340 [details] [diff] [review] skip.patch Review of attachment 814340 [details] [diff] [review]: ----------------------------------------------------------------- Disabled: http://hg.mozilla.org/qa/mozmill-tests/rev/b5a65140e3dc (beta) We still want to disable on all affected branches as I saw a failure on esr24 as well today. Also, please try to reproduce on the remote machines or watch a few runs there. Today reproduced quite often, it might be an issue with the page, but we cannot know for sure until we catch it.
Created attachment 814347 [details] [diff] [review] skip_esr17.patch Here is the skip patch for esr17, the previous patch applies cleanly on aurora, beta, release, esr24
Nightly is affected too http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea329953 Esr 17 is affected too http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea27b11a Esr 24 is affected too http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea4f7515
Ok, lets skip this until we have mozmill 2.0 running in CI: Disabled: http://hg.mozilla.org/qa/mozmill-tests/rev/15c8233978b8 (default) http://hg.mozilla.org/qa/mozmill-tests/rev/3a5f2c543803 (mozilla-aurora) http://hg.mozilla.org/qa/mozmill-tests/rev/2da3a2a39254 (mozilla-release) http://hg.mozilla.org/qa/mozmill-tests/rev/e8aa8bc990cb (mozilla-esr24) http://hg.mozilla.org/qa/mozmill-tests/rev/9899ffda642d (mozilla-esr17)
Comment on attachment 814347 [details] [diff] [review] skip_esr17.patch Review of attachment 814347 [details] [diff] [review]: ----------------------------------------------------------------- Landed as noted above. I see that the skip in the manifest file for bug 708491 should have been taken out a good while ago. We should when unskipping as a simple backout would again disable the test. We should do these things separately to avoid confusion.
Created attachment 820946 [details] [diff] [review] patch_v1.0.patch The event-listener that updates this element https://developer.mozilla.org/en-US/docs/XPCOM_Interface_Reference/nsIWebProgressListener#onSecurityChange() it's called after a security transition and after a "load" event. Also the UI get's updated asynchronous. Waiting for the UI to change will be more safe than only checking it after a pageload. Reports: http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b757f9 http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b7c1ea http://mozmill-crowd.blargon7.com/#/remote/report/bb6e314abd8f1e30e8af61ecd4b76032
Comment on attachment 820946 [details] [diff] [review] patch_v1.0.patch Review of attachment 820946 [details] [diff] [review]: ----------------------------------------------------------------- Looks like a good fix. Lets prove that with our daily testruns before enabling on all branches. http://hg.mozilla.org/qa/mozmill-tests/rev/e99ceebcfe34 (default)
This is still failing, it failed today 4 times on Aurora, I could't reproduce it. I would say to disable this on Nightly again, until we get a better understanding of why is failing. http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa276fa987 http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2772b3ad http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27727f2b http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2772df08
Lets not skip it yet if it only failed 4 times today. If it hasn't failed since we landed the fix, and we can't reproduce it manually/locally, I'd say to leave it in and monitor failures closely. Please update the bug if/when this fails again, ATM we don't have a TOP failures for remote tests, so it is harder to see them all in the dashboard.
I have managed to reproduce this locally on a Win8 VM while testing the environment for 220.127.116.11 http://mozmill-crowd.blargon7.com/#/remote/report/d17690a112360a2b3155acaa2784f6a2 Will try to see if its further reproducile
This failed a lot this week so I suggest we backout the last patch as it did't fixed the problem.
Ok lets get this skipped again, fails roughly 10 times per day.
Backed out: http://hg.mozilla.org/qa/mozmill-tests/rev/17b2cd9bd872 (default) http://hg.mozilla.org/qa/mozmill-tests/rev/8afbeba6b6f4 (mozilla-aurora)
An update here, I had ran this test for 600 times today and I couldn't reproduce it, we might want to re-enable it.
Reenabled on default, if all goes well will do the rest tomorrow: http://hg.mozilla.org/qa/mozmill-tests/rev/8f1bd29492eb
Re-enabled: http://hg.mozilla.org/qa/mozmill-tests/rev/69a2e8fafb45 (aurora) http://hg.mozilla.org/qa/mozmill-tests/rev/1832161a6a5e (beta) http://hg.mozilla.org/qa/mozmill-tests/rev/9841ecc9e6da (release) http://hg.mozilla.org/qa/mozmill-tests/rev/7bf8efb8f799 (esr24) Hopefully now it works fine with 2.0.