This just started happening recently on Nightly and Aurora though intermittently. It appears to be Mac only. Test: testPasswordManager :: testPasswordNotSaved.js :: testPasswordNotSaved() Fail: Pane has been changed - expected 'panePrivacy'
Assigning to Alex
Created attachment 581694 [details] [diff] [review] disable test: default/aurora [checked-in] I could not figure out the issue. I'm still investigating, but skipping the test for now, as to our policy.
Comment on attachment 581694 [details] [diff] [review] disable test: default/aurora [checked-in] Landed: http://hg.mozilla.org/qa/mozmill-tests/rev/43597e45052b (default) http://hg.mozilla.org/qa/mozmill-tests/rev/dfabcd7f5ecf (mozilla-aurora)
I ran this today on Lion, 100 times for each branch. And out of that it only failed once on Nightly. Can we unskip the test?
(In reply to Alex Lakatos[:AlexLakatos] from comment #4) > I ran this today on Lion, 100 times for each branch. And out of that it only > failed once on Nightly. Can we unskip the test? Were you able to reproduce the failure locally before this test was skipped?
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5) > Were you able to reproduce the failure locally before this test was skipped? It was failing more than once during 200 runs. But not a lot more. I could not reproduce the failure when running the test once, instead of running it in a for loop.
Then I would say the situation has not improved, and since there is no change in Firefox code we can't just re-enable the test. Until we can figure out why it intermittently fails we cannot re-enable this test.
Please check out bug 671382. I'm working to fix the paneId setter right now. Taking all the failures into consideration, I'll double the efforts with Alex's help in finding a robust fix.
Alex, do you have an update for us?
It's passing now, but I'm still running it over and over to determine if we are safe to re-enable it.
Ran the test 100 times on it's own on Win XP, Ubuntu and Mac. It did not fail once. Ran a full functional testrun with the test enabled, three times, and it did not fail. Here are the reports: XP: http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c18cf47 http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c194353 http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c19d31e Ubuntu: http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c1b4cb3 http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c1c2e46 http://mozmill-crowd.blargon7.com/#/functional/report/c67fb1fc2ea8384105b843e32c1ce95a Mac: http://mozmill-crowd.blargon7.com/#/functional/report/44fc451c2e6b66b62172f2e13e05b865 http://mozmill-crowd.blargon7.com/#/functional/report/44fc451c2e6b66b62172f2e13e05a494 http://mozmill-crowd.blargon7.com/#/functional/report/44fc451c2e6b66b62172f2e13e057979
Based on Comment 12, I think we're safe to re-enable it for default and go from there. Henrik, what do you think?
Ok, lets try that. Landed on default as: http://hg.mozilla.org/qa/mozmill-tests/rev/1397c0ccc72a Alex, please check which branches are affected. I assume all of the supported ones. If it sticks green we can transplant to older branches by mid of this week.
(In reply to Henrik Skupin (:whimboo) from comment #14) > Ok, lets try that. Landed on default as: > http://hg.mozilla.org/qa/mozmill-tests/rev/1397c0ccc72a > > Alex, please check which branches are affected. I assume all of the > supported ones. If it sticks green we can transplant to older branches by > mid of this week. Latest run in CI passes. http://mozmill-ci.blargon7.com/#/functional/report/9acea0c923846813ba8ce41994000af5 This needs to land on aurora, beta, release and esr10.
Pushed: http://hg.mozilla.org/qa/mozmill-tests/rev/c55ad3bc7803 (aurora) http://hg.mozilla.org/qa/mozmill-tests/rev/e9af10605968 (beta) http://hg.mozilla.org/qa/mozmill-tests/rev/b570a5a1027c (release) http://hg.mozilla.org/qa/mozmill-tests/rev/799e387e806a (esr10) Please update all the Litmus tests which are disabled.
https://litmus.mozilla.org/show_test.cgi?id=55632 https://litmus.mozilla.org/show_test.cgi?id=63986 https://litmus.mozilla.org/show_test.cgi?id=15639
An update for the litmus testcase of the esr10 branch is still missing.
I'm not seeing an esr branch in litmus. There are several tests for the aurora branch, but I can't see an esr branch.
This is Firefox 10.0!
Edited this a while back but seems I forgot to flip the flag. https://litmus.mozilla.org/show_test.cgi?id=40737