Created attachment 573936 [details] Dialog on Mac MODULE: /remote/restartTests/testDiscoveryPane_installAddonWithEULA/test1.js I've been noticing a lot recently that the above test is hanging Firefox on the Add-on Install dialog. This reproduces reliably when testing the recent 9.0b1 and Dave ran into it when investigating endurance testruns on Nightly. It seems to only occur on Linux and Mac. The general behaviour is the Firefox hangs on the install dialog. When you Force Quit, Mozmill resumes the testrun but fails to report it to brasstacks and does not spit out an error in console. My recommendation is that we disable this test ASAP as it is causing huge problems in the reliability of our test results.
Were software update dialogs open from OSX or Ubuntu itself? If yes, were those in the foreground? Whenever another window gets the focus the countdown timer will stop and the install button will not be enabled. Not sure why Mozrunner doesn't force a kill of Firefox in such a case after the jsbridge timeout happend.
The dialogs appear to be in the foreground. However, I have not caught it at time of incident. I typically catch it after it has been hung for a few minutes.
I would propose we raise an MozQA/infrastructure bug so Al can deactivate those update dialogs from appearing. IMO they are highly involved in the broken behavior we have here. I still would see it fixed in Mozmill 1.5.x but there is likely no chance to, and we have to wait for our upgrade to Mozmill 2.x.
Henrik, those "Software Installation" dialogs are not software updates. They are the add-on installation dialogs. I don't think the OS Software Update dialogs are getting in the way in the case of this failure. Regardless, this test needs to be disabled for further investigation as is blocks us from being able to even run our testruns for releases.
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #4) > Henrik, those "Software Installation" dialogs are not software updates. They > are the add-on installation dialogs. I don't think the OS Software Update > dialogs are getting in the way in the case of this failure. > > Regardless, this test needs to be disabled for further investigation as is > blocks us from being able to even run our testruns for releases. Based on your screenshot i don't think this test is the problem The installEula test is using the 'Echofon' add-on from amo production to test on Furthermore, the EULA test is ran only on release branch, and it is being skipped on the other branches due to add-on compatibility issues. as we can see clearly from the test code http://hg.mozilla.org/qa/mozmill-tests/rev/c32846cb4d40 Is the screenshot the right one? Or maybe another test is causing the issue. Note: in the screenshot the add-on is empty.xpi, from local data folder
It is possible that another test is causing this hang, but it definitely one of the recently added Addons tests. I only assumed it was EULA because it was the last test I saw in terminal upon failure.
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #6) > It is possible that another test is causing this hang, but it definitely one > of the recently added Addons tests. I only assumed it was EULA because it > was the last test I saw in terminal upon failure. No problem - we'll just have to see which test, but having the empty.xpi extension install dialog there it should not be a problem.
The only test using empty.xpi is restartTests/testAddons_installMultipleExtensions, so changing the Summary to show that. I'll skip that test for now
Created attachment 574314 [details] [diff] [review] skip patch v1.0 [checked-in] Skipping the test due to upcoming release testing. Will investigate further tomorrow
Comment on attachment 574314 [details] [diff] [review] skip patch v1.0 [checked-in] ># HG changeset patch ># User Alex Lakatos <alex.lakatos@softvision.ro> ># Date 1321288500 -7200 ># Node ID 4ba612d968c92a0e036daea438ca6e29afb399ba ># Parent 0cc6f214904cf95cbf7856f89d6d22856af00db6 >imported patch skipMultiple One nit, please use a more detailed commit message. I will change this on check-in but please use something more descriptive in the future. "Bug 701908 - disabled testAddons_installMultipleExtensions due to hang"
Comment on attachment 574314 [details] [diff] [review] skip patch v1.0 [checked-in] Landed: http://hg.mozilla.org/qa/mozmill-tests/rev/291b480d1cba (default) http://hg.mozilla.org/qa/mozmill-tests/rev/d728657e8fca (mozilla-aurora) http://hg.mozilla.org/qa/mozmill-tests/rev/53f2950b2f8a (mozilla-beta) http://hg.mozilla.org/qa/mozmill-tests/rev/d17ed98def90 (mozilla-release)
Alex, any update here? I kinda would like to see this test re-enabled.
I have investigated this bug today by running testruns with clean repo (where test installMultipleExtensions is skipped) and also with the test unskipped. What I have noticed is that the following tests are feezing for 1 minute, either single runned or in testruns, with installMultipleExtensions skipped or unskipped: * testAddons_enableDisableExtension - test3.js and test4.js * testAddons_uninstallExtension - test3.js and test4.js * testAddons_changeTheme - test2.js and test3.js What these tests have in common is the use of restartLink, all other restart tests are restarting just by switching to the next test. Despite this fact, all tests are passing. installMultipleExtensions skipped: * http://mozmill-crowd.blargon7.com/#/functional/report/d87d47fd1034f072b9bece6ee609168f installMultipleExtensions unkipped: * http://mozmill-crowd.blargon7.com/#/functional/report/d87d47fd1034f072b9bece6ee60a35b0
(In reply to Andreea Matei [:AndreeaMatei] from comment #13) > What I have noticed is that the following tests are feezing for 1 minute, > either single runned or in testruns, with installMultipleExtensions skipped > or unskipped: > * testAddons_enableDisableExtension - test3.js and test4.js > * testAddons_uninstallExtension - test3.js and test4.js > * testAddons_changeTheme - test2.js and test3.js We should file this as a separate bug for investigation. Sounds like it is major enough to fix through we do not see any failure. I haven't noticed that myself yet, but it's great if you can constantly reproduce. So any help would be appreciated. > http://mozmill-crowd.blargon7.com/#/functional/report/ > d87d47fd1034f072b9bece6ee609168f > http://mozmill-crowd.blargon7.com/#/functional/report/ > d87d47fd1034f072b9bece6ee60a35b0 You missed to test on Mac. That's certainly helpful to do too.
Mac results: installMultipleExtensions skipped: * http://mozmill-crowd.blargon7.com/#/functional/report/d87d47fd1034f072b9bece6ee62e64b6 installMultipleExtensions unskipped: * http://mozmill-crowd.blargon7.com/#/functional/report/d87d47fd1034f072b9bece6ee62e8887 The freezing doesn't happen on Mac. In Ubuntu, after each test freezed I get: Timeout: bridge.describe("3d659dd2-e5e2-11e1-9eec-0017315c6909", bridge.registry["{95f2b261-e7bf-4efe-9b0e-1c7e70a5a2a5}"]) I will file a separate bug.
Ok, so lets try to re-enable the test now. Please come up with a patch which also removes the disabled key from the manifest.
Created attachment 651669 [details] [diff] [review] enable patch v1 Enabling the test and edit manifest.ini file.
http://hg.mozilla.org/qa/mozmill-tests/rev/4eca052da816
Created attachment 661716 [details] [diff] [review] [ESR10] enable patch v1 Patch that enables the test on esr10.
This failure has only been fixed in default before which is now also available in aurora. It still misses to be checked into beta, release, and esr10. So before I want to go ahead with the esr10 patch please check beta and release first.
Comment on attachment 661716 [details] [diff] [review] [ESR10] enable patch v1 Please ask for review again once beta and release are done.
Created attachment 662461 [details] [diff] [review] [beta, release] enable patch v1 Enables test patch for beta and release branches.
Comment on attachment 661716 [details] [diff] [review] [ESR10] enable patch v1 Enables test patch for ESR10.
http://hg.mozilla.org/qa/mozmill-tests/rev/f23143437512 (beta) http://hg.mozilla.org/qa/mozmill-tests/rev/38d66f18912b (release) http://hg.mozilla.org/qa/mozmill-tests/rev/847463ebfcc6 (esr10) Andreea, can you please make sure to update the Litmus test for Firefox 10?
Updated tests in Litmus: * https://litmus.mozilla.org/manage_testcases.cgi?testcase_id=42075 * https://litmus.mozilla.org/manage_testcases.cgi?testcase_id=42611
or
