Happens on and off. On multiple tests. Have not found a reliable reproductible scenario yet. It does appear to happen more in resource constrained environments. I am starting to suspect a more abstract / underlying issue here, but am not familiar enough with Mozmill's code yet. Basically a page appears to not ever load (or fire the appropriate event). Observed this on local pages (chrome://..) where the page appeared to be fully loaded, but firefox was still "loading" it (spinning wheel in tab-icon) apparently forever. A recent list of failures: http://mozmill-ci.blargon7.com/#/endurance/report/23d8fbdd0190d4b0496d6b129f5a6c19 http://mozmill-ci.blargon7.com/#/functional/report/c77cfb8878087b850538b93cf20f90ad http://mozmill-crowd.blargon7.com/#/endurance/report/23d8fbdd0190d4b0496d6b129f59c018 http://mozmill-crowd.blargon7.com/#/endurance/report/23d8fbdd0190d4b0496d6b129f59b64b http://mozmill-crowd.blargon7.com/#/endurance/report/23d8fbdd0190d4b0496d6b129f5a9696 http://mozmill-crowd.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f598d24 http://mozmill-crowd.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f595325 Possible duplicates (most of them have been resolved with WORKSFORME): -- there might be other real problems within them, but they might just be the same underlying issue -- bug 665900 bug 758151 bug 760411 bug 767821 bug 767822 bug 779488 bug 794750 == We should gather all reports here, maybe we'll find a common denominator that will help us identify this issue.
(In reply to Andrei Eftimie from comment #0) > Observed this on local pages (chrome://..) where the page appeared to be > fully loaded, but firefox was still "loading" it (spinning wheel in > tab-icon) apparently forever. Do you have an example or can you reproduce this problem?
Happened in /testSecurity/testSafeBrowsingNotificationBar.js on Linux Ubuntu 12.10 (x86) and Aurora on 1/26: http://mozmill-ci.blargon7.com/#/functional/report/72e15dc943833b8fcba70aeb519a1567
Happened again today (2013-02-03) in /testSecurity/testGreenLarry.js on Mac OS X 10.6.8 (x86_64) - Firefox 21.0a1: http://mozmill-ci.blargon7.com/#/functional/report/0dbaa964aec88a2f5637c68c9689a151
Another 3 failed runs while trying to load the following URL: http://www.mozilla.org/en-US/about/contact All on 20.0a2 on WinXP, Vista and 7 in /testToolbar/testStopReloadButtons.js : http://mozmill-ci.blargon7.com/#/functional/report/0dbaa964aec88a2f5637c68c967d8cc3 http://mozmill-ci.blargon7.com/#/functional/report/0dbaa964aec88a2f5637c68c967d9825 http://mozmill-ci.blargon7.com/#/functional/report/0dbaa964aec88a2f5637c68c967d77c4
(In reply to mario garbi from comment #8) > Happened again today (2013-02-03) in /testSecurity/testGreenLarry.js on Mac > OS X 10.6.8 (x86_64) - Firefox 21.0a1: > http://mozmill-ci.blargon7.com/#/functional/report/ > 0dbaa964aec88a2f5637c68c9689a151 The security tests try to load the following URL: https://addons.mozilla.org/en-US/firefox/licenses/5.txt which returns a 404. (Its still a pretty formated 404, and has the green badge, but am wondering if it can have an effect on our tests failing)
No, that shouldn't make a difference here.
This failure most likely is responsible for all of mess we have today with the OS X machines. Please start working on it as a team and figure out why we are failing that much here. Thanks.
Andrei wants to fire off a new thread on the public mailing list for discussion but to quickly wrap up my comments on those issues: * Our tests were failing because AMO didn't work as expected from within the MV network. It seems to be fixed now but we might want to see those issues again. It's a reason why I want to see all those boxes in MPT for better stability. But those failures showed real problems. * The modal dialog failures happen because Firefox pings AMO for compatibility updates when installing an extension or theme. We might want to stop that. Lets further discuss this on bug 809124. * Updating Mozmill for an additional check for DOMContentLoaded as Andrei pointed out may be helpful but we will have to check the impact of this patch. All this code is very complicated and there is a high risk for a regression. Also we most likely wouldn't take it for Mozmill 1.5. * We might want to check functional tests which load pages on AMO and update them for local pages or pages on mozqa.com.
We had another failure on waitForPageLoad on a local page: http://mozmill-ci.blargon7.com/#/functional/report/66aad0ebfa7a01580628b3af4c2563f2
We had another couple of failures with "WaitForPageLoad" : http://mozmill-ci.blargon7.com/#/functional/report/66aad0ebfa7a01580628b3af4cf1a1cd http://mozmill-ci.blargon7.com/#/functional/report/66aad0ebfa7a01580628b3af4cf1afa9 http://mozmill-ci.blargon7.com/#/endurance/report/66aad0ebfa7a01580628b3af4cb32da0
Happened today over testSearch/testSearchSelection.js, with 10.6.8 OS X, Nightly it locale: http://mozmill-ci.blargon7.com/#/functional/report/2a6536e9db9f5f44ed48c58510a280c5
Failed in /testLayout/testNavigateFTP.js, Nightly, Mac OS X 10.7.5, de: http://mozmill-ci.blargon7.com/#/functional/report/677f05f4f4af7748bfaf0197ee3c1ac5
Happened again over over "testSearch/testSearchSelection.js" with Firefox 23.0a1 de on Windows NT 6.2.9200: http://mozmill-ci.blargon7.com/#/functional/report/740ab23e062758ae8a240b263d2d7b95
Happened today on Aurora FR with MAC OS 10.6.8: http://mozmill-ci.blargon7.com/#/functional/report/cb9e33213cf3d2f37d7b0f8dcc0b08b5
Was anyone ever able to reproduce this locally? If yes, a HTTP log would be quite helpful here: https://developer.mozilla.org/en/HTTP_Logging
Happened again today on Mac OS X 10.8.2 (x86_64) with Firefox 23.0a1 fr View the build in Jenkins: http://release1.qa.mtv1.mozilla.com:8080/job/mozilla-central_functional/116/ View the results in the Mozmill Dashboard: http://mozmill-ci.blargon7.com/#/functional/report/ea82256a8ae9808d91b7e8145e2547b6 http://mozmill-ci.blargon7.com/#/functional/report/ea82256a8ae9808d91b7e8145e26519e
Actually this is most likely nothing we can do and should do in our tests. This is basically the fault of Mozmill and its broken waitForPageLoad behavior. I fixed that in bug 760720 but we most likely don't want to spend our time on backports. We will see this failure for sure on and off but we will not fix it. We have to live with it until we make use of Mozmill 2.0. Otilia, please find another entry as replacement.
Happened again on Windows 7 with Nightly, locale de http://mozmill-daily.blargon7.com/#/remote/report/6d6f6a58b02eeffc06eafa8bea3cc659
Happened twice yesterday on Linux 13.04 and OS X 10.6.8, both Nightly de: http://mozmill-daily.blargon7.com/#/remote/report/ec449c026814fd64783d738c02722ca9 http://mozmill-daily.blargon7.com/#/remote/report/ec449c026814fd64783d738c027408b4
testNavigateFTP.js has failed today on Ubuntu 12.04, default, it locale: http://mozmill-daily.blargon7.com/#/remote/report/850f11101678d9fbefeb95e0bd4735a2
Which line failed in detail for this test? It might be that a click was not successful. That's important information I miss via the last comment.
Hm, I don't see the details when hovering over the failure, I get only the scrollbars and some empty canvases. Neither by inspecting the element, the last div seems to be just empty. Same applies to other failure reports, not sure when the dashboard got this issue.
File a dashboard issue and check the raw report for the information?
In the test is at: http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/remote/testLayout/testNavigateFTP.js#l18 In controller.js: https://github.com/mozilla/mozmill/blob/master/mozmill/mozmill/extension/resource/driver/controller.js#L986 stack 0 BaseError@resource://mozmill/modules/errors.js:27 1 AssertionError@resource://mozmill/modules/errors.js:76 2 Assert_waitFor@resource://mozmill/modules/assertions.js:607 3 browserAdditions/controller.waitForPageLoad@resource://mozmill/driver/controller.js:989 4 testNavigateFTP@resource://mozmill/modules/frame.js -> file:///home/mozauto/jenkins/workspace/mozilla-central_remote/data/mozmill-tests/firefox/tests/remote/testLayout/testNavigateFTP.js:18 5 Runner.prototype.execFunction@resource://mozmill/modules/frame.js:752 6 Runner.prototype.runTestModule@resource://mozmill/modules/frame.js:706 7 Runner.prototype.runTestFile@resource://mozmill/modules/frame.js:690 8 runTestFile@resource://mozmill/modules/frame.js:775 9 Bridge.prototype._execFunction@resource://jsbridge/modules/Bridge.jsm:140 10 Bridge.prototype.execFunction@resource://jsbridge/modules/Bridge.jsm:147 11 @resource://jsbridge/modules/Server.jsm:32 12 message controller.waitForPageLoad(): Timeout waiting for page loaded. fileName resource://mozmill/modules/errors.js name AssertionError lineNumber 27
I think we should enhance the waitForPageLoad method in Mozmill so it gives us more information about the load status of the page when the timeout happens. Andreea, can you please file that bug?
Sure, filed bug 957973.
We had about 5 failures today on Windows machines with test /testLayout/testNavigateFTP.js Firefox Beta 28.0 pl - Windows 7 64 (mm-win-7-64-4) http://mozmill-release.blargon7.com/#/remote/report/e248c845ffb052c61b3635551b58b31e Firefox Beta 28.0 ru - Windows Vista 32 (mm-win-vista-32-4) http://mm-ci-master.qa.scl3.mozilla.com:8080/job/release-mozilla-beta_remote/6120/ Firefox Beta 28.0 ru - Windows 8.1 32 (mm-win-81-32-3) http://mozmill-release.blargon7.com/#/remote/report/e248c845ffb052c61b3635551b58d23b Firefox Beta 28.0 pt-BR - Windows 8 64 (mm-win-8-64) http://mozmill-release.blargon7.com/#/remote/report/e248c845ffb052c61b3635551b58e0c7 Firefox Beta 28.0 ru - Windows 7 32 (mm-win-7-32-1) http://mozmill-release.blargon7.com/#/remote/report/e248c845ffb052c61b3635551b58e1d3
Failed on testAddons_InstallAddonWithoutEULA/test1.js on Win 8.1, default de: http://mozmill-daily.blargon7.com/#/remote/report/e248c845ffb052c61b3635551bb1a7d1 Failed on /testLayout/testNavigateFTP.js on Win 8.1, default en-US: http://mozmill-daily.blargon7.com/#/remote/report/e248c845ffb052c61b3635551bb06162 Failed on /testAddons/testSearchAddons.js on Win 8.1, esr24 en-US: http://mozmill-daily.blargon7.com/#/remote/report/e248c845ffb052c61b3635551bbc7f8b Note: These all were on the same machine win-81-64-3 which had some issues - bug 978684.
We had failures today in /testLayout/testNavigateFTP.js -- Windows XP 32, Aurora de. http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb2159c6d -- Windows 7 64, Aurora de. http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb2159327 -- Windows 7 32, Aurora fr. http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb2159826 -- Windows 7 64, Aurora fr. http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb2158568 -- Windows 7 32, Nightly de http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb219cb57 -- Windows 7 64, Nightly fr http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb219bd8f -- Windows 7 64, Nightly de http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb219b9c9 -- Windws 8 64, Nightly de http://mozmill-daily.blargon7.com/#/remote/report/97976110c92e838016d17f7fb219da56
Failure today in testLayout\testNavigateFTP.js on Nightly fr (20140325030201) Windows Vista 32 (mm-win-vista-32-4) message: "controller.waitForPageLoad(URI=ftp://ftp.mozqa.com/data/, readyState=complete)" http://mozmill-daily.blargon7.com/#/remote/report/6881c00549f3ec04f84ba63c97145067
testNavigateFTP failed several times today on windows machines: http://mozmill-daily.blargon7.com/#/remote/failure?app=All&branch=All&platform=All&from=2014-03-28&to=&test=%2FtestLayout%2FtestNavigateFTP.js&func=testNavigateFTP
Interesting when we fail for local pages: > controller.waitForPageLoad(URI=http://www.mozqa.com/data/firefox/video/test_ogv_video_60s.html, readyState=interactive) http://mozmill-daily.blargon7.com/#/endurance/report/0bd22723d1fdbb446269f765b969812b
This failed 4 times today with Beta,I tried to reproduce it locally but I couldn't reproduce it, I ran a complete testrun and the failing test for 50 times, so I don't think this is related to the new added locales but just network issue. http://mozmill-release.blargon7.com/#/remote/failure?app=Firefox&branch=32.0&platform=Win&from=2014-08-05&to=&test=%2FtestSecurity%2FtestSafeBrowsingNotificationBar.js&func=testNotificationBar
Nothing we want to fix for Mozmill. The adequate Firefox UI Tests issue is covered by bug 1206000.