Closed Bug 1214191 Opened 4 years ago Closed Last month

TEST-UNEXPECTED-ERROR | testing/firefox-ui/tests/puppeteer/test_windows.py TestBaseWindow.test_open_close | TimeoutException: Timed out after 5.1 seconds with message: No focused window has been found.

Categories

(Testing :: Firefox UI Tests, defect, P2)

43 Branch
x86_64
Windows 10
defect

Tracking

(firefox42 wontfix, firefox43 wontfix, firefox44 wontfix, firefox45 wontfix, firefox-esr60 disabled, firefox-esr68 disabled, firefox60 disabled, firefox68 disabled, firefox69 disabled, firefox70 fixed)

RESOLVED FIXED
mozilla70
Tracking Status
firefox42 --- wontfix
firefox43 --- wontfix
firefox44 --- wontfix
firefox45 --- wontfix
firefox-esr60 --- disabled
firefox-esr68 --- disabled
firefox60 --- disabled
firefox68 --- disabled
firefox69 --- disabled
firefox70 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled])

Attachments

(2 files, 1 obsolete file)

Traceback (most recent call last):
  File "/home/mozauto/jenkins/workspace/mozilla-aurora_functional/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/home/mozauto/jenkins/workspace/mozilla-aurora_functional/venv/lib/python2.7/site-packages/firefox_puppeteer/tests/test_windows.py", line 114, in test_open_close
    Wait(self.marionette).until(lambda _: win1.focused)  # catch the no focused window
  File "/home/mozauto/jenkins/workspace/mozilla-aurora_functional/venv/local/lib/python2.7/site-packages/marionette_driver/wait.py", line 143, in until
    cause=last_exc)

TimeoutException: TimeoutException: Timed out after 5.1 seconds
This largely happened today with the 42.0rc build 2 on OS X and Windows.
The failures today were present on mm-osx-1011-2 which had the system preferences window open. Maybe that had an influence to it. On Windows mm-win7-32-3 was affected. It has a screen resolution of 1024x768 only. Given that we want to bump it for Linux machines to 1280x960, we might also want to do it for Windows. I temporarily increased it, so lets see if we can reproduce the problem over the weekend or not.
OS: Unspecified → All
Hardware: Unspecified → All
Looks like that on mm-osx-1011-2 those tests are constantly failing. Maybe I should check if I can reproduce on that machine.
Product: Mozilla QA → Testing
No more failure since a couple of months.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME
Happened again on mozilla-central over the weekend:
https://treeherder.mozilla.org/logviewer.html#?job_id=4701006&repo=mozilla-central
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: TEST-UNEXPECTED-ERROR | test_windows.py TestBaseWindow.test_open_close | TimeoutException: TimeoutException: Timed out after 5.1 seconds → TEST-UNEXPECTED-ERROR | test_windows.py TestBaseWindow.test_open_close | TimeoutException: Timed out after 5.1 seconds with message: No focused window has been found.
There have been 31 failures in the last week.
Since February 14th it seems that the number of failures has increased. 

Failures per platform:
- windows10-64: 27
- windows10-64-ccov: 3
- windows10-64-nightly: 1

Failures per build type:
- debug: 15
- opt: 13
- pgo: 3

Here is a recent relevant log file and a snippet with the error:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=162936012&lineNumber=32177

23:29:44     INFO -  1518996584390	Marionette	TRACE	32 <- [1,365,null,{}]
23:29:44     INFO -  1518996584390	Marionette	TRACE	32 -> [0,366,"getPageSource",{}]
23:29:44     INFO -  1518996584396	Marionette	TRACE	32 <- [1,366,null,{"value":"<html xmlns=\"http://www.w3.org/1999/xhtml\" hasBrowserHandlers=\"true\">\n<head>\n  <title>About About ... ssifier\">about:url-classifier</a></li><li><a href=\"about:webrtc\">about:webrtc</a></li></ul>\n  </div>\n</body>\n</html>"}]
23:29:44     INFO - TEST-UNEXPECTED-ERROR | testing\firefox-ui\tests\puppeteer\test_windows.py TestBaseWindow.test_open_close | TimeoutException: Timed out after 5.1 seconds with message: No focused window has been found.
23:29:44     INFO - Traceback (most recent call last):
23:29:44     INFO -   File "Z:\task_1518996375\build\venv\lib\site-packages\marionette_harness\marionette_test\testcases.py", line 159, in run
23:29:44     INFO -     testMethod()
Flags: needinfo?(hskupin)
Whiteboard: [stockwell needswork]
There was a spike by last week but then the failures dropped again. Lets see what happens in the next days.
Flags: needinfo?(hskupin)
So nowadays the failures are all Windows 10 64bit, both opt and debug. The number of failures increased around beginning of February:

https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1214191&startday=2018-01-01&endday=2018-02-25&tree=all
OS: All → Windows 10
Hardware: All → x86_64
Attached patch Disable on Win10 all builds (obsolete) — Splinter Review
Hi! The patch is for disabling on win10-64 all builds. The manifest I edited was this one https://searchfox.org/mozilla-central/source/testing/firefox-ui/tests/puppeteer/manifest.ini 
Please have a look and check if I did it correctly. Thanks.
Attachment #8956202 - Flags: review?(jmaher)
Comment on attachment 8956202 [details] [diff] [review]
Disable on Win10 all builds

Review of attachment 8956202 [details] [diff] [review]:
-----------------------------------------------------------------

this is not a web-platform-test, so you will need:
skip-if = os == "win" && bits == "64" # Bug 1214191
Attachment #8956202 - Flags: review?(jmaher) → review-
Modified it.
Attachment #8956202 - Attachment is obsolete: true
Attachment #8956207 - Flags: review?(jmaher)
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/deaf7280ea37
Disable test_windows.py TestBaseWindow.test_open_close on Windows 10 for frequent failures. r=jmaher
Keywords: leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Rob, what is the process those days to get a Windows 10 64bit one click loaner? I would need one to investigate this problem. Thanks.
Flags: needinfo?(rthijssen)
Priority: -- → P2
still no one click loaners but the interim solution still works:
https://wiki.mozilla.org/ReleaseEngineering/How_To/Self_Provision_a_TaskCluster_Windows_Instance

the gpg key url issue where we couldn't handle non raw gpg urls has been fixed.

the one click effort is in bug 1368961.
Flags: needinfo?(rthijssen)

This is basically blocked on bug 1335085.

Depends on: 1335085
Summary: TEST-UNEXPECTED-ERROR | test_windows.py TestBaseWindow.test_open_close | TimeoutException: Timed out after 5.1 seconds with message: No focused window has been found. → TEST-UNEXPECTED-ERROR | testing/firefox-ui/tests/puppeteer/test_windows.py TestBaseWindow.test_open_close | TimeoutException: Timed out after 5.1 seconds with message: No focused window has been found.
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/648b37f833c1
[fxui] Re-enable testing/firefox-ui/tests/puppeteer/test_windows.py. r=webdriver-reviewers,ato
Status: REOPENED → RESOLVED
Closed: 4 years agoLast month
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
Assignee: csabou → hskupin
Keywords: test-disabled
You need to log in before you can comment on or make changes to this bug.