Closed Bug 1318724 Opened 3 years ago Closed 2 years ago

Webdriver tests unstable on Linux64 debug

Categories

(Testing :: web-platform-tests, defect)

defect
Not set

Tracking

(firefox52 disabled, firefox-esr52 disabled, firefox53 disabled, firefox54 disabled, firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox52 --- disabled
firefox-esr52 --- disabled
firefox53 --- disabled
firefox54 --- disabled
firefox55 --- fixed

People

(Reporter: jgraham, Assigned: jgraham)

Details

(Whiteboard: Backed out wpt update for neverending W(7) jobs on Windows 7 VM debug:https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=f98b750bb85085a61341dc267f5ebf313820f960&filter-searchStr=2e76e93115e24a1caffbba81d961e84fb651fa8c)

Attachments

(2 files)

They are flipping from FAIL to ERROR, so disabling the whole file for now.
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f62879c48004
Disable unstable WebDriver navigation test on a CLOSED TREE, a=testonly
https://hg.mozilla.org/mozilla-central/rev/f62879c48004
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
Tests in context.py have also been failing silently (job still green) on Linux64 debug, and actions tests fail intermittently. 

I've been able to reproduce on a one-click loaner, although only intermittently. The browser starts then stops before the tests even start, but I see no evidence of the browser crashing, so I think it's being killed by the harness. The harness reports errors in fixture setup because starting a session fails. It looks like a race in wptrunner with respect to Marionette server start-up.
Summary: Webdriver navigation tests unstable on Linux64 debug → Webdriver tests unstable on Linux64 debug
As James and I noticed this morning it's mostly because of a timeout set for the HTTP server. Increasing it from 5s to somewhat >10s make it working for me.

https://dxr.mozilla.org/mozilla-central/source/testing/web-platform/tests/tools/webdriver/webdriver/transport.py#9

James, you said that you want to fix it. Can you please add a reference to the bug? Thanks.
Flags: needinfo?(james)
Thanks James! When looking at the order of the commits I think you will have to flip the commits, so that it doesn't produce a broken commit with test failures.
Comment on attachment 8850646 [details]
Bug 1318724 - Increase the HTTP connection timeout for WebDriver tests,

https://reviewboard.mozilla.org/r/123196/#review125844
Attachment #8850646 - Flags: review?(ato) → review+
The original patch on this bug was pushed a couple of months ago, and the bug was marked as fixed. 

James, for your follow-up you should really have chosen a new bug, or was it just a typo in the commit message? It would be great to finally get this patch pushed.
Flags: needinfo?(james)
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/fdbeb5e7c238
Reenable WebDriver spec tests, r=maja_zf
https://hg.mozilla.org/integration/autoland/rev/5827fdd9ac6d
Increase the HTTP connection timeout for WebDriver tests, r=ato
Oh, the bug was marked as fixed erroneously. The formerly landed patch simple disabled some tests.
Status: RESOLVED → REOPENED
Flags: needinfo?(james)
Resolution: FIXED → ---
Assignee: nobody → james
Status: REOPENED → ASSIGNED
Target Milestone: mozilla53 → ---
https://hg.mozilla.org/mozilla-central/rev/fdbeb5e7c238
https://hg.mozilla.org/mozilla-central/rev/5827fdd9ac6d
Status: ASSIGNED → RESOLVED
Closed: 3 years ago2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
James, can you care about the necessary uplift if that is doable for wpt-tests? Thanks.
Flags: needinfo?(james)
I'm not aware that uplift here is necessary.
Flags: needinfo?(james)
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e26fcd9ef4ae
Increase the HTTP connection timeout for WebDriver tests, r=ato
https://hg.mozilla.org/mozilla-central/rev/bca7ff5030f8
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: Backed out wpt update for neverending W(7) jobs on Windows 7 VM debug:https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=f98b750bb85085a61341dc267f5ebf313820f960&filter-searchStr=2e76e93115e24a1caffbba81d961e84fb651fa8c
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ab00229faf2c
Increase the HTTP connection timeout for WebDriver tests, r=ato
https://hg.mozilla.org/mozilla-central/rev/ab00229faf2c
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Flags: needinfo?(james)
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6432a9f65912
Increase the HTTP connection timeout for WebDriver tests, r=ato
You need to log in before you can comment on or make changes to this bug.