Webdriver tests unstable on Linux64 debug

RESOLVED FIXED in Firefox 55

Status

Testing
web-platform-tests
RESOLVED FIXED
a year ago
6 months ago

People

(Reporter: jgraham, Assigned: jgraham)

Tracking

unspecified
mozilla55
Points:
---

Firefox Tracking Flags

(firefox52 disabled, firefox-esr52 disabled, firefox53 disabled, firefox54 disabled, firefox55 fixed)

Details

(Whiteboard: Backed out wpt update for neverending W(7) jobs on Windows 7 VM debug:https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=f98b750bb85085a61341dc267f5ebf313820f960&filter-searchStr=2e76e93115e24a1caffbba81d961e84fb651fa8c)

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments)

(Assignee)

Description

a year ago
They are flipping from FAIL to ERROR, so disabling the whole file for now.

Comment 1

a year ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f62879c48004
Disable unstable WebDriver navigation test on a CLOSED TREE, a=testonly

Comment 2

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/f62879c48004
Status: NEW → RESOLVED
Last Resolved: a year ago
status-firefox53: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla53

Comment 3

10 months ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/902078bd1d15
status-firefox52: --- → fixed
Tests in context.py have also been failing silently (job still green) on Linux64 debug, and actions tests fail intermittently. 

I've been able to reproduce on a one-click loaner, although only intermittently. The browser starts then stops before the tests even start, but I see no evidence of the browser crashing, so I think it's being killed by the harness. The harness reports errors in fixture setup because starting a session fails. It looks like a race in wptrunner with respect to Marionette server start-up.
Summary: Webdriver navigation tests unstable on Linux64 debug → Webdriver tests unstable on Linux64 debug
As James and I noticed this morning it's mostly because of a timeout set for the HTTP server. Increasing it from 5s to somewhat >10s make it working for me.

https://dxr.mozilla.org/mozilla-central/source/testing/web-platform/tests/tools/webdriver/webdriver/transport.py#9

James, you said that you want to fix it. Can you please add a reference to the bug? Thanks.
Flags: needinfo?(james)
(Assignee)

Comment 6

8 months ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=103064f4ad303cc33b343a08ce5f93d4b2298ca3
Flags: needinfo?(james)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Thanks James! When looking at the order of the commits I think you will have to flip the commits, so that it doesn't produce a broken commit with test failures.

Comment 10

8 months ago
mozreview-review
Comment on attachment 8850646 [details]
Bug 1318724 - Increase the HTTP connection timeout for WebDriver tests,

https://reviewboard.mozilla.org/r/123196/#review125844
Attachment #8850646 - Flags: review?(ato) → review+

Comment 11

8 months ago
mozreview-review
Comment on attachment 8850645 [details]
Bug 1318724 - Reenable WebDriver spec tests,

https://reviewboard.mozilla.org/r/123194/#review125978
Attachment #8850645 - Flags: review?(mjzffr) → review+
The original patch on this bug was pushed a couple of months ago, and the bug was marked as fixed. 

James, for your follow-up you should really have chosen a new bug, or was it just a typo in the commit message? It would be great to finally get this patch pushed.
Flags: needinfo?(james)

Comment 13

8 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/fdbeb5e7c238
Reenable WebDriver spec tests, r=maja_zf
https://hg.mozilla.org/integration/autoland/rev/5827fdd9ac6d
Increase the HTTP connection timeout for WebDriver tests, r=ato
Oh, the bug was marked as fixed erroneously. The formerly landed patch simple disabled some tests.
Status: RESOLVED → REOPENED
Flags: needinfo?(james)
Resolution: FIXED → ---
Assignee: nobody → james
Status: REOPENED → ASSIGNED
status-firefox52: fixed → disabled
status-firefox53: fixed → disabled
status-firefox54: --- → disabled
status-firefox55: --- → disabled
status-firefox-esr52: --- → disabled
Target Milestone: mozilla53 → ---

Comment 15

8 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/fdbeb5e7c238
https://hg.mozilla.org/mozilla-central/rev/5827fdd9ac6d
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago8 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
James, can you care about the necessary uplift if that is doable for wpt-tests? Thanks.
status-firefox55: disabled → fixed
Flags: needinfo?(james)
(Assignee)

Comment 17

8 months ago
I'm not aware that uplift here is necessary.
Flags: needinfo?(james)

Comment 18

7 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e26fcd9ef4ae
Increase the HTTP connection timeout for WebDriver tests, r=ato

Comment 19

7 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/e26fcd9ef4ae

Comment 20

7 months ago
Backout by archaeopteryx@coole-files.de:
https://hg.mozilla.org/mozilla-central/rev/bca7ff5030f8
Backed out changeset e26fcd9ef4ae
https://hg.mozilla.org/mozilla-central/rev/bca7ff5030f8
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: Backed out wpt update for neverending W(7) jobs on Windows 7 VM debug:https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=f98b750bb85085a61341dc267f5ebf313820f960&filter-searchStr=2e76e93115e24a1caffbba81d961e84fb651fa8c
Backed out for neverending W(7) jobs on Windows 7 VM debug:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=f98b750bb85085a61341dc267f5ebf313820f960&filter-searchStr=2e76e93115e24a1caffbba81d961e84fb651fa8c
https://hg.mozilla.org/mozilla-central/rev/bca7ff5030f8
Status: REOPENED → RESOLVED
Last Resolved: 8 months ago7 months ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Flags: needinfo?(james)

Comment 23

7 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ab00229faf2c
Increase the HTTP connection timeout for WebDriver tests, r=ato

Comment 24

7 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/ab00229faf2c
Status: REOPENED → RESOLVED
Last Resolved: 7 months ago7 months ago
Resolution: --- → FIXED
(Assignee)

Updated

7 months ago
Flags: needinfo?(james)

Comment 25

6 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6432a9f65912
Increase the HTTP connection timeout for WebDriver tests, r=ato

Comment 26

6 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/6432a9f65912
You need to log in before you can comment on or make changes to this bug.