TVw sometimes fails: Unable to find geckodriver binary in common test package

RESOLVED FIXED in Firefox 59

Status

Testing
General
RESOLVED FIXED
2 months ago
2 months ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

Trunk
mozilla59
Points:
---

Firefox Tracking Flags

(firefox59 fixed)

Details

Attachments

(1 attachment)

(Assignee)

Description

2 months ago
Test verification for web-platform tests, TVw, currently runs as tier 3. Sometimes it fails with "Unable to find geckodriver binary in common test package" as in

https://treeherder.mozilla.org/logviewer.html#?job_id=147996796&repo=mozilla-central&lineNumber=757

TVw sometimes (when applicable tests are modified) tries to run wdspec tests, which normally only run on Linux. If TVw tries to run wdspec on non-Linux platforms, this failure occurs.
(Assignee)

Comment 1

2 months ago
Created attachment 8932951 [details] [diff] [review]
do not attempt verification of wdspec tests if geckodriver is not available
Attachment #8932951 - Flags: review?(james)
(Assignee)

Comment 2

2 months ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad5d1bc18db15ff8d4ad315138d0bbc2d916b417&filter-tier=1&filter-tier=2&filter-tier=3

wdspec test find_element_from_element.py is verified on Linux and is not verified on Windows.
Comment on attachment 8932951 [details] [diff] [review]
do not attempt verification of wdspec tests if geckodriver is not available

Review of attachment 8932951 [details] [diff] [review]:
-----------------------------------------------------------------

I assume this happens in the case of an artifact build, or (currently) on non-linux platforms. Please at least add a warning to the logs in this case.

::: testing/mozharness/scripts/web_platform_tests.py
@@ +326,5 @@
>          if self.config.get("verify") is True:
>              verify_suites = self.query_verify_category_suites(None, None)
> +            if "wdspec" in verify_suites:
> +                # geckodriver is required for wdspec, but not always available
> +                geckodriver_path = os.path.join(dirs["abs_test_bin_dir"], "geckodriver")

If we run this on Windows, this path will be wrong. Does mozharness not have a generic facility for getting the path to an executable?
Attachment #8932951 - Flags: review?(james) → review+
(Assignee)

Comment 4

2 months ago
It's hard to use query_exe() because the taskcluster jobs remove configured executables with the remove_executables configuration. Rather than try to sort out that mess, I moved the path into the config file as a new parameter and noted the limitation. Also added the suggested warning. Thanks!

https://treeherder.mozilla.org/#/jobs?repo=try&revision=831c5aecef0d8fee3a256b34c75c504516befd00&filter-tier=1&filter-tier=2&filter-tier=3

Comment 5

2 months ago
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8e8ee880a5a8
In TVw, skip wdspec tests if geckodriver is not available; r=jgraham

Comment 6

2 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/8e8ee880a5a8
Status: NEW → RESOLVED
Last Resolved: 2 months ago
status-firefox59: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You need to log in before you can comment on or make changes to this bug.