Test verification for web-platform tests, TVw, currently runs as tier 3. Sometimes it fails with "Unable to find geckodriver binary in common test package" as in https://treeherder.mozilla.org/logviewer.html#?job_id=147996796&repo=mozilla-central&lineNumber=757 TVw sometimes (when applicable tests are modified) tries to run wdspec tests, which normally only run on Linux. If TVw tries to run wdspec on non-Linux platforms, this failure occurs.
Created attachment 8932951 [details] [diff] [review] do not attempt verification of wdspec tests if geckodriver is not available
Attachment #8932951 - Flags: review?(james)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad5d1bc18db15ff8d4ad315138d0bbc2d916b417&filter-tier=1&filter-tier=2&filter-tier=3 wdspec test find_element_from_element.py is verified on Linux and is not verified on Windows.
Comment on attachment 8932951 [details] [diff] [review] do not attempt verification of wdspec tests if geckodriver is not available Review of attachment 8932951 [details] [diff] [review]: ----------------------------------------------------------------- I assume this happens in the case of an artifact build, or (currently) on non-linux platforms. Please at least add a warning to the logs in this case. ::: testing/mozharness/scripts/web_platform_tests.py @@ +326,5 @@ > if self.config.get("verify") is True: > verify_suites = self.query_verify_category_suites(None, None) > + if "wdspec" in verify_suites: > + # geckodriver is required for wdspec, but not always available > + geckodriver_path = os.path.join(dirs["abs_test_bin_dir"], "geckodriver") If we run this on Windows, this path will be wrong. Does mozharness not have a generic facility for getting the path to an executable?
Attachment #8932951 - Flags: review?(james) → review+
It's hard to use query_exe() because the taskcluster jobs remove configured executables with the remove_executables configuration. Rather than try to sort out that mess, I moved the path into the config file as a new parameter and noted the limitation. Also added the suggested warning. Thanks! https://treeherder.mozilla.org/#/jobs?repo=try&revision=831c5aecef0d8fee3a256b34c75c504516befd00&filter-tier=1&filter-tier=2&filter-tier=3
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/mozilla-inbound/rev/8e8ee880a5a8 In TVw, skip wdspec tests if geckodriver is not available; r=jgraham
Status: NEW → RESOLVED
Last Resolved: 2 months ago
status-firefox59: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You need to log in before you can comment on or make changes to this bug.