1421446 - TVw sometimes fails: Unable to find geckodriver binary in common test package

Status: RESOLVED FIXED

(Testing :: General, enhancement)

Description

[Geoff Brown [:gbrown]]

Test verification for web-platform tests, TVw, currently runs as tier 3. Sometimes it fails with "Unable to find geckodriver binary in common test package" as in

https://treeherder.mozilla.org/logviewer.html#?job_id=147996796&repo=mozilla-central&lineNumber=757

TVw sometimes (when applicable tests are modified) tries to run wdspec tests, which normally only run on Linux. If TVw tries to run wdspec on non-Linux platforms, this failure occurs.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: do not attempt verification of wdspec tests if geckodriver is not available

Comment 2

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad5d1bc18db15ff8d4ad315138d0bbc2d916b417&filter-tier=1&filter-tier=2&filter-tier=3

wdspec test find_element_from_element.py is verified on Linux and is not verified on Windows.

Comment 3

[James Graham [:jgraham]]

Comment on attachment 8932951 [details] [diff] [review]
do not attempt verification of wdspec tests if geckodriver is not available

Review of attachment 8932951 [details] [diff] [review]:
-----------------------------------------------------------------

I assume this happens in the case of an artifact build, or (currently) on non-linux platforms. Please at least add a warning to the logs in this case.

::: testing/mozharness/scripts/web_platform_tests.py
@@ +326,5 @@
>          if self.config.get("verify") is True:
>              verify_suites = self.query_verify_category_suites(None, None)
> +            if "wdspec" in verify_suites:
> +                # geckodriver is required for wdspec, but not always available
> +                geckodriver_path = os.path.join(dirs["abs_test_bin_dir"], "geckodriver")

If we run this on Windows, this path will be wrong. Does mozharness not have a generic facility for getting the path to an executable?

Comment 4

[Geoff Brown [:gbrown]]

It's hard to use query_exe() because the taskcluster jobs remove configured executables with the remove_executables configuration. Rather than try to sort out that mess, I moved the path into the config file as a new parameter and noted the limitation. Also added the suggested warning. Thanks!

https://treeherder.mozilla.org/#/jobs?repo=try&revision=831c5aecef0d8fee3a256b34c75c504516befd00&filter-tier=1&filter-tier=2&filter-tier=3

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8e8ee880a5a8
In TVw, skip wdspec tests if geckodriver is not available; r=jgraham

Comment 6

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/8e8ee880a5a8