1914348 - Running web-platform tests using --product=chrome/safari does not work

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P2)

Description

[Jan Jaeschke [:jjaschke]]

Running this command on macOS:

./mach wpt --product=[chrome|safari] /path/to/test

Causes import errors:

safari

ModuleNotFoundError: No module named 'webdriver.client'

  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 560, in run_wpt
    return run_web_platform_tests(command_context, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 548, in run_web_platform_tests
    return wpt_runner.run(logger, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands_base.py", line 55, in run
    kwargs = self.setup.kwargs_wptrun(kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 188, in kwargs_wptrun
    browser_cls, kwargs = run.setup_wptrunner(venv, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wpt/run.py", line 980, in setup_wptrunner
    wptcommandline.check_args(wptrunner_kwargs)
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/wptcommandline.py", line 580, in check_args
    set_from_config(kwargs)
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/wptcommandline.py", line 466, in set_from_config
    kwargs["product"] = products.Product(kwargs["config"], kwargs["product"])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/products.py", line 20, in __init__
    module = product_module(config, product)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/products.py", line 11, in product_module
    module = importlib.import_module("wptrunner.browsers." + product)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/safari.py", line 14, in <module>
    from ..executors.executorwebdriver import (WebDriverTestharnessExecutor,  # noqa: F401
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorwebdriver.py", line 48, in <module>
    from webdriver.client import Session

chrome

ImportError: cannot import name 'error' from 'webdriver' (unknown location)

  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 560, in run_wpt
    return run_web_platform_tests(command_context, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 548, in run_web_platform_tests
    return wpt_runner.run(logger, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands_base.py", line 55, in run
    kwargs = self.setup.kwargs_wptrun(kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/mach_commands.py", line 188, in kwargs_wptrun
    browser_cls, kwargs = run.setup_wptrunner(venv, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wpt/run.py", line 980, in setup_wptrunner
    wptcommandline.check_args(wptrunner_kwargs)
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/wptcommandline.py", line 580, in check_args
    set_from_config(kwargs)
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/wptcommandline.py", line 466, in set_from_config
    kwargs["product"] = products.Product(kwargs["config"], kwargs["product"])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/products.py", line 20, in __init__
    module = product_module(config, product)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/products.py", line 11, in product_module
    module = importlib.import_module("wptrunner.browsers." + product)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/chrome.py", line 18, in <module>
    from ..executors.executorchrome import (  # noqa: F401
  File "/Users/jan-niklasjaschke/mozilla-unified-git/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorchrome.py", line 9, in <module>
    from webdriver import error

On other machines, this seems to be only reproducible using ./mach wpt --product=safari --webdriver-binary=/usr/bin/safaridriver /path/to/test.

Already tried bootstrapping, clobber, build.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This actually regressed by bug 1892709, which added WebDriver BiDi support to the testdriver. It's a downstream sync from https://github.com/web-platform-tests/wpt/pull/45823. I've heard that there were problems upstream as well but we would have to check the current state of that code.

It would be good to know if it also fails by running tests directly in the upstream repository.

Sasha, could you maybe take a look?

Comment 2

[Alexandra Borovova [:Sasha]]

Attachment: Bug 1914348 - Fix webdriver and wptrunner module imports.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4137badaba5a
Fix webdriver and wptrunner module imports. r=webdriver-reviewers,jdescottes

Comment 4

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/47808 for changes under testing/web-platform/tests

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 6

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/4137badaba5a

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a8e61418b730
[wpt PR 47808] - [Gecko Bug 1914348] Fix webdriver and wptrunner module imports., a=testonly

Comment 9

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/a8e61418b730