1555212 - ./mach wpt --product chrome doesn't work

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P2)

Description

[(no longer active)]

ehsan@local:~/moz/src$ ./mach wpt --product chrome dom/historical.html
 0:00.03 INFO Skipping manifest download because existing file is recent

Usage:
  pip install [options] <requirement specifier> [package-index-options] ...
  pip install [options] -r <requirements file> [package-index-options] ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: --prefer-binary
Error running mach:

    ['wpt', '--product', 'chrome', 'dom/historical.html']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

CalledProcessError: Command '(u'/home/ehsan/moz/src/obj-ff-opt/_virtualenvs/init/bin/pip', 'install', '--prefer-binary', '-r', '/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/requirements_chrome.txt')' returned non-zero exit status 2

  File "/home/ehsan/moz/src/testing/web-platform/mach_commands.py", line 280, in run_wpt
    return self.run_web_platform_tests(**params)
  File "/home/ehsan/moz/src/testing/web-platform/mach_commands.py", line 272, in run_web_platform_tests
    return wpt_runner.run(logger, **params)
  File "/home/ehsan/moz/src/testing/web-platform/mach_commands_base.py", line 37, in run
    kwargs = self.setup.kwargs_wptrun(kwargs)
  File "/home/ehsan/moz/src/testing/web-platform/mach_commands.py", line 141, in kwargs_wptrun
    **kwargs)
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wpt/run.py", line 522, in setup_wptrunner
    setup_cls.install_requirements()
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wpt/run.py", line 175, in install_requirements
    self.venv.install_requirements(os.path.join(wpt_root, "tools", "wptrunner", self.browser.requirements))
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wpt/virtualenv.py", line 109, in install_requirements
    self.pip_path, "install", "--prefer-binary", "-r", requirements_path
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wpt/utils.py", line 48, in call
    return subprocess.check_output(args)
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
ehsan@local:~/moz/src$

Comment 1

[James Graham [:jgraham]]

Our in-tree pip is unfortunately very outdated, and doesn't support the options required upstream. There's a workaround you can use in the short term; something like

source /home/ehsan/moz/src/obj-ff-opt/_virtualenvs/init/bin/activate
pip install --upgrade pip
deactivate
./mach wpt --product chrome dom/historical.html

obviously you need to do that once per clean objdir, so it's not ideal. I need to look at either updating the in-tree pip, which I understand has caused problems in the past, or hacking things so we don't go down the install_requirements codepath e.g. by pre-installing those requirements from the mach frontend.

Comment 2

[James Graham [:jgraham]]

Attachment: Bug 1555212 - Make wpt --product install the dependencies in the mach layer,

This avoids a mismatch in the available arguments between upstream pip and
the version vendored into m-c at the cost of the code being more liable
to break when upstream changes.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9ed7d3559da8
Make wpt --product install the dependencies in the mach layer, r=ato

Comment 4

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/9ed7d3559da8