1692254 - `./mach vendor python` "make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir'"

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, defect)

Description

[Mitchell Hentges [:mhentges] 🦀]

On central (ee330ff) run ./mach vendor python.

/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/scripts/compile.py:306: FutureWarning: --index and --no-index are deprecated and will be removed in future versions. Use --emit-index-url/--no-emit-index-url instead.
  warnings.warn(
Traceback (most recent call last):
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/home/mitch/dev/firefox/third_party/python/Click/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/mitch/dev/firefox/third_party/python/Click/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/mitch/dev/firefox/third_party/python/Click/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mitch/dev/firefox/third_party/python/Click/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/mitch/dev/firefox/third_party/python/Click/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/scripts/compile.py", line 458, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/resolver.py", line 173, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/resolver.py", line 278, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/resolver.py", line 388, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/repositories/local.py", line 75, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 231, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
  File "/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 155, in resolve_reqs
    preparer = self.command.make_requirement_preparer(
TypeError: make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir'
Error running mach:

    ['vendor', 'python']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file vendor| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['/home/mitch/dev/firefox/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/bin/pip-compile', 'requirements-mach-vendor-python.in', '--no-header', '--no-index', '--output-file', '/home/mitch/dev/firefox/third_party/python/requirements.txt', '--generate-hashes']' returned non-zero exit status 1.

  File "/home/mitch/dev/firefox/python/mozbuild/mozbuild/vendor/mach_commands.py", line 174, in vendor_python
    vendor_command.vendor(**kwargs)
  File "/home/mitch/dev/firefox/python/mozbuild/mozbuild/vendor/vendor_python.py", line 44, in vendor
    subprocess.check_output(
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

This error only occurs if ~/.cache/pip-tools is empty (or third_party/python/requirements.txt is cleared, such as if you had CTRL-C'd while ./mach vendor python was running).

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

pip 20.3.1 doesn't have wheel_download_dir as a kwarg in its make_requirement_preparer(...) function anymore, thus this failure.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1692254: Bump pip-tools package

pip-tools@5.3.1 is only supposed to support up
to pip 20.1.1 anyways, according to their docs.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1680802

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/723f34f07882
Bump pip-tools package r=sheehan,firefox-build-system-reviewers,andi,glandium

Comment 6

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/723f34f07882