Open Bug 1598482 Opened 2 years ago Updated 1 year ago

Missing Taskcluster third_party dependencies make 'mach vendor python' hard to use


(Firefox Build System :: General, defect, P3)



(Not tracked)


(Reporter: mars, Unassigned)


(Blocks 1 open bug)


The taskcluster dependency on aiohttp and it's sub-dependencies are missing from the third_party/python directory. The missing deps show up in the "to be upgraded" list whenever someone attempts to use mach vendor python to upgrade an individual package. It makes use of mach vendor python confusing and tedious to repair.

Steps to reproduce:

  1. mach vendor python taskcluster==6.0.0
  2. hg st third_party to see the list of missing packages the vendor command has added.

Expected output:
No changes to the third_party/python/ directory.

Actual output:
aiohttp, yarl and friends have appeared in the third_party/python/ directory.

To see how we know taskcluster is causing the problem, run:

  1. cd obj-*/_virtualenvs/init_py3/bin
  2. ./pip-compile ../../../../third_party/python/ --output-file=-

It sounds like the fix would be to install those dependencies?

Product: Taskcluster → Firefox Build System

Yes, we either need to vendor the dependencies, or stop vendoring the taskcluster client and install it at runtime as needed (only appears used by mozrelease).

FWIW this issue has gotten worse and doing no-op "mach vendor python" runs now installs even more packages.

I wonder if a feasible solution is to "declare bankruptcy" by just vendoring all these packages once and for all in a giant patch.

I'm +1 for vendoring the missing libraries in a single patch. I'm up for reviewing the patch for whoever does this.

Bug 1601140 fixed this, but I don't know how to properly resolve this bug. Dupe, worksforme, depend, morph into "vendor this again once it's py3" and unblock mach-busted ... ? Any preferences?

You need to log in before you can comment on or make changes to this bug.