1747837 - Something like mach create-mach-environment is dearly needed

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Mike Hommey [:glandium]]

I was setting up a build on an exotic platform that we don't have wheels for for psutil, zstandard, and glean, so the mach virtualenv attempted to build them, but I didn't have the python3-dev package installed. It still went through, and then bootstrapping the wasi sysroot failed because zstandard was not available. Up to that, nothing is really surprising.

The problem is that after I installed the python3-dev package, there wasn't an obvious way to fix the mach virtualenv:

• it doesn't have the pip command
• the mach create-mach-environment command was removed

It turns out even though it doesn't have the pip command, it has the pip module, so you can run python -m pip with the virtualenv python, but the location of the virtualenv itself is non-trivial for users.

./mach python --virtualenv mach -m pip ... would have been helpful and would avoid having to know the location of the virtualenv, but instead of using the mach virtualenv, it creates a new virtualenv named mach in the objdir (which I would argue is a bug).

Even if you do know how to run pip for the mach virtualenv, it's also non-obvious what you might want to retry installing that failed the first time.

All in all, the easiest is to remove the virtualenv and recreate it, but it all comes back to knowing where the virtualenv is in the first place.

So all in all, I think we need a command to force-refresh the mach virtualenv.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug is a defect, but please change it back in case of error.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1717051

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

This will be improved by this patch, which will have ./mach bootstrap attempt to install Python packages.

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1747837: Attempt to reinstall optional dependencies during bootstrap

Currently, developers don't have a way to have the Mach virtualenv
re-attempt to install optional dependencies (such as glean).

As part of ./mach bootstrap (the general catch-all "re-create my dev
environment" command), we should retry installing optional dependencies.

This also matches the "glean isn't installed" error message
recommendation.

Note: This doesn't address the case in which command virtualenvs
need their optional dependencies attempted to be reinstalled.
However, since we don't have any such cases yet, I'm satisfied with
deferring that work.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mhentges, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[Phabricator Automation]

Comment on attachment 9257064 [details]
Bug 1747837: Attempt to reinstall optional dependencies during bootstrap

Revision D123242 was moved to bug 1717051. Setting attachment 9257064 [details] to obsolete.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/573fe67e2e21
Attempt to reinstall optional dependencies during bootstrap r=ahal

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/573fe67e2e21