Open Bug 1597809 Opened 8 months ago Updated 4 months ago

[mozbuild] Pipenv misbehaves, mozbuild/test/configure fails when pyenv is installed on the host

Categories

(Firefox Build System :: General, defect, P3)

Unspecified
Linux
defect

Tracking

(Not tracked)

People

(Reporter: mars, Unassigned)

References

(Blocks 1 open bug)

Details

When pyenv is present on the host it changes how mozbuild/virtualenv.py and Pipenv behave. This leads to unexpected failures in the mozbuild/test/configure tests and causes unnecessary virtual environment rebuilds during ./mach configure.

The mozbuild virtualenv management system makes a few assumptions about the host environment:

  • The path to the Python the user is installing in the Firefox dev environment is a native executable, not a symlink or wrapper script.
  • Pipenv (used to manage the Python test virtualenv) will use the system Python when it's asked to install Python 2 or Python 3.

pyenv breaks both of these assumptions.

Pipenv has magic that will search pyenv's installed Pythons and use them if a $HOME/.pyenv directory is present. Those versions may not match the Pythons installed on the system. The user could use the system python2 or python3 to bootstrap their dev environment, then have Pipenv pick up a different python2 or python3 from the pyenv version registry. This leads to unnecessary virtualenv rebuilds and test suite failures.

Because Pipenv picks a different Python the tests in mozbuild/tests/configure rebuild the same virtualenv in parallel: the rebuilds clobber each other and the tests fail. This makes mozconfigure development and testing difficult.

This can be worked around by removing pyenv from the system. This can be done by running mozbuild and the test suite in a docker container. Alternatively we can disable pyenv's shell integration, removing everything pyenv and pipenv related from $PATH, and rename the $HOME/.pyenv directory.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: General → Bootstrap Configuration

Sadly, bugbug made a mistake. Reverting.

Component: Bootstrap Configuration → General
You need to log in before you can comment on or make changes to this bug.