Closed Bug 1876182 Opened 1 year ago Closed 1 year ago

Updating moz-phab externally-managed-environment error on ubuntu 23.04/debian 12/macOS homebrew

Categories

(Conduit :: moz-phab, defect, P3)

Product:
Conduit
Component:
moz-phab
Platform:
All
Linux
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: glandium, Assigned: shtrom)

References

(Blocks 1 open bug)

Details

Attachments

(3 files)

self-update: continue on failure of opportunistic update (Bug 1876182) r?sheehan
48 bytes, text/x-phabricator-request
Details | Review
self-update: force PIP_BREAK_SYSTEM_PACKAGES when upgrading (Bug 1876182) r?sheehan
48 bytes, text/x-phabricator-request
Details | Review
self-update: set PIP_BREAK_SYSTEM_PACKAGES for non `--user` installs (Bug 1876182) r?sheehan
48 bytes, text/x-phabricator-request
Details | Review

The same thing as bug 1831442 happens when running moz-phab self-update:

$ moz-phab self-update
Version 1.5.0 of `moz-phab` is now available
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
CommandError: command '/usr/bin/python3' failed to complete successfully
Run moz-phab again with '--trace' to show debugging output

Workaround: Running mach install-moz-phab or PIP_BREAK_SYSTEM_PACKAGES=1 moz-phab self-update

I needed to append -f for the first variant: mach install-moz-phab -f

Command Arguments:
  --force, -f  Force installation even if already installed.

But then it worked for me.

(In reply to Manuel Bucher [:manuel] from comment #2)

I needed to append -f for the first variant: mach install-moz-phab -f

Command Arguments:
  --force, -f  Force installation even if already installed.

But then it worked for me.

Same here. I am on arch linux.

This error now also happens with Homebrew python as of 3.12: https://github.com/orgs/Homebrew/discussions/3404#discussioncomment-8549228

Summary: Updating moz-phab externally-managed-environment error on ubuntu 23.04/debian 12 → Updating moz-phab externally-managed-environment error on ubuntu 23.04/debian 12/macOS homebrew
Duplicate of this bug: 1892280

Workaround: Running mach install-moz-phab or PIP_BREAK_SYSTEM_PACKAGES=1 moz-phab self-update

fwiw, I arrived here after trying mach install-moz-phab

Also... mach install-moz-phab -f as suggested by Manuel, for me gives...

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['/opt/homebrew/opt/python@3.12/bin/python3.12', '-m', 'pip', 'show', '-f', 'MozPhab']' returned non-zero exit status 1.

  File "/Users/joe/projects/mozilla/mozilla-unified/tools/phabricator/mach_commands.py", line 117, in install_moz_phab
    info = subprocess.check_output(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,

I wonder if it's time to switch to uv as the recommended way to install across all platforms (using uv tool install MozPhab):

~$ moz-phab
zsh: command not found: moz-phab
~$ brew install uv
==> Downloading https://ghcr.io/v2/homebrew/core/uv/manifests/0.4.7
Already downloaded: /Users/byron/Library/Caches/Homebrew/downloads/58d99f76d775787e4fef9df9018c668445b3e2595161f29cafa050b8f2a6e1bf--uv-0.4.7.bottle_manifest.json
==> Fetching uv
==> Downloading https://ghcr.io/v2/homebrew/core/uv/blobs/sha256:5c1707312ea0acc82cfbdfdbba4df75bafa52f63fe6e8c0dd345af100e996f19
Already downloaded: /Users/byron/Library/Caches/Homebrew/downloads/ba64fe4f56df9e3729bdbe04c5a1cfde8800512bcdf3b9b576af60cd69138d68--uv--0.4.7.arm64_sonoma.bottle.tar.gz
==> Pouring uv--0.4.7.arm64_sonoma.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions
==> Summary
🍺  /opt/homebrew/Cellar/uv/0.4.7: 14 files, 22.3MB
~$ uv tool install MozPhab
Resolved 23 packages in 7ms
Installed 23 packages in 22ms
 + appdirs==1.4.4
 + attrs==24.2.0
 + certifi==2024.8.30
 + click==8.1.7
 + colorama==0.4.6
 + diskcache==5.6.3
 + distro==1.9.0
 + glean-parser==14.5.2
 + glean-sdk==60.5.0
 + jinja2==3.1.4
 + jsonschema==4.23.0
 + jsonschema-specifications==2023.12.1
 + markupsafe==2.1.5
 + mozphab==1.6.0
 + packaging==24.1
 + python-hglib==2.6.2
 + pyyaml==6.0.2
 + referencing==0.35.1
 + rpds-py==0.20.0
 + semver==3.0.2
 + sentry-sdk==2.14.0
 + setuptools==74.1.2
 + urllib3==2.2.2
Installed 1 executable: moz-phab
~$ moz-phab version
MozPhab 1.6.0 (Python 3.12.6, Darwin)
Duplicate of this bug: 1908680

We should probably try to avoid using homebrew python. It regularly causes subtle problems.

(In reply to :glob ✱ from comment #7)

I wonder if it's time to switch to uv as the recommended way to install across all platforms (using uv tool install MozPhab):

It would be a workaround until uv decides to enforce the same PEP... The real solution would be to use a virtualenv.

(In reply to Mike Hommey [:glandium] from comment #10)

(In reply to :glob ✱ from comment #7)

I wonder if it's time to switch to uv as the recommended way to install across all platforms (using uv tool install MozPhab):

It would be a workaround until uv decides to enforce the same PEP... The real solution would be to use a virtualenv.

uv tool uses virtualenvs.

(In reply to Mike Hommey [:glandium] from comment #9)

We should probably try to avoid using homebrew python. It regularly causes subtle problems.

Turns out uv has something for this too: uv python manages python versions and installations.

(In reply to :glob ✱ from comment #12)

(In reply to Mike Hommey [:glandium] from comment #9)

We should probably try to avoid using homebrew python. It regularly causes subtle problems.

Turns out uv has something for this too: uv python manages python versions and installations.

I'm not sure how much trust we can put in that, though. From a supply chain point of view. There's also the problem that this would only really support tier 1 platforms.

I was more thinking in terms of preferring the system python3 (the one that comes from the xcode command line tools that we require in the first place)

Duplicate of this bug: 1922187

requiring a python package manager written in rust ? couldnt we have something more ... classic ? in bug #1922187 i referred to pipx as it seemed being the natural replacement for pip, but using plain virtualenv could work too, instead of requiring more exotic/arcane tools...

So, a high level summary of my understanding is as follows:

This mechanism has been introduced by PEP 668 https://peps.python.org/pep-0668/ to prevent pip and a distro's package manager stepping on each other's feet. In essence, pip refuses to manage the packages in a system-installed Python (provided the system puts the right marker file). This is what we are seeing there.

We already tell pip to clobber to its heart's content in mach https://searchfox.org/mozilla-central/rev/7db217e3832e9a884be0c2300779e098f92be3fe/tools/phabricator/mach_commands.py#104, so that's how it installs moz-phab. Presumably, we could tell the self-update to then do the same thing. It's a bit dodgy, but that should work.

mach also forces the install to be --user https://searchfox.org/mozilla-central/rev/7db217e3832e9a884be0c2300779e098f92be3fe/tools/phabricator/mach_commands.py#101, so the clobbering is not even an issue anyway.

I personally use pipx to install python packages system-wide, but I'd be reluctant to do this here, as I'd rather avoid forcing a package manager onto the user. I'm also unsure about bespoke virtualenvs, because I've seen them break badly when python versions get updated, particularly on OSX and brew. This would also lead to problem with getting the command in one of the PATH entry, which I suspect will vary wildly.

Assignee: nobody → omehani
Status: NEW → ASSIGNED

mach also forces the install to be --user https://searchfox.org/mozilla-central/rev/7db217e3832e9a884be0c2300779e098f92be3fe/tools/phabricator/mach_commands.py#101, so the clobbering is not even an issue anyway.

It may still be a bit of an issue, in that installing MozPhab as a user may still install modules that shadow system packages. From my reading of the PEP, this is considered an issue. However, this is an latent issue we already have.

We can pass the --break-system-packages option to pip, but it was only added in https://github.com/pypa/pip/blob/d5c8c115f9dcfc06d2e22c553888acc6a10f61f8/NEWS.rst#2301-2023-02-17, and will break users of previous versions of pip, if any, so the ENV variable is probably a safer bet.

Bookworm shipped with Python 3.9 and pip 20.3.4, so using the option could be a problem on outdated setups (if any).

(In reply to Markus Stange [:mstange] from comment #4)

This error now also happens with Homebrew python as of 3.12: https://github.com/orgs/Homebrew/discussions/3404#discussioncomment-8549228

I'd like to determine if your moz-phab has been installed with --user. Could you run the following on the machine that you ran into this issue with and give us the output?

python  -c 'import mozphab as m; print(m.__file__)'

Could you also confirm that the behaviour still happens, by running moz-phab self-update?

Flags: needinfo?(mstange.moz)

(In reply to Joe Walker [:jwalker] from comment #6)

Also... mach install-moz-phab -f as suggested by Manuel, for me gives...

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['/opt/homebrew/opt/python@3.12/bin/python3.12', '-m', 'pip', 'show', '-f', 'MozPhab']' returned non-zero exit status 1.

  File "/Users/joe/projects/mozilla/mozilla-unified/tools/phabricator/mach_commands.py", line 117, in install_moz_phab
    info = subprocess.check_output(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,

Joe, it looks like your stack trace is truncated. Are you able to reproduce the issue and get a full one?

Flags: needinfo?(jwalker)

(In reply to Olivier Mehani [:shtrom] from comment #20)

(In reply to Markus Stange [:mstange] from comment #4)

This error now also happens with Homebrew python as of 3.12: https://github.com/orgs/Homebrew/discussions/3404#discussioncomment-8549228

I'd like to determine if your moz-phab has been installed with --user. Could you run the following on the machine that you ran into this issue with and give us the output?

python  -c 'import mozphab as m; print(m.__file__)'

% python  -c 'import mozphab as m; print(m.__file__)'
zsh: command not found: python
% python3 -c 'import mozphab as m; print(m.__file__)' 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mozphab'

In the meantime I've downgraded to homebrew python 3.11 for other reasons (bug 1875940 comment 3), and I may have reinstalled moz-phab a few times since then too. Not sure how - maybe I used ./mach install-moz-phab this time. The first link on https://www.google.com/search?q=moz-phab+site%3Ahttps%3A%2F%2Ffirefox-source-docs.mozilla.org%2F brings me to https://firefox-source-docs.mozilla.org/devtools/contributing/making-prs.html which unfortunately doesn't have a link to a page that tells me how to install moz-phab.

Could you also confirm that the behaviour still happens, by running moz-phab self-update?

Running it now I got a different warning ("Please avoid running setup.py directly") and it then proceeded to update to 1.6.1. Running moz-phab self-update again, I get no more warnings. But this is with python 3.11.

Flags: needinfo?(mstange.moz)

mach installs moz-phab with PIP_BREAK_SYSTEM_PACKAGES=1. This change
replicates the setup to avoid failures updating an
externally-managed-environment.

This relies strongly on the assumption than users have moz-phab
installed by mach, rather than other means, which would potentially
trigger more issues related to externally managed environments.

Signed-off-by: Olivier Mehani <omehani@mozilla.com>

Merged. We need to confirm the behaviour on OS X.

We'll create two RCs, and make sure upgrading from one to the other happens without issue.

  1. Create a new RC 1.7.0rc0 and create a release on PyPi (in progress)
  2. Make sure Mac is running Python 3.12+ from Homebrew
  3. Set updater.get_pre_releseases=True in ~/.moz-phab-config
  4. On MozPhab 1.6.1, run moz-phab self-update (this is expected to fail)
  5. On MozPhab 1.6.1, run PIP_BREAK_SYSTEM_PACKAGES=1 moz-phab self-update (this should upgrade to 1.7.0rc0 without error)
  6. Create a new RC 1.7.0rc1, and create a release on PyPi
  7. On MozPhab 1.7.0rc0, run moz-phab self-update (this should upgrade to 1.7.0rc1 without error)
See Also: → 1924916

Joe, it looks like your stack trace is truncated. Are you able to reproduce the issue and get a full one?

I know this is a bit stale now, but no, sorry - I found a workaround at the time.

Flags: needinfo?(jwalker)

(In reply to Olivier Mehani [:shtrom] from comment #26)

Merged. We need to confirm the behaviour on OS X.

We'll create two RCs, and make sure upgrading from one to the other happens without issue.

  1. Create a new RC 1.7.0rc0 and create a release on PyPi (in progress)
  2. Make sure Mac is running Python 3.12+ from Homebrew
  3. Set updater.get_pre_releseases=True in ~/.moz-phab-config
  4. On MozPhab 1.6.1, run moz-phab self-update (this is expected to fail)
  5. On MozPhab 1.6.1, run PIP_BREAK_SYSTEM_PACKAGES=1 moz-phab self-update (this should upgrade to 1.7.0rc0 without error)
  6. Create a new RC 1.7.0rc1, and create a release on PyPi
  7. On MozPhab 1.7.0rc0, run moz-phab self-update (this should upgrade to 1.7.0rc1 without error)

I worked with Connor to confirm that this works as expected on OS X.

The fix for this is out in 1.7.0.

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED

Of course, you'll run into the issue when upgrading to 1.7.0. The fix is to force it, one last time, with

PIP_BREAK_SYSTEM_PACKAGES=1 moz-phab self-update
Severity: -- → S3
Priority: -- → P3
See Also: → 1956074
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: