Closed Bug 1776642 Opened 2 years ago Closed 2 years ago

Cannot run `./mach bootstrap` with Python 3.10 on macOS - FileNotFoundError: [Errno 2] No such file or directory: '~/.mozbuild/srcdirs/mozilla-unified-7d5ac9f8ea6d/_virtualenvs/mach/lib/python3.10/site-packages/mach.pth'

Tracking

(Not tracked)

Status:

RESOLVED DUPLICATE of bug 1766497

People

(Reporter: jlorenzo, Unassigned)

References

Details

Attachments

(5 obsolete files)

Bug 1776642 - Bump virtualenv from 20.7.2 to 20.15.0 in order to run `mach boostrap` on Python 3.10 under macOS r=jcristau 2 years ago Johan Lorenzo [:jlorenzo] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1776642 - Bump virtualenv from 20.7.2 to 20.15.0 in order to run `mach boostrap` on Python 3.10 under macOS r=jcristau 2 years ago Johan Lorenzo [:jlorenzo] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1776642 - part 3: Bump virtualenv from 20.7.2 to 20.15.0 in order to run `mach boostrap` on Python 3.10 under macOS r=jcristau 2 years ago Johan Lorenzo [:jlorenzo] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1776642 - part 1: Bump jsmin to 3.0.0 to fix incompatibility with setuptools>=58 r=jcristau 2 years ago Johan Lorenzo [:jlorenzo] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1776642 - part 2: Stop installing 'carrot' in mach tests and use a more recent package instead r=jcristau 2 years ago Johan Lorenzo [:jlorenzo] 48 bytes, text/x-phabricator-request		Details \| Review

Johan Lorenzo [:jlorenzo]

Reporter

Description

•

2 years ago

Symptoms

On Friday, I wanted to bootstrap my machine one more time but I kept getting this error:


created virtual environment CPython3.10.5.final.0-64 in 22ms
  creator CPython3Posix(dest=~/.mozbuild/srcdirs/mozilla-unified-7d5ac9f8ea6d/_virtualenvs/mach, clear=False, no_vcs_ignore=False, global=False)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
Traceback (most recent call last):
  File "~/hg/mozilla-unified/./mach", line 96, in <module>
    main(sys.argv[1:])
  File "~/hg/mozilla-unified/./mach", line 88, in main
    mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))
  File "~/hg/mozilla-unified/./mach", line 35, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "~/hg/mozilla-unified/./mach", line 21, in load_mach
    return mach_initialize.initialize(dir_path)
  File "~/hg/mozilla-unified/build/mach_initialize.py", line 153, in initialize
    _activate_python_environment(
  File "~/hg/mozilla-unified/build/mach_initialize.py", line 99, in _activate_python_environment
    mach_environment.activate()
  File "~/hg/mozilla-unified/python/mach/mach/site.py", line 359, in activate
    self.ensure()
  File "~/hg/mozilla-unified/python/mach/mach/site.py", line 346, in ensure
    self._build()
  File "~/hg/mozilla-unified/python/mach/mach/site.py", line 384, in _build
    _create_venv_with_pthfile(
  File "~/hg/mozilla-unified/python/mach/mach/site.py", line 1188, in _create_venv_with_pthfile
    with open(os.path.join(platlib_site_packages_dir, PTH_FILENAME), "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: '~/.mozbuild/srcdirs/mozilla-unified-7d5ac9f8ea6d/_virtualenvs/mach/lib/python3.10/site-packages/mach.pth'

That was odd to me because I did manage to get bootstrap to work when I set up my new machine a few weeks ago. After deep diving into mach/site.py, virtualenv, python 3.9 vs python 3.10, and sysconfig. I managed to get the bottom of it.

My config

MacBook Pro arm64 (M1)
macOS 12.4
Python 3.10.5 installed via brew
Python 3.9.13 also installed via brew

A regression introduced in Python 3.10

Luckily, I managed to get ./mach bootstrap in a working state when I forced Python 3.9 in the shebang of mach[1]. diffoscope showed a weird change in the directory layout depending on what Python version mach uses to create a virtualenv. It turns out the site-packages folder is located under:

mozilla-unified-7d5ac9f8ea6d/_virtualenvs/mach/opt/homebrew/lib/python3.10/site-packages/ instead of
mozilla-unified-7d5ac9f8ea6d/_virtualenvs/mach/lib/python3.10/site-packages

Note the extraneous opt/homebrew/ in the first path. Thus, something was introduced in Python 3.10. It's a matter of figuring out what 🤔

Why were `sysconfig_paths` hardcoded?

I hooked a debugger at various points including this one[1]. It turns out the self.interpreter gets different values, notably in sysconfig_paths:

# Python 3.9 
PythonInfo({
    [...]

    'sysconfig_paths': {
        'purelib': '{base}/lib/python{py_version_short}/site-packages',
        'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages',
        'scripts': '{base}/bin',
        'data': '{base}'
    },
})


# Python 3.10
PythonInfo({
    [...]
    
    'sysconfig_paths': {
        [...]

        'purelib': '/opt/homebrew/lib/python{py_version_short}/site-packages',
        'platlib': '/opt/homebrew/{platlibdir}/python{py_version_short}/site-packages',
        'scripts': '/opt/homebrew/bin',
        'data': '/opt/homebrew'
    },
)

I was surprised to see all these hardcoded values! After debugging deeper and deeper, it turns out it's a change in the python library called sysconfig. virtualenv calls sysconfig.get_path()[3] and the behavior of this function changed in Python 3.10. It now determines a preferred scheme used to provide different sysconfig_paths depending on the type of system you're on. Details of such changes are greatly documented here[4]. The maintainer of the virtualenv packages is aware of these details and made the necessary changes in [5], which shipped in virtualenv 20.10.0.

What is the right fix?

We're currently using virtualenv 20.7.2 which doesn't include the aforementioned path. Hence it should be a matter of bumping our vendored version. I'm on it!

[1] https://searchfox.org/mozilla-central/rev/aa7c30cd45a7a8bbd2c969a31302db8f575796e6/mach#1
[2] https://searchfox.org/mozilla-central/rev/aa7c30cd45a7a8bbd2c969a31302db8f575796e6/third_party/python/virtualenv/virtualenv/create/describe.py#37
[3] https://searchfox.org/mozilla-central/rev/aa7c30cd45a7a8bbd2c969a31302db8f575796e6/third_party/python/virtualenv/virtualenv/discovery/py_info.py#76
[4] https://bugs.python.org/issue45413
[5] https://github.com/pypa/virtualenv/commit/8da79db86d8a5c74d03667a40e64ff832076445e