1680612 - |mach create-mach-environment| "Would build wheel with unsupported tag ('cp38', 'cp38', 'macosx_10_14_x86_64)"

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Mitchell Hentges [:mhentges] 🦀]

This happens in ./mach bootstrap and ./mach create-mach-environment -f on Intel Mac Big Sur with the XCode (system) Python.

Notes:

• It's reproducible by hand by:
  1. source ~/.mozbuild/_virtualenvs/mach/bin/source
  2. export ARCHFLAGS="-arch x86_64" see this patch
  3. pip install zstandard
• It's not reproducible with a python3 -m venv ... virtualenv, so there's something to do with the way that we create our mach virtualenv that causes this

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

:glandium, I'm getting a little bit behind with triage items, would you be able to take a look at this? Since it's related to our vendored virtualenv package, hopefully you'll have an easier time diagnosing this than me?

Comment 2

[Mike Hommey [:glandium]]

Can't reproduce :(

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Hmm, note that you may need to rm -r ~/Library/Caches/pip to reproduce this.

Also, I did a little more digging and found that it can be reproduced outside of mach:

1. Ensure you're running the system (XCode) python
2. pip3 install --user virtualenv==20.0.31 (not reproducible with the most recent virtualenv==20.2.2)
3. python3 -m virtualenv tmp-env
4. source tmp-env/bin/activate
5. export ARCHFLAGS="-arch x86_64"
6. pip install zstandard==0.13.0

I'll dig into this more and report a virtualenv bug

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Also worth noting that upgrading wheel within the virtualenv solves this problem, much like 1680802

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

After vendoring the new virtualenv, I'm seeing

mhentges@mhentges-64lvdr firefox % ./mach create-mach-environment -f
Traceback (most recent call last):
  File "/Users/mhentges/dev/firefox/third_party/python/virtualenv/virtualenv.py", line 55, in <module>
    run()
  File "/Users/mhentges/dev/firefox/third_party/python/virtualenv/virtualenv.py", line 49, in run
    from virtualenv.__main__ import run as run_virtualenv
  File "/Users/mhentges/dev/firefox/third_party/python/virtualenv/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run, session_via_cli
  File "/Users/mhentges/dev/firefox/third_party/python/virtualenv/virtualenv/run/__init__.py", line 6, in <module>
    from ..app_data import make_app_data
  File "/Users/mhentges/dev/firefox/third_party/python/virtualenv/virtualenv/app_data/__init__.py", line 12, in <module>
    from .read_only import ReadOnlyAppData
ModuleNotFoundError: No module named 'virtualenv.app_data.read_only'
Error running mach:

    ['create-mach-environment', '-f']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file create-mach-environment| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Failed to create virtualenv: /Users/mhentges/.mozbuild/_virtualenvs/mach (virtualenv.py retcode: 1)

  File "/Users/mhentges/dev/firefox/python/mozbuild/mozbuild/mach_commands.py", line 2096, in create_mach_environment
    manager.build(sys.executable)
  File "/Users/mhentges/dev/firefox/python/mozbuild/mozbuild/virtualenv.py", line 526, in build
    self.create(python)
  File "/Users/mhentges/dev/firefox/python/mozbuild/mozbuild/virtualenv.py", line 280, in create
    raise Exception(

# This is so sad

Doing a debugging run now to see what's up.

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

This is different but should be fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1680802