1697833 - "python3 -m venv" fails with WinError 2

Status: RESOLVED FIXED

(Firefox Build System :: MozillaBuild, defect)

Description

[Mitchell Hentges [:mhentges] 🦀]

I'm using MozillaBuild 3.3.

$ python3 -m venv env
Error: [WinError 2] The system cannot find the file specified

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

See this msys2 issue.
Sounds like it's been fixed in mid-2019 with this patch - but we use msys, not msys2 when bundling MozillaBuild, eh?

EDIT: this issue was unrelated.

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1697833: Fix "python3 -m venv" failing with "WinError 2"

The root cause of this failure is that venv tries to execute
<venv>/Scripts/<basename(sys.executable), which doesn't exist.

This is because:

• On Windows, venv only creates python[w].exe in virtualenvs, not
  python3.exe or python3.x.exe
• Unlike "regular" Windows, we create nix-y python2.exe and
  python3.exe copies in their respective Python installation
  directories. This enables the nix-y python(2|3) usage in the shell.

So, when running python3 -m venv:

1. The general directories are created
2. <venv>/Scripts/python.exe is created
3. the "executable" is set to <venv>/Scripts/python3.exe, which
   doesn't exist

This patch works around the issue by having venv *alwaysset the virtualenv context's executable topython.exe. Sincepython.exe` is
always created, this should be correct.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1697833: Don't install "virtualenv" by default

• When Mach uses virtualenv, it leans on its vendored copy
• When pip-installed tools need virtualenv, they fetch it
  automatically
• MozillaBuild itself doesn't need virtualenv
• If users or downstream consumers need virtualenv, they can install
  it

Depends on D132274

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1697833: Remove "python3.7.exe"

These binaries don't seem useful:

• To select between Python 2 and 3, use python2 and python3
  respectively
• There aren't multiple versions of, say, Python 3 installed.
  python3.x.exe is one-to-one with python3.exe
• If a downstream consumer is verifying the Python version, they can
  assert the MozillaBuild version or python3.exe --version instead.
• If this is to maintain consistency with *nix-y environments: this
  facade falls apart when virtualenvs are created anyways - they only
  create python.exe, and if you tweak venv to also create
  python3.exe/python3.x.exe, they fail to load due to a "missing
  python3x.dll" error.

Depends on D132276

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1697833: "python" should refer to Python 3, not Python 2

Python 2 is essentially just kicking around for hg compatibility.
If users run python (without specifying a version), use the Python 3
installation by default.

Depends on D132277

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-build/rev/812ff43eced7
Fix "python3 -m venv" failing with "WinError 2" r=glob
https://hg.mozilla.org/mozilla-build/rev/fe6a9008dfcf
Don't install "virtualenv" by default r=glandium
https://hg.mozilla.org/mozilla-build/rev/1b936ffc14ec
Remove "python3.7.exe" r=glandium
https://hg.mozilla.org/mozilla-build/rev/72ae55fc5db3
"python" should refer to Python 3, not Python 2 r=glandium

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1697833: Bump Python3, remove old `venv` patch

Python 3.9.10 has incorporated a fix for the venv issue,
so our patch is no longer needed.

Depends on D136413

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-build/rev/1705b34319e1
Bump Python3, remove old `venv` patch r=glandium