1596422 - ./mach lint does not work on MacOS 10.15 due to Python 3 virtualenv errors

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Trying to run ./mach lint results in the following error:

./mach lint testing/mochitest/runtests.py
Using base prefix '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7'
/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3/bin/python3
Also creating executable in /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3/bin/python
Traceback (most recent call last):
  File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 2349, in <module>
    main()
  File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 1342, in install_python
    shutil.copy(original_python, py_executable)
UnboundLocalError: local variable 'original_python' referenced before assignment
Error running mach:

    ['lint', 'testing/mochitest/runtests.py']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Failed to create virtualenv: /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3

  File "/Users/tritter/Documents/moz/mozilla-unified/tools/lint/mach_commands.py", line 63, in lint
    self._activate_virtualenv()
  File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/base.py", line 818, in _activate_virtualenv
    self.virtualenv_manager.ensure()
  File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 178, in ensure
    return self.build(python)
  File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 477, in build
    self.create(python)
  File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 217, in create
    'Failed to create virtualenv: %s' % self.virtualenv_root)

If I remove obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/ and run it again, I get the same error.

I have a fresh mach bootstrap, I just updated and rebased central, and my mozconfig is:

ac_add_options --with-macos-sdk=/Users/tritter/Documents/src/osx-sdk/MacOSX10.11.sdk
ac_add_options --enable-debug
mk_add_options MOZ_OBJDIR=obj-x86_64-apple-darwin18.5.0.opt
ac_add_options --enable-optimize

Comment 1

[Māris Fogels [:mars] (please needinfo)]

This is one of those fun bugs where Google returns a single hit: https://github.com/pypa/virtualenv/issues/750.

Could you please rerun the mach command with the environment variable _VIRTUALENV_DEBUG set to 1? virtualenv.py should dump a bunch of extra information to the console.

$ env _VIRTUALENV_DEBUG=1 ./mach lint testing/mochitest/runtests.py

Comment 2

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Attachment: more verbose output.txt

Comment 3

[Māris Fogels [:mars] (please needinfo)]

The version of virtualenv.py bundled in the Firefox source tree is out of date for the development tools that you are using. We need to upgrade to virtualenv.py v16.7.6.

Issue: https://github.com/pypa/virtualenv/issues/1402
Fixed by: https://github.com/pypa/virtualenv/pull/1425
Release: https://virtualenv.pypa.io/en/latest/changes/#v16-7-6-2019-10-16

Comment 4

[Māris Fogels [:mars] (please needinfo)]

We are on virtualenv v15.2.0: https://searchfox.org/mozilla-central/source/third_party/python/requirements.txt#121

Comment 5

[Sebastian Zartner [:sebo]]

Attachment: mach lint output on Windows 10

For what it's worth, I have the same issue on Windows 10. See the attached output

Sebastian

Comment 6

[Māris Fogels [:mars] (please needinfo)]

(In reply to Sebastian Zartner [:sebo] from comment #5)

For what it's worth, I have the same issue on Windows 10. See the attached
output

Sebastian

Thanks for the report. That's a different error, "AssertionError: SRE module mismatch". Could you please file a new bug for it?

Comment 7

[Sebastian Zartner [:sebo]]

Ah, didn't realize that it's a different reason. Thanks for the fast response! I've filed bug 1599244 for it now.

Sebastian

Comment 8

[Māris Fogels [:mars] (please needinfo)]

We've found a workaround for this bug. It only affects developers who are building Firefox with the Xcode-bundled Python 3. Developers who install Python 3 from Homebrew and set $PATH appropriately are not affected. See bug 1600720 comment 7 for details.

Comment 9

[Māris Fogels [:mars] (please needinfo)]

:tjr, a fix for this has landed in mozilla-central (see bug 1464038). If you are still building with Xcode could you please try pulling the latest mozilla-central and see if ./mach lint works?

Comment 10

[Māris Fogels [:mars] (please needinfo)]

@Ricky Are you able to verify this bugfix on your system? It should be possible by setting $PATH to the Python 3 framework installation. env PATH=/path/to/Python3.framework/some/bin ./mach lint.

Comment 11

[Ricky Stewart]

Unfortunately I don't seem to have a Python3.framework directory on my machine.

Comment 12

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Seems to work for me!