Closed Bug 1636016 Opened 5 years ago Closed 5 years ago

bootstrap failed after upgrading to python 3

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: jlast, Unassigned)

Details

it looks like running ./mach bootstrap failed due to a python depenency problem

==> python
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> python@3.8
Python has been installed as
  /usr/local/opt/python@3.8/bin/python3

You can install Python packages with
  /usr/local/opt/python@3.8/bin/pip3 install <package>
They will install into the site-package directory
  /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages

See: https://docs.brew.sh/Homebrew-and-Python

python@3.8 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have python@3.8 first in your PATH run:
  echo 'export PATH="/usr/local/opt/python@3.8/bin:$PATH"' >> ~/.zshrc

For compilers to find python@3.8 you may need to set:
  export LDFLAGS="-L/usr/local/opt/python@3.8/lib"

For pkg-config to find python@3.8 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/python@3.8/lib/pkgconfig"

subprocess.CalledProcessError: Command '['/usr/local/bin/hg', '--version']' returned non-zero exit status 1.

  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/mach_commands.py", line 44, in bootstrap
    bootstrapper.bootstrap()
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/bootstrap.py", line 453, in bootstrap
    self.instance.install_system_packages()
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/osx.py", line 188, in install_system_packages
    _, hg_modern, _ = self.is_mercurial_modern()
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/base.py", line 552, in is_mercurial_modern
    our = self._parse_version(hg, 'version', self._hg_cleanenv())
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/base.py", line 524, in _parse_version
    return self._parse_version_impl(path, name, env, "--version")
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/base.py", line 515, in _parse_version_impl
    universal_newlines=True)
  File "/Users/jlaster/src/moz/gecko-dev/python/mozboot/mozboot/base.py", line 443, in check_output
    return subprocess.check_output(*args, **kwargs)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)

The error might not be great, but what it's telling you is that mercurial is broken:

subprocess.CalledProcessError: Command '['/usr/local/bin/hg', '--version']' returned non-zero exit status 1.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID

(BTW, it seems mach itself is using python 3.7, not 3.8)

Yep, i'm attributing this to a python mismatch, when I run hg --version i get this

ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha512
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 43, in <module>
    dispatch.run()
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 150, in __getattr__
    self._load()
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 94, in _load
    _origimport, head, globals, locals, None, level)
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 43, in _hgextimport
    return importfunc(name, globals, *args, **kwargs)
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/mercurial/dispatch.py", line 625, in <module>
    class lazyaliasentry(object):
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/mercurial/dispatch.py", line 636, in lazyaliasentry
    @util.propertycache
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 150, in __getattr__
    self._load()
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 94, in _load
    _origimport, head, globals, locals, None, level)
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 43, in _hgextimport
    return importfunc(name, globals, *args, **kwargs)
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/mercurial/util.py", line 184, in <module>
    'md5': hashlib.md5,
  File "/usr/local/Cellar/mercurial/5.0.2/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 151, in __getattr__
    return getattr(self._module, attr)
AttributeError: 'module' object has no attribute 'md5'
No longer blocks: mach-busted

I get this error when I run ./mach build which also looks python related

 0:02.92 IndentationError: unexpected indent
 0:02.92 Traceback (most recent call last):
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/configure.py", line 181, in <module>
 0:02.92     sys.exit(main(sys.argv))
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/configure.py", line 52, in main
 0:02.92     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/configure/__init__.py", line 498, in run
 0:02.92     func(*args)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/configure/__init__.py", line 542, in _value_for
 0:02.92     return self._value_for_depends(obj)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/util.py", line 1018, in method_call
 0:02.92     cache[args] = self.func(instance, *args)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/configure/__init__.py", line 551, in _value_for_depends
 0:02.92     value = obj.result()
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/util.py", line 1018, in method_call
 0:02.92     cache[args] = self.func(instance, *args)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
 0:02.92     return self._func(*resolved_args)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/configure/__init__.py", line 1161, in wrapped
 0:02.92     return new_func(*args, **kwargs)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/build/moz.configure/init.configure", line 556, in virtualenv_python2
 0:02.92     manager.build(python)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/virtualenv.py", line 492, in build
 0:02.92     self.create(python)
 0:02.92   File "/Users/jlaster/src/moz/gecko-dev/python/mozbuild/mozbuild/virtualenv.py", line 259, in create
 0:02.92     self.virtualenv_root, result))
 0:02.92 Exception: Failed to create virtualenv: /Users/jlaster/src/moz/gecko-dev/objdir-frontend-artifacts/_virtualenvs/init (virtualenv.py retcode: 1)
 0:03.02 *** Fix above errors and then restart with               "./mach build"
 0:03.02 make: *** [configure] Error 1

Can you attach the full unabridged log? Evidently what's at fault is the IndentationError, but from what you've pasted it's impossible to tell what the source of the IndentationError is. virtualenv.py passes lint so I assume it's something in the subprocess.

Flags: needinfo?(jlaster)

This was fixed by uninstalling and re-installing python@2

brew uninstall python@2
brew install python@2

big thanks to glandium, who was really helpful on slack.

Flags: needinfo?(jlaster)
You need to log in before you can comment on or make changes to this bug.