1660790 - `./mach bootstrap` fails with "ModuleNotFoundError: No module named 'zstandard'"

Status: RESOLVED DUPLICATE of bug 1660559

(Firefox Build System :: General, defect)

Description

[Jonathan Watt [:jwatt]]

Trying to run ./mach bootstrap this morning on Ubuntu 20.04:

Error running mach:
 
    ['artifact', 'toolchain', '--bootstrap', '--from-build', 'linux64-clang']
 
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file artifact| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
 
If filing a bug, please include the full output of mach, including this error
message.
 
The details of the failure are as follows:
 
ModuleNotFoundError: No module named 'zstandard'
 
  File "/home/jwatt/mozilla/trees/u/python/mozbuild/mozbuild/artifact_commands.py", line 465, in artifact_toolchain
    unpack_file(local)
  File "/home/jwatt/mozilla/trees/u/python/mozbuild/mozbuild/action/tooltool.py", line 918, in unpack_file
    import zstandard
  File "/home/jwatt/mozilla/trees/u/build/mach_bootstrap.py", line 474, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
Sentry is attempting to send 0 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit
Error running mach:
 
    ['bootstrap']
 
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file bootstrap| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
 
If filing a bug, please include the full output of mach, including this error
message.
 
The details of the failure are as follows:
 
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/home/jwatt/mozilla/trees/u/mach', 'artifact', 'toolchain', '--bootstrap', '--from-build', 'linux64-clang']' returned non-zero exit status 1.
 
  File "/home/jwatt/mozilla/trees/u/python/mozboot/mozboot/mach_commands.py", line 45, in bootstrap
    bootstrapper.bootstrap()
  File "/home/jwatt/mozilla/trees/u/python/mozboot/mozboot/bootstrap.py", line 575, in bootstrap
    self.maybe_install_private_packages_or_exit(state_dir,
  File "/home/jwatt/mozilla/trees/u/python/mozboot/mozboot/bootstrap.py", line 425, in maybe_install_private_packages_or_exit
    self.instance.ensure_stylo_packages(state_dir, checkout_root)
  File "/home/jwatt/mozilla/trees/u/python/mozboot/mozboot/linux_common.py", line 73, in ensure_stylo_packages
    self.install_toolchain_artifact(state_dir, checkout_root, stylo.LINUX_CLANG)
  File "/home/jwatt/mozilla/trees/u/python/mozboot/mozboot/base.py", line 356, in install_toolchain_artifact
    subprocess.check_call(cmd, cwd=state_dir)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
Sentry is attempting to send 0 pending error messages

Comment 1

[Daniel Holbert [:dholbert]]

I'm hitting this issue as well (also on Ubuntu 20.04).

Superficially, the error seems to be suggesting that zstandard isn't found; but oddly, just before this error output, I see the following (in particular "Requirement already satisfied: zstandard"):

Setting up artifact clang.tar.zst
Using artifact from local cache: /scratch/work/builds/.mozbuild/toolchains/d3619a19a060bce0-clang.tar.zst
Requirement already satisfied: zstandard<=0.13.0,>=0.9.0 in /scratch/work/builds/.mozbuild/_virtualenvs/mach/lib/python3.8/site-packages (0.13.0)
Requirement already satisfied: cffi>=1.11 in /scratch/work/builds/.mozbuild/_virtualenvs/mach/lib/python3.8/site-packages (from zstandard<=0.13.0,>=0.9.0) (1.14.2)
Requirement already satisfied: pycparser in /scratch/work/builds/.mozbuild/_virtualenvs/mach/lib/python3.8/site-packages (from cffi>=1.11->zstandard<=0.13.0,>=0.9.0) (2.20)

Comment 2

[Daniel Holbert [:dholbert]]

I seem to have worked around this locally, possibly by installing the zstd package in my .mozbuild/_virtualenvs/mach directory, though I'm not entirely sure if that's what did it.

I don't know if these steps are advisable, but for the record, the steps that may've worked around this for me were:

cd /path/to/.mozbuild/_virtualenvs/mach
source ./bin/activate 
pip install zstd

After that, I was able to get through a bootstrap and I've been unable to reproduce the issue; including after deleting this virtualenv and recreating it from scratch (without the zstd package) via ./mach create-mach-environment

Comment 3

[Ricky Stewart]

Bug 1660559 is tracking this, and the patch there should fix this permanently.

There are a few ways to work around this. Deleting ~/.mozbuild/_virtualenvs wholesale before running bootstrap is, I think, the easiest. pip install zstandard in your global Python environment is another easy workaround in the meantime while we wait for the patch to land.