Open Bug 1534560 Opened 6 years ago Updated 2 years ago

mach bootstrap can't download clang (anymore?)

Categories

(Firefox Build System :: Bootstrap Configuration, defect)

defect

Tracking

(Not tracked)

People

(Reporter: Gijs, Unassigned)

Details

My internet connection is fine, but somehow downloading clang is impossible according to mach bootstrap (re-untarring node every time I run it is also not very helpful):

$ ./mach bootstrap
mach bootstrap is not fully implemented in MozillaBuild

Note on Artifact Mode:

Artifact builds download prebuilt C++ components rather than building
them locally. Artifact builds are faster!

Artifact builds are recommended for people working on Firefox or
Firefox for Android frontends, or the GeckoView Java API. They are unsuitable
for those working on C++ code. For more information see:
https://developer.mozilla.org/en-US/docs/Artifact_builds.

Please choose the version of Firefox you want to build:
1. Firefox for Desktop Artifact Mode
2. Firefox for Desktop
3. GeckoView/Firefox for Android Artifact Mode
4. GeckoView/Firefox for Android
Your choice: 2
Running pip to ensure Mercurial is up-to-date...
Requirement already up-to-date: Mercurial in d:\mozilla-build\python\lib\site-packages (4.9)
You are using pip version 18.1, however version 19.0.3 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Your version of Python (2.7.15) is new enough.
Your version of Rust (1.32.0) is new enough.
Rust supports i686-pc-windows-msvc, x86_64-pc-windows-msvc targets.

Mozilla recommends a number of changes to Mercurial to enhance your
experience with it.

Would you like to run a configuration wizard to ensure Mercurial is
optimally configured?

  1. Yes
  2. No

Please enter your reply: 2
 0:04.76 Downloading node.tar.bz2
 0:04.77 Downloading to temporary location c:\Users\Gijs Kruitbosch\.mozbuild\toolchains\a6b3c45c1168c62b-node.tar.bz2
 0:04.77 Downloaded artifact to c:\Users\Gijs Kruitbosch\.mozbuild\toolchains\a6b3c45c1168c62b-node.tar.bz2
 0:04.84 rm tree: c:\Users\Gijs Kruitbosch\.mozbuild\node
 0:05.61 untarring "c:\Users\Gijs Kruitbosch\.mozbuild\node.tar.bz2"
 0:04.36 Downloading clang.tar.bz2
 0:04.36 Downloading to temporary location c:\Users\Gijs Kruitbosch\.mozbuild\toolchains\4fca7e3684ceb7f4-clang.tar.bz2
 0:05.44 Downloading... 0.0 %
 0:09.29 Downloading... 5.0 %
 0:12.66 Downloading... 10.0 %
 0:31.13 Downloading... 15.0 %
 0:49.28 Downloading... 20.0 %
Error running mach:

    ['artifact', 'toolchain', '--from-build', 'win64-clang-cl']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:1941)

  File "d:\mozilla-central\python/mozbuild/mozbuild/mach_commands.py", line 1503, in artifact_toolchain
    record.fetch_with(cache)
  File "d:\mozilla-central\python/mozbuild/mozbuild/mach_commands.py", line 1371, in fetch_with
    self.filename = cache.fetch(self.url)
  File "d:\mozilla-central\python/mozbuild\mozbuild\artifacts.py", line 864, in fetch
    dl.wait()
  File "d:\mozilla-central\third_party/python/dlmanager\dlmanager\manager.py", line 101, in wait
    self.raise_if_error()
  File "d:\mozilla-central\third_party/python/dlmanager\dlmanager\manager.py", line 116, in raise_if_error
    six.reraise(*self.__error)
  File "d:\mozilla-central\third_party/python/dlmanager\dlmanager\manager.py", line 168, in _download
    for chunk in response.iter_content(chunk_size):
  File "d:\mozilla-central\third_party/python/requests\requests\models.py", line 660, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "d:\mozilla-central\third_party/python/requests\requests\packages\urllib3\response.py", line 344, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "d:\mozilla-central\third_party/python/requests\requests\packages\urllib3\response.py", line 301, in read
    data = self._fp.read(amt)
  File "d:\mozilla-build\python\lib\httplib.py", line 597, in read
    s = self.fp.read(amt)
  File "d:\mozilla-build\python\lib\socket.py", line 384, in read
    data = self._sock.recv(left)
  File "d:\mozilla-build\python\lib\ssl.py", line 772, in recv
    return self.read(buflen)
  File "d:\mozilla-build\python\lib\ssl.py", line 659, in read
    v = self._sslobj.read(len)
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

CalledProcessError: Command '['d:\\mozilla-build\\python\\python2.7.exe', u'd:\\mozilla-central\\mach', u'artifact', u'toolchain', u'--from-build', u'win64-clang-cl']' returned non-zero exit status 1

  File "d:\mozilla-central\python/mozboot/mozboot/mach_commands.py", line 43, in bootstrap
    bootstrapper.bootstrap()
  File "d:\mozilla-central\python/mozboot\mozboot\bootstrap.py", line 514, in bootstrap
    checkout_root)
  File "d:\mozilla-central\python/mozboot\mozboot\bootstrap.py", line 386, in maybe_install_private_packages_or_exit
    self.instance.ensure_stylo_packages(state_dir, checkout_root)
  File "d:\mozilla-central\python/mozboot\mozboot\mozillabuild.py", line 89, in ensure_stylo_packages
    self.install_toolchain_artifact(state_dir, checkout_root, stylo.WINDOWS_CLANG)
  File "d:\mozilla-central\python/mozboot\mozboot\base.py", line 307, in install_toolchain_artifact
    subprocess.check_call(cmd, cwd=state_dir)
  File "d:\mozilla-build\python\lib\subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)

We should

  1. indicate where we're downloading from and/or how to complete the steps manually
  2. not throw this level of errors, and ideally support download resume
  3. check that the python ssl stuff supports whatever our server is doing, and ensure the downloads are coming from a saner server / enforce a better client library if not.
$ python --version
Python 2.7.15

With apologies for the inflammatory summary. :-(

Summary: mach bootstrap sucks at downloading files, apparently → mach bootstrap can't download clang (anymore?)
Component: General → Bootstrap Configuration
Product: Developer Services → Firefox Build System
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.