./mach bootstrap fails if download cache is corrupted
Categories
(Firefox Build System :: Bootstrap Configuration, defect)
Tracking
(firefox-esr60 unaffected, firefox65 unaffected, firefox66 fixed, firefox67 fixed)
Tracking | Status | |
---|---|---|
firefox-esr60 | --- | unaffected |
firefox65 | --- | unaffected |
firefox66 | --- | fixed |
firefox67 | --- | fixed |
People
(Reporter: emk, Assigned: tomprince)
References
Details
(Keywords: regression)
Attachments
(1 file, 1 obsolete file)
$ ./mach bootstrap mach bootstrap is not fully implemented in MozillaBuild Note on Artifact Mode: Artifact builds download prebuilt C++ components rather than building them locally. Artifact builds are faster! Artifact builds are recommended for people working on Firefox or Firefox for Android frontends. They are unsuitable for those working on C++ code. For more information see: https://developer.mozilla.org/en-US/docs/Artifact_builds. Please choose the version of Firefox you want to build: 1. Firefox for Desktop Artifact Mode 2. Firefox for Desktop 3. Firefox for Android Artifact Mode 4. Firefox for Android Your choice: 2 Running pip to ensure Mercurial is up-to-date... Requirement already up-to-date: Mercurial in d:\mozilla-build\python\lib\site-packages (4.8.1) You are using pip version 10.0.1, however version 18.1 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. Your version of Python (2.7.15) is new enough. Your version of Rust (1.31.1) is new enough. Rust supports i686-pc-windows-msvc, x86_64-pc-windows-msvc targets. Mozilla recommends a number of changes to Mercurial to enhance your experience with it. Would you like to run a configuration wizard to ensure Mercurial is optimally configured? 1. Yes 2. No Please enter your reply: 2 0:20.98 Downloading clang.tar.bz2 0:20.98 Downloading to temporary location c:\Users\kimu\.mozbuild\toolchains\5a8e6758878eade2-clang.tar.bz2 0:20.98 Downloaded artifact to c:\Users\kimu\.mozbuild\toolchains\5a8e6758878eade2-clang.tar.bz2 0:24.23 rm tree: c:\Users\kimu\.mozbuild\clang 0:24.23 untarring "c:\Users\kimu\.mozbuild\clang.tar.bz2" Error running mach: ['artifact', 'toolchain', '--from-build', 'win64-clang-cl'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: IOError: invalid data stream File "e:\m\mozilla-unified\python/mozbuild/mozbuild/mach_commands.py", line 1563, in artifact_toolchain unpack_file(local, record.setup) File "e:\m\mozilla-unified\python/mozbuild\mozbuild\action\tooltool.py", line 533, in unpack_file tar.extractall() File "d:\mozilla-build\python\lib\tarfile.py", line 2081, in extractall self.extract(tarinfo, path) File "d:\mozilla-build\python\lib\tarfile.py", line 2118, in extract self._extract_member(tarinfo, os.path.join(path, tarinfo.name)) File "d:\mozilla-build\python\lib\tarfile.py", line 2194, in _extract_member self.makefile(tarinfo, targetpath) File "d:\mozilla-build\python\lib\tarfile.py", line 2235, in makefile copyfileobj(source, target) File "d:\mozilla-build\python\lib\tarfile.py", line 266, in copyfileobj shutil.copyfileobj(src, dst) File "d:\mozilla-build\python\lib\shutil.py", line 63, in copyfileobj buf = fsrc.read(length) File "d:\mozilla-build\python\lib\tarfile.py", line 831, in read buf += self.fileobj.read(size - len(buf)) File "d:\mozilla-build\python\lib\tarfile.py", line 743, in read return self.readnormal(size) File "d:\mozilla-build\python\lib\tarfile.py", line 758, in readnormal return self.__read(size) File "d:\mozilla-build\python\lib\tarfile.py", line 748, in __read buf = self.fileobj.read(size) Error running mach: ['bootstrap'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: CalledProcessError: Command '['d:\\mozilla-build\\python\\python2.7.exe', u'e:\\m\\mozilla-unified\\mach', u'artifact', u'toolchain', u'--from-build', u'win64-clang-cl']' returned non-zero exit status 1 File "e:\m\mozilla-unified\python/mozboot/mozboot/mach_commands.py", line 43, in bootstrap bootstrapper.bootstrap() File "e:\m\mozilla-unified\python/mozboot\mozboot\bootstrap.py", line 507, in bootstrap checkout_root) File "e:\m\mozilla-unified\python/mozboot\mozboot\bootstrap.py", line 383, in maybe_install_private_packages_or_exit self.instance.ensure_stylo_packages(state_dir, checkout_root) File "e:\m\mozilla-unified\python/mozboot\mozboot\mozillabuild.py", line 53, in ensure_stylo_packages self.install_toolchain_artifact(state_dir, checkout_root, stylo.WINDOWS_CLANG) File "e:\m\mozilla-unified\python/mozboot\mozboot\base.py", line 312, in install_toolchain_artifact subprocess.check_call(cmd, cwd=state_dir) File "d:\mozilla-build\python\lib\subprocess.py", line 190, in check_call raise CalledProcessError(retcode, cmd)
Reporter | ||
Comment 1•5 years ago
|
||
I can no longer build Firefox at all because clang-cl is not installed.
Reporter | ||
Comment 2•5 years ago
|
||
Manually extracting %userprofile%\.mozbuild\clang.tar.bz2 also failed (CRC error). is 5a8e6758878eade2-clang.tar.bz2 broken?
Reporter | ||
Comment 3•5 years ago
|
||
The revision is <https://hg.mozilla.org/mozilla-central/rev/0def5ac36b5b>.
Updated•5 years ago
|
Reporter | ||
Comment 4•5 years ago
•
|
||
Reporter | ||
Comment 5•5 years ago
|
||
Reverting bug 1512188 fixed the issue. Please backout. I'll backout by myself if nobody responds.
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Updated•5 years ago
|
Comment 6•5 years ago
|
||
I'm away so I'll leave this for @tomprince to look at.
Assignee | ||
Comment 7•5 years ago
|
||
I'm fairly certain that you have have a corrupt download at c:\Users\kimu.mozbuild\toolchains\5a8e6758878eade2-clang.tar.bz2 and removing it will cause it to be redownloaded an succeed. The artifact itself is being used in automation builds, so isn't corrupted.
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 8•5 years ago
|
||
Assignee | ||
Comment 9•5 years ago
|
||
I've attached a patch that should catch the corruption and attempt to download again.
Reporter | ||
Comment 10•5 years ago
|
||
Comment on attachment 9035003 [details] [diff] [review]
Backed out 8 changesets (bug 1512285, bug 1512188) for breaking ./mach bootstrap on Windows msys
(In reply to Tom Prince [:tomprince] from comment #7)
I'm fairly certain that you have have a corrupt download at c:\Users\kimu.mozbuild\toolchains\5a8e6758878eade2-clang.tar.bz2 and removing it will cause it to be redownloaded an succeed. The artifact itself is being used in automation builds, so isn't corrupted.
You are right, removing cache fixed the issue. Thank you!
Reporter | ||
Comment 11•5 years ago
•
|
||
Updated•5 years ago
|
Comment 12•5 years ago
|
||
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/e793c88c1fd8 Fix hash verification of downloaded toolchains; r=glandium
Comment 13•5 years ago
|
||
bugherder |
Updated•5 years ago
|
Comment 14•5 years ago
|
||
Not sure how many people are going to try to bootstrap off a release branch, but I could see it being possible for someone wanting to build 66 once it goes to release. Might as well uplift this a=NPOTB.
Comment 15•5 years ago
|
||
bugherder uplift |
Updated•5 years ago
|
Description
•