1519816 - Intermittent toolchains failure with snapshot.debian.org connection error

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Natalia Csoregi [:nataliaCs]]

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=toolchains%2Copt%2Ctoolchain%2Cbuilds%2Cfor%2Clinux%2C64-bits%2Ctoolchain-linux64-custom-v8%2Ctl%28custom-v8%29&selectedJob=221690246&revision=edca8877b0505cd1c31beaf6d907ca32e022aa52

Task details: https://tools.taskcluster.net/groups/CERj2ysQQmKhLGs5Av3sxg/tasks/GdMnO7CNRfCtrCsAMByjjA/runs/0

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=221690246&repo=mozilla-central

[task 2019-01-14T10:40:10.974Z] Err:7 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch InRelease
[task 2019-01-14T10:40:10.974Z] Could not connect to snapshot.debian.org:80 (2001:1af8:4020:b030:deb::185). - connect (101: Network is unreachable) [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:10.974Z] Err:8 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch-updates InRelease
[task 2019-01-14T10:40:10.974Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:10.974Z] Err:9 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch-backports InRelease
[task 2019-01-14T10:40:10.974Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:10.974Z] Err:10 http://snapshot.debian.org/archive/debian-security/20170830T000511Z stretch/updates InRelease
[task 2019-01-14T10:40:10.974Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:11.403Z] Reading package lists...
[task 2019-01-14T10:40:11.419Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch/InRelease Could not connect to snapshot.debian.org:80 (2001:1af8:4020:b030:deb::185). - connect (101: Network is unreachable) [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:11.419Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch-updates/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:11.419Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch-backports/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:11.419Z] W: Failed to fetch http://snapshot.debian.org/archive/debian-security/20170830T000511Z/dists/stretch/updates/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:1af8:4020:b030:deb::185 80]
[task 2019-01-14T10:40:11.419Z] W: Some index files failed to download. They have been ignored, or old ones used instead.
[task 2019-01-14T10:40:11.420Z] Finding missing packages...
[task 2019-01-14T10:40:12.000Z] The following command failed: sudo apt-get install --reinstall libasound2:i386 libcap2:i386 libelf-dev:i386 libfontconfig1:i386 libglib2.0-0:i386 libgpm2:i386 libncurses5:i386 libnss3:i386 libpango1.0-0:i386 libpci3:i386 libssl-dev:i386 libssl1.1:i386 libtinfo-dev:i386 libudev1:i386 libuuid1:i386 libx11-xcb1:i386 libxcomposite1:i386 libxcursor1:i386 libxdamage1:i386 libxi6:i386 libxrandr2:i386 libxss1:i386 libxtst6:i386 linux-libc-dev:i386 ant apache2-bin autoconf binutils binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-mips64el-linux-gnuabi64 binutils-mipsel-linux-gnu bison bzip2 cdbs cmake curl dbus-x11 devscripts dpkg-dev elfutils fakeroot flex g++ g++-arm-linux-gnueabihf g++-mingw-w64-i686 gawk git-core git-svn gperf intltool lib32ncurses5-dev lib32z1-dev libapache2-mod-php7.0 libappindicator3-1 libappindicator3-dev libasound2 libasound2-dev libatk1.0-0 libatspi2.0-0 libatspi2.0-dev libav-tools libbluetooth-dev libbrlapi-dev libbrlapi0.6 libbz2-1.0 libbz2-dev libc6 libc6-dev-armhf-cross libcairo2 libcairo2-dev libcap-dev libcap2 libcups2 libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexpat1 libffi-dev libffi6 libfontconfig1 libfreetype6 libgbm-dev libglib2.0-0 libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgnome-keyring0 libgtk-3-0 libgtk-3-dev libjpeg-dev libkrb5-dev libnspr4 libnspr4-dev libnss3 libnss3-dev libpam0g libpam0g-dev libpango1.0-0 libpci-dev libpci3 libpcre3 libpixman-1-0 libpng16-16 libpulse-dev libpulse0 libsctp-dev libspeechd-dev libspeechd2 libsqlite3-0 libsqlite3-dev libssl-dev libstdc++6 libtinfo-dev libtool libudev-dev libudev1 libuuid1 libwayland-egl1-mesa libwww-perl libx11-6 libx11-xcb1 libxau6 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 libxfixes3 libxi6 libxinerama1 libxkbcommon-dev libxrandr2 libxrender1 libxslt1-dev libxss-dev libxt-dev libxtst-dev libxtst6 linux-libc-dev-armhf-cross locales openbox p7zip patch perl php7.0-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-numpy python-opencv python-openssl python-psutil python-yaml realpath rpm ruby subversion texinfo uuid-dev wdiff x11-utils xcompmgr xsltproc xutils-dev xvfb xz-utils zip zlib1g
[task 2019-01-14T10:40:12.000Z]
[task 2019-01-14T10:40:12.000Z] It produces the following output:
[task 2019-01-14T10:40:12.427Z] Reading package lists...
[task 2019-01-14T10:40:12.512Z] Building dependency tree...
[task 2019-01-14T10:40:12.513Z] Reading state information...
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libasound2:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libcap2:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libelf-dev:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libfontconfig1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libglib2.0-0:i386
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by glob 'libglib2.0-0'
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by regex 'libglib2.0-0'
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libgpm2:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libncurses5:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libnss3:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libpango1.0-0:i386
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by glob 'libpango1.0-0'
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by regex 'libpango1.0-0'
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libpci3:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libssl-dev:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libssl1.1:i386
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by glob 'libssl1.1'
[task 2019-01-14T10:40:12.577Z] E: Couldn't find any package by regex 'libssl1.1'
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libtinfo-dev:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libudev1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libuuid1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libx11-xcb1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxcomposite1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxcursor1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxdamage1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxi6:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxrandr2:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxss1:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package libxtst6:i386
[task 2019-01-14T10:40:12.577Z] E: Unable to locate package linux-libc-dev:i386
[task 2019-01-14T10:40:12.578Z]
[task 2019-01-14T10:40:12.578Z] You will have to install the above packages yourself.
[task 2019-01-14T10:40:12.578Z]
[fetches 2019-01-14T10:40:12.579Z] removing /builds/worker/workspace/build
[fetches 2019-01-14T10:40:14.554Z] finished
[taskcluster 2019-01-14 10:40:14.917Z] === Task Finished ===
[taskcluster 2019-01-14 10:40:15.034Z] Artifact "public/build" not found at "/builds/worker/artifacts/"
[taskcluster 2019-01-14 10:40:15.764Z] Unsuccessful task run with exit code: 100 completed in 539.035 seconds

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Perhaps snapshot has added anoter rando-IP-block to their blacklist, similar to bug 1486071. But it looks like this is a toolchain task, rather than a docker-build task. Should we retry those, too?

Comment 2

[Mike Hommey [:glandium]]

Toolchain jobs shouldn't be installing packages.

Comment 3

[Mike Hommey [:glandium]]

Can we move the dependencies install to the docker image?

Comment 4

[Greg Mierzwinski [:sparky]]

Yes, we would have to move the following into the docker image: link1

With that change, we would also need to rebuild the docker image any time we rerun the custom-v8 task by updating this function: link2

I don't see anything obvious in the dependencies script link3 that would require us to also run it in the v8 build task so there shouldn't be any issues if it's completely removed from link1.

Comment 5

[Mike Hommey [:glandium]]

I don't think you need to use the v8 code to install its dependencies, nor to rebuild the docker image every time you build v8. Presumably, the dependencies don't change often, and when they do change, you can update the packages you install in the docker image.

Comment 6

[Intermittent Failures Robot]

1 failures in 2378 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• toolchains: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1519816&startday=2019-01-14&endday=2019-01-20&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 2383 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• toolchains: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1519816&startday=2019-01-21&endday=2019-01-27&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 2628 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• toolchains: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1519816&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 9

[Cristina Coroiu [:ccoroiu]]

This fail appeared again and it's perma failing:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=toolchains%2Copt%2Ctoolchain%2Cbuilds%2Cfor%2Clinux%2C64-bits%2Ctoolchain-linux64-custom-v8%2Ctl%28custom-v8%29&revision=7ab4a0c9980f2e89cd9af2bda3cdf656d5c1b0a1&selectedJob=228813084

Failure log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=228813084&repo=mozilla-central&lineNumber=542

[task 2019-02-16T10:36:44.700Z] ./build/install-build-deps.sh: line 198: file: command not found
[task 2019-02-16T10:36:56.765Z] ./build/install-build-deps.sh: line 490: file: command not found
[task 2019-02-16T10:36:56.766Z] Skipping debugging symbols.
[task 2019-02-16T10:36:56.767Z] Including 32-bit libraries.
[task 2019-02-16T10:36:56.767Z] Including ARM cross toolchain.
[task 2019-02-16T10:36:56.767Z] Including NaCl, NaCl toolchain, NaCl ports dependencies.
[task 2019-02-16T10:36:57.487Z] Ign:1 https://queue.taskcluster.net/v1/task/KnWgLU-XQICX9LWYch_PrQ/artifacts/public/build debian/ InRelease
[task 2019-02-16T10:36:57.784Z] Ign:2 https://queue.taskcluster.net/v1/task/Le8mJpxFR3Wey5WxVTotWA/artifacts/public/build debian/ InRelease
[task 2019-02-16T10:36:58.012Z] Ign:3 https://queue.taskcluster.net/v1/task/KnWgLU-XQICX9LWYch_PrQ/artifacts/public/build debian/ Release
[task 2019-02-16T10:36:58.236Z] Ign:4 https://queue.taskcluster.net/v1/task/Le8mJpxFR3Wey5WxVTotWA/artifacts/public/build debian/ Release
[task 2019-02-16T10:36:58.807Z] Hit:5 https://s3.us-west-2.amazonaws.com/taskcluster-public-artifacts/KnWgLU-XQICX9LWYch_PrQ/0/public/build debian/ Packages
[task 2019-02-16T10:36:58.886Z] Hit:6 https://s3.us-west-2.amazonaws.com/taskcluster-public-artifacts/Le8mJpxFR3Wey5WxVTotWA/0/public/build debian/ Packages
[task 2019-02-16T10:40:59.276Z] Err:7 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch InRelease
[task 2019-02-16T10:40:59.276Z] Could not connect to snapshot.debian.org:80 (2001:630:206:4000:1a1a:0:c13e:ca1b). - connect (101: Network is unreachable) [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.276Z] Err:8 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch-updates InRelease
[task 2019-02-16T10:40:59.276Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.276Z] Err:9 http://snapshot.debian.org/archive/debian/20170830T000511Z stretch-backports InRelease
[task 2019-02-16T10:40:59.276Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.276Z] Err:10 http://snapshot.debian.org/archive/debian-security/20170830T000511Z stretch/updates InRelease
[task 2019-02-16T10:40:59.276Z] Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.747Z] Reading package lists...
[task 2019-02-16T10:40:59.763Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch/InRelease Could not connect to snapshot.debian.org:80 (2001:630:206:4000:1a1a:0:c13e:ca1b). - connect (101: Network is unreachable) [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.763Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch-updates/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.763Z] W: Failed to fetch http://snapshot.debian.org/archive/debian/20170830T000511Z/dists/stretch-backports/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.763Z] W: Failed to fetch http://snapshot.debian.org/archive/debian-security/20170830T000511Z/dists/stretch/updates/InRelease Unable to connect to snapshot.debian.org:http: [IP: 2001:630:206:4000:1a1a:0:c13e:ca1b 80]
[task 2019-02-16T10:40:59.763Z] W: Some index files failed to download. They have been ignored, or old ones used instead.
[task 2019-02-16T10:40:59.764Z] Finding missing packages...
[task 2019-02-16T10:40:59.764Z] Packages required: libasound2:i386 libcap2:i386 libelf-dev:i386 libfontconfig1:i386 libglib2.0-0:i386 libgpm2:i386 libncurses5:i386 libnss3:i386 libpango1.0-0:i386 libpci3:i386 libssl-dev:i386 libssl1.1:i386 libtinfo-dev:i386 libudev1:i386 libuuid1:i386 libx11-xcb1:i386 libxcomposite1:i386 libxcursor1:i386 libxdamage1:i386 libxi6:i386 libxrandr2:i386 libxss1:i386 libxtst6:i386 linux-libc-dev:i386 ant apache2-bin autoconf binutils binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-mips64el-linux-gnuabi64 binutils-mipsel-linux-gnu bison bzip2 cdbs cmake curl dbus-x11 devscripts dpkg-dev elfutils fakeroot flex g++ g++-arm-linux-gnueabihf g++-mingw-w64-i686 gawk git-core git-svn gperf intltool lib32ncurses5-dev lib32z1-dev libapache2-mod-php7.0 libappindicator3-1 libappindicator3-dev libasound2 libasound2-dev libatk1.0-0 libatspi2.0-0 libatspi2.0-dev libav-tools libbluetooth-dev libbrlapi-dev libbrlapi0.6 libbz2-1.0 libbz2-dev libc6 libc6-dev-armhf-cross libcairo2 libcairo2-dev libcap-dev libcap2 libcups2 libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexpat1 libffi-dev libffi6 libfontconfig1 libfreetype6 libgbm-dev libglib2.0-0 libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgnome-keyring0 libgtk-3-0 libgtk-3-dev libjpeg-dev libkrb5-dev libnspr4 libnspr4-dev libnss3 libnss3-dev libpam0g libpam0g-dev libpango1.0-0 libpci-dev libpci3 libpcre3 libpixman-1-0 libpng16-16 libpulse-dev libpulse0 libsctp-dev libspeechd-dev libspeechd2 libsqlite3-0 libsqlite3-dev libssl-dev libstdc++6 libtinfo-dev libtool libudev-dev libudev1 libuuid1 libwayland-egl1-mesa libwww-perl libx11-6 libx11-xcb1 libxau6 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 libxfixes3 libxi6 libxinerama1 libxkbcommon-dev libxrandr2 libxrender1 libxslt1-dev libxss-dev libxt-dev libxtst-dev libxtst6 linux-libc-dev-armhf-cross locales openbox p7zip patch perl php7.0-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-numpy python-opencv python-openssl python-psutil python-yaml realpath rpm ruby subversion texinfo uuid-dev wdiff x11-utils xcompmgr xsltproc xutils-dev xvfb xz-utils zip zlib1g
[task 2019-02-16T10:40:59.764Z]
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libasound2:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libcap2:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libelf-dev:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libfontconfig1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libglib2.0-0:i386
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by glob 'libglib2.0-0'
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by regex 'libglib2.0-0'
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libgpm2:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libncurses5:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libnss3:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libpango1.0-0:i386
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by glob 'libpango1.0-0'
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by regex 'libpango1.0-0'
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libpci3:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libssl-dev:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libssl1.1:i386
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by glob 'libssl1.1'
[task 2019-02-16T10:41:00.398Z] E: Couldn't find any package by regex 'libssl1.1'
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libtinfo-dev:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libudev1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libuuid1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libx11-xcb1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxcomposite1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxcursor1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxdamage1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxi6:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxrandr2:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxss1:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package libxtst6:i386
[task 2019-02-16T10:41:00.398Z] E: Unable to locate package linux-libc-dev:i386
[task 2019-02-16T10:41:00.399Z] The following command failed: sudo apt-get install --reinstall libasound2:i386 libcap2:i386 libelf-dev:i386 libfontconfig1:i386 libglib2.0-0:i386 libgpm2:i386 libncurses5:i386 libnss3:i386 libpango1.0-0:i386 libpci3:i386 libssl-dev:i386 libssl1.1:i386 libtinfo-dev:i386 libudev1:i386 libuuid1:i386 libx11-xcb1:i386 libxcomposite1:i386 libxcursor1:i386 libxdamage1:i386 libxi6:i386 libxrandr2:i386 libxss1:i386 libxtst6:i386 linux-libc-dev:i386 ant apache2-bin autoconf binutils binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-mips64el-linux-gnuabi64 binutils-mipsel-linux-gnu bison bzip2 cdbs cmake curl dbus-x11 devscripts dpkg-dev elfutils fakeroot flex g++ g++-arm-linux-gnueabihf g++-mingw-w64-i686 gawk git-core git-svn gperf intltool lib32ncurses5-dev lib32z1-dev libapache2-mod-php7.0 libappindicator3-1 libappindicator3-dev libasound2 libasound2-dev libatk1.0-0 libatspi2.0-0 libatspi2.0-dev libav-tools libbluetooth-dev libbrlapi-dev libbrlapi0.6 libbz2-1.0 libbz2-dev libc6 libc6-dev-armhf-cross libcairo2 libcairo2-dev libcap-dev libcap2 libcups2 libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexpat1 libffi-dev libffi6 libfontconfig1 libfreetype6 libgbm-dev libglib2.0-0 libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgnome-keyring0 libgtk-3-0 libgtk-3-dev libjpeg-dev libkrb5-dev libnspr4 libnspr4-dev libnss3 libnss3-dev libpam0g libpam0g-dev libpango1.0-0 libpci-dev libpci3 libpcre3 libpixman-1-0 libpng16-16 libpulse-dev libpulse0 libsctp-dev libspeechd-dev libspeechd2 libsqlite3-0 libsqlite3-dev libssl-dev libstdc++6 libtinfo-dev libtool libudev-dev libudev1 libuuid1 libwayland-egl1-mesa libwww-perl libx11-6 libx11-xcb1 libxau6 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 libxfixes3 libxi6 libxinerama1 libxkbcommon-dev libxrandr2 libxrender1 libxslt1-dev libxss-dev libxt-dev libxtst-dev libxtst6 linux-libc-dev-armhf-cross locales openbox p7zip patch perl php7.0-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-numpy python-opencv python-openssl python-psutil python-yaml realpath rpm ruby subversion texinfo uuid-dev wdiff x11-utils xcompmgr xsltproc xutils-dev xvfb xz-utils zip zlib1g
[task 2019-02-16T10:41:00.399Z]
task 2019-02-16T10:41:00.399Z] It produces the following output:
[task 2019-02-16T10:41:00.859Z] Reading package lists...
[task 2019-02-16T10:41:00.961Z] Building dependency tree...
[task 2019-02-16T10:41:00.961Z] Reading state information...
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libasound2:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libcap2:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libelf-dev:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libfontconfig1:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libglib2.0-0:i386
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by glob 'libglib2.0-0'
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by regex 'libglib2.0-0'
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libgpm2:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libncurses5:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libnss3:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libpango1.0-0:i386
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by glob 'libpango1.0-0'
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by regex 'libpango1.0-0'
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libpci3:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libssl-dev:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libssl1.1:i386
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by glob 'libssl1.1'
[task 2019-02-16T10:41:01.035Z] E: Couldn't find any package by regex 'libssl1.1'
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libtinfo-dev:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libudev1:i386
[task 2019-02-16T10:41:01.035Z] E: Unable to locate package libuuid1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libx11-xcb1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxcomposite1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxcursor1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxdamage1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxi6:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxrandr2:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxss1:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package libxtst6:i386
[task 2019-02-16T10:41:01.036Z] E: Unable to locate package linux-libc-dev:i386
[task 2019-02-16T10:41:01.037Z]
[task 2019-02-16T10:41:01.037Z] You will have to install the above packages yourself.
[task 2019-02-16T10:41:01.037Z]
[fetches 2019-02-16T10:41:01.037Z] removing /builds/worker/workspace/build
[fetches 2019-02-16T10:41:02.931Z] finished
[taskcluster 2019-02-16 10:41:04.805Z] === Task Finished ===
[taskcluster 2019-02-16 10:41:04.896Z] Artifact "public/build" not found at "/builds/worker/artifacts/"
[taskcluster 2019-02-16 10:41:05.262Z] Unsuccessful task run with exit code: 100 completed in 535.928 seconds

Mike, can you please take a look?

Comment 10

[Mike Hommey [:glandium]]

Not perma red just yet... just needed another nudge.

Comment 11

[Mike Hommey [:glandium]]

Attachment: Bug 1519816 - Simplify build-custom-v8 script and avoid installing packages during build.

The build script currently is doing some unnecessary steps:

• Running gclient only prints out an help message. It is a step
  indicated in various documentations, but is only necessary to keep
  depot-tools up-to-date, which they are, since we just cloned it.
• The fetch v8 command creates a v8 directory, no need to create
  another layer.
• gclient sync is run as part of fetch. Same as gclient, this step
  is only given in documentations to keep things up-to-date on an existing
  clone, but we just freshly got one.
• Same goes for git pull && gclient sync
• git checkout master is not necessary, as fetch gets us there
  already (albeit, in a detached head state)
• install-build-deps.sh installs build dependencies for chrome or
  whatever. That's way too much for v8, that barely needs pkg-config and
  glib, which we now install in the docker image.

Comment 12

[Intermittent Failures Robot]

5 failures in 2832 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1
• mozilla-central: 4

Platform breakdown:

• toolchains: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1519816&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 13

[Greg Mierzwinski [:sparky]]

Here's a try run that uses this patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e6679fffc91a5cd2bed4f9442a2bd8519a156a51

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c0dc998686a9
Simplify build-custom-v8 script and avoid installing packages during build. r=sparky

Comment 15

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/c0dc998686a9