1233054 - [opt] Luciddream jobs failing in desktop-test container to load libfreetype.so.6

Status: RESOLVED WONTFIX

(Testing :: General, defect)

Description

[Armen [:armenzg]]

From log [1]:
https://public-artifacts.taskcluster.net/f8W5SORxQYyNik-09fN9wg/0/public/logs/live_backing.log

From [2]
> make USE_LOCAL_XULRUNNER_SDK=1 XULRUNNER_DIRECTORY=xulrunner-sdk DESKTOP=0 DESKTOP_SHIMS=1 DEBUG=0 NOFTU=0
> /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory

It seems the file is under /usr/lib/x86_64-linux-gnu/libfreetype.so.6 [3]
In my host it seems that I have the file under here as well:
/usr/lib/i386-linux-gnu/libfreetype.so.6

I'm going to install and see if it works.

STR:
chown worker /home/worker
su worker
mkdir workspace && cd workspace
curl --fail -o mozharness.zip --retry 10 -L https://queue.taskcluster.net/v1/task/E7qIavdBR1-P_T8iTv7CKA/artifacts/public/build/mozharness.zip
unzip mozharness.zip
python2.7 /home/worker/workspace/mozharness/scripts/luciddream_unittest.py --config-file mozharness/configs/luciddream/linux_config.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/E7qIavdBR1-P_T8iTv7CKA/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/E7qIavdBR1-P_T8iTv7CKA/artifacts/public/build/test_packages.json


########## [2]
02:26:09     INFO - Downloading https://api.pub.build.mozilla.org/tooltool/sha512/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462 to /home/worker/workspace/gaia/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462
02:26:09     INFO - retry: Calling _download_file with args: (), kwargs: {'url': 'https://api.pub.build.mozilla.org/tooltool/sha512/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462', 'file_name': '/home/worker/workspace/gaia/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462'}, attempt #1
02:26:14     INFO - Downloaded 38304247 bytes.
02:26:14     INFO - mkdir: /home/worker/workspace/gaia/xulrunner-sdk
02:26:14     INFO - Running command: ['unzip', '-q', '-o', '/home/worker/workspace/gaia/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462'] in /home/worker/workspace/gaia/xulrunner-sdk
02:26:14     INFO - Copy/paste: unzip -q -o /home/worker/workspace/gaia/dc9503b21c87b5a469118746f99e4f41d73888972ce735fa10a80f6d218086da0e3da525d9a4cd8e4ea497ec199fef720e4a525873d77a1af304ac505e076462
02:26:15     INFO - Return code: 0
02:26:15     INFO - Sending environment as make vars because of bug 1028816
02:26:15     INFO - Running command: ['make', 'USE_LOCAL_XULRUNNER_SDK=1', 'XULRUNNER_DIRECTORY=xulrunner-sdk', 'DESKTOP=0', 'DESKTOP_SHIMS=1', 'DEBUG=0', 'NOFTU=0'] in /home/worker/workspace/gaia
02:26:15     INFO - Copy/paste: make USE_LOCAL_XULRUNNER_SDK=1 XULRUNNER_DIRECTORY=xulrunner-sdk DESKTOP=0 DESKTOP_SHIMS=1 DEBUG=0 NOFTU=0
02:26:15     INFO -  /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory
02:26:15     INFO -  /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory
02:26:15     INFO -  Test SDK directory: xulrunner-sdk
02:26:15     INFO -  test -f /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell
02:26:15     INFO -  GAIA_DIR="/home/worker/workspace/gaia" GAIA_DISTRIBUTION_DIR="/home/worker/workspace/gaia/distribution" LD_LIBRARY_PATH="/home/worker/workspace/gaia/xulrunner-sdk/bin/" /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell -f "/home/worker/workspace/gaia/build/xpcshell-commonjs.js" -e "run('preload');"
02:26:15     INFO -  /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory
02:26:15     INFO -  /home/worker/workspace/gaia/xulrunner-sdk/bin/xpcshell: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory
02:26:15     INFO -  make: *** [preload-app] Error 127


########## [3]
armenzg@armenzg-thinkpad:~/repos/pulse_actions$ docker run -ti --rm --device=/dev/video1:/dev/video1 taskcluster/desktop-test:0.5.0
Unable to find image 'taskcluster/desktop-test:0.5.0' locally
0.5.0: Pulling from taskcluster/desktop-test

Digest: sha256:44d9dc2296d4fd537dfd77ae17063b332cd463853d4d0cc1ad461eaf28bd25bc
Status: Downloaded newer image for taskcluster/desktop-test:0.5.0
root@taskcluster-worker:~# find / -name "libfreetype*"
/usr/share/doc/libfreetype6
/usr/share/doc/libfreetype6-dev
/usr/share/doc-base/libfreetype6-dev
/usr/lib/x86_64-linux-gnu/libfreetype.so
/usr/lib/x86_64-linux-gnu/libfreetype.so.6.8.0
/usr/lib/x86_64-linux-gnu/libfreetype.so.6
/usr/lib/x86_64-linux-gnu/libfreetype.la
/usr/lib/x86_64-linux-gnu/libfreetype.a
/var/lib/dpkg/info/libfreetype6:amd64.postinst
/var/lib/dpkg/info/libfreetype6-dev.list
/var/lib/dpkg/info/libfreetype6-dev.md5sums
/var/lib/dpkg/info/libfreetype6:amd64.symbols
/var/lib/dpkg/info/libfreetype6:amd64.postrm
/var/lib/dpkg/info/libfreetype6:amd64.list
/var/lib/dpkg/info/libfreetype6:amd64.shlibs
/var/lib/dpkg/info/libfreetype6:amd64.md5sums
/var/lib/doc-base/documents/libfreetype6-dev

Comment 1

[(not currently active) Ted Mielczarek]

Obviously you've noticed this is running a 32-bit binary, so you need the 32-bit versions of libs installed.

Comment 2

[Armen [:armenzg]]

I don't know why but in the install step of libxcb-render0 it removed a very large set of packages.

The binary started after installing all of these packages [1]

I will be making a push with a new image to see what happens.

[1]
Installed packages:
apt-get install libfreetype6:i386
apt-get install libnspr4:i386
apt-get install libfontconfig:i386
apt-get install libxrender-dev:i386
apt-get install libxcomposite1:i386
apt-get install libxcb-render0:i386
apt-get install libcairo2:i386
apt-get install libgtk2.0-0:i386
apt-get install libxt6:i386

Comment 3

[Armen [:armenzg]]

Attachment: ldd xulrunner-sdk/bin/libxul.so output

Comment 4

[Armen [:armenzg]]

Attachment: luciddream.diff

I don't know why I need to install the other dependencies at the top for libgtk2.0-dev to work.

In any case, this is failing to build due to conflicts.

armenzg@armenzg-thinkpad:~/repos/mozilla-central/testing/docker$ ./build.sh ubuntu1204-test
...
# This allows ubuntu-desktop to be installed without human interaction
export DEBIAN_FRONTEND=noninteractive
apt-get install -y --force-yes ${apt_packages[@]}
Reading package lists...
Building dependency tree...
Reading state information...
tar is already the newest version.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libxrender-dev : Conflicts: libxrender-dev:i386 but 1:0.9.6-2ubuntu0.2 is to be installed
 libxrender-dev:i386 : Conflicts: libxrender-dev but 1:0.9.6-2ubuntu0.2 is to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c bash /tmp/system-setup.sh' returned a non-zero code: 100

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8699157 [details] [diff] [review]
luciddream.diff

Review of attachment 8699157 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/docker/ubuntu1204-test/system-setup.sh
@@ +115,5 @@
> +apt_packages+=('libnspr4:i386')
> +apt_packages+=('libfontconfig:i386')
> +apt_packages+=('libxrender-dev:i386')
> +apt_packages+=('libxcomposite1:i386')
> +apt_packages+=('libxcb-render0:i386')

This libxcb stuff comes from a custom repo later in the system-setup.sh

Comment 6

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8800b7484c4e

Comment 7

[Armen [:armenzg]]

I won't be looking at this since Luciddream jobs are only for opt builds.

Comment 8

[Armen [:armenzg]]

This is not needed to make Linux64 debug test jobs tier 2 (bug 1171033).

Blocking instead opt tracker (bug 1243025).

Comment 9

[Geoff Brown [:gbrown]]

TC Linux64 opt tests (other than luciddream) are now running as Tier 2. 

This is still an issue: https://treeherder.mozilla.org/#/jobs?repo=try&revision=64805dc4feb0&filter-tier=1&filter-tier=2&filter-tier=3

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

:gbrown, can you figure this one out or drive getting a resolution on this?  It would be nice to get to 100% here.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

can we check with the team who cares about this job to ensure there is still interest in this?

Comment 12

[Geoff Brown [:gbrown]]

(In reply to Joel Maher (:jmaher) from comment #11)
> can we check with the team who cares about this job to ensure there is still
> interest in this?

That seems sensible. 

:jgriffin, do you know who we should contact?

Comment 13

[Jonathan Griffin (:jgriffin)]

Alexandre, do you care about luciddream tests? I'm guessing not, since no tests have been added since the suite was enabled. If you don't anticipate using it in the near future, we'd like to turn it off.

Comment 14

[Alexandre Poirot [:ochameau]]

I thought it was already disabled? Not everywhere yet?

Comment 15

[Geoff Brown [:gbrown]]

No. It is still running on trunk (mozilla-central, etc) for Linux x64 opt and Linux x64 pgo. It runs as "Tier 3", so it is not visible by default, but it is running. I don't see any open bugs for disabling it.

Comment 16

[Alexandre Poirot [:ochameau]]

Ah ok. Yes Tier 3 is like disabled to me. Then feel free to disable it completely.

Comment 17

[Geoff Brown [:gbrown]]

OK. Thanks.

So, we won't fix this and won't migrate luciddream to taskcluster. When those platforms are fully migrated to taskcluster, we'll drop the luciddream buildbot jobs along with all the other buildbot jobs.