Closed Bug 1013634 Opened 10 years ago Closed 9 years ago

All emulators in automation report "Failed to load libGL.so"

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jgriffin, Assigned: rail)

References

Details

Attachments

(2 files, 1 obsolete file)

Every emulator test job in buildbot is reporting the following error when the emulator is launched:

emulator: Initializing hardware OpenGLES emulation support
Failed to load libGL.so
error libGL.so: cannot open shared object file: No such file or directory
Failed to load libGL.so
error libGL.so: cannot open shared object file: No such file or directory

This happens with both successful and failing jobs.  It's nice that this isn't causing us to fail, but it may be causing us to run more slowly than we would otherwise, and could be the source of other problems.

The emulator expects this file to live at /usr/lib/x86_64-linux-gnu/libGL.so (for 64-bit linux) and should be symlinked from wherever it is to that location.
Good catch.

For reference:
> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/libGL.so
(s/i386/x86_64)

From https://developer.mozilla.org/en-US/Firefox_OS/Firefox_OS_build_prerequisites#Emulator_build_issues
See Also: → 989048
An alternative is to have libgl1-mesa-dev installed, but I am not sure how to do this?
Andrew, do you know how should I fix this for test machine, is this about mozharness?
Flags: needinfo?(ahalberstadt)
The code seems in mozharness [1], I tried to import mozharness to my mecurial repository, but it doesn't show up. Tried also Bitbucket, but the build was failed with HTTP 404 [2], probably the build server is not allow to access external resource.

[1] http://dxr.mozilla.org/build:mozharness/source/scripts/android_emulator_build.py#309
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=d5a6d75f91d2
Flags: needinfo?(ahalberstadt)
Try tries (sic!) to "probe" https://bitbucket.org/janus926/mozharness/rev/bug-1013634 per http://hg.mozilla.org/build/tools/file/0d8d4f943f6e/buildfarm/utils/repository_manifest.py#l148, fails to retrieve it and doesn't try to clone the repo after that. If you use your user repo on hg.m.o it should work as expected.
Thanks for the information! I've already filed bug 1115600 for the issue of cloning mozharness to my user repo.
The code I pointed out at comment 4 is not the right one, the same error message still shows up.
This would work, but seems like it would be better if the machines just came pre-configured with this. Rail, how difficult would it be to get them set up with this symlink?

> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/libGL.so
Flags: needinfo?(rail)
(In reply to Andrew Halberstadt [:ahal] from comment #9)
> This would work, but seems like it would be better if the machines just came
> pre-configured with this. Rail, how difficult would it be to get them set up
> with this symlink?
> 
> > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/libGL.so

Definitely it would be better than adding hacks to the code. libgl1-mesa-dev has the file in the package, installing it should fix the problem. One of the downsides though will be some other extra packages installed (all of them are *-dev, so should not interfere with other libraries).

I'll create a puppet patch for this.
Flags: needinfo?(rail)
Attached patch [puppet] install libgl1-mesa-dev (obsolete) — Splinter Review
Attachment #8542970 - Flags: review?(dustin)
Grabbing the bug to track the deployment.
Assignee: nobody → rail
Attachment #8542970 - Flags: review?(dustin) → review+
I'm going to deploy this ... not today. :) In the worst case scenario on Mon/Tue.
Comment on attachment 8542970 [details] [diff] [review]
[puppet] install libgl1-mesa-dev

remote:   https://hg.mozilla.org/build/puppet/rev/e9d1afd8c374
remote:   https://hg.mozilla.org/build/puppet/rev/2cf318e35f3c

This will show up in production tomorrow.
Attachment #8542970 - Flags: checked-in+
:ting, can you verify that the problem is fixed without applying your patch (attachment 8542490 [details] [diff] [review])?
Flags: needinfo?(tchou)
Comment on attachment 8542490 [details] [diff] [review]
patch v1

Thanks Rail!
Attachment #8542490 - Flags: review?(ahalberstadt)
Hmm, the patch symlinks 32-bit libraries, probably we need to install libgl1-mesa-dev:i386. I'll poke this next week. Meanwhile I'll backout the puppet patch.
Flags: needinfo?(tchou)
Comment on attachment 8542970 [details] [diff] [review]
[puppet] install libgl1-mesa-dev

remote:   https://hg.mozilla.org/build/puppet/rev/f6e23a227371
remote:   https://hg.mozilla.org/build/puppet/rev/37806d3f195a
Attachment #8542970 - Flags: checked-in+ → checked-in-
It'd be nice if we can fix this by the end of this week.
this is what's expected to be installed:

$ apt-get install libgl1-mesa-dev:i386
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libdrm-dev:i386 libkms1:i386 libpthread-stubs0:i386 libpthread-stubs0-dev:i386 libx11-dev:i386 libx11-doc libxau-dev:i386 libxcb1-dev:i386 libxdmcp-dev:i386 libxext-dev:i386
  mesa-common-dev:i386 x11proto-core-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev xorg-sgml-doctools xtrans-dev
Suggested packages:
  libxcb-doc:i386
Recommended packages:
  libx11-doc:i386
The following NEW packages will be installed:
  libdrm-dev:i386 libgl1-mesa-dev:i386 libkms1:i386 libpthread-stubs0:i386 libpthread-stubs0-dev:i386 libx11-dev:i386 libx11-doc libxau-dev:i386 libxcb1-dev:i386
  libxdmcp-dev:i386 libxext-dev:i386 mesa-common-dev:i386 x11proto-core-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev xorg-sgml-doctools xtrans-dev
0 upgraded, 18 newly installed, 0 to remove and 121 not upgraded.
Need to get 4893 kB of archives.
After this operation, 19.1 MB of additional disk space will be used.
Attachment #8546392 - Flags: review?
Attachment #8542970 - Attachment is obsolete: true
Attachment #8546392 - Flags: review? → review+
Comment on attachment 8546392 [details] [diff] [review]
install-mesa-32bit.diff

remote:   https://hg.mozilla.org/build/puppet/rev/d7eaf090016f
remote:   https://hg.mozilla.org/build/puppet/rev/561fab128519

We'll see the results tomorrow.

Sorry for the delay, upcoming releases and merge day keep me busy.
Attachment #8546392 - Flags: checked-in+
This should be fixed now. \o/
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Thank you Rail!
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.