1150630 - Build 32bits b2g-desktop/mulet

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Alexandre Poirot [:ochameau]]

Simulators used to release both 32 and 64bits xpis,
that to ensure app developers can work on 32b environment as well.

It would be really great to be able to continue doing so.
There isn't much need to run tests on it.
Just knowing that it builds is fine. And also *having builds* is what is the most handy!

Comment 1

[Pete Moore [:pmoore][:pete]]

I've had a look at building 32bit mulet on linux, I've currently hit this issue:

Here is my try push:

  * tree herder: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a896d092829e
  * task inspector: http://docs.taskcluster.net/tools/task-inspector/#2YmSkeMxTfqqQrcMHPqo8g/0
  * log: https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/2YmSkeMxTfqqQrcMHPqo8g/0/public/logs/live_backing.log

The error comes at:

[1G[J[34m 0:10.53(B[m configure:17482: checking for gtk+-2.0 >= 2.18.0 gtk+-unix-print-2.0 glib-2.0 >= 2.22 gobject-2.0 gdk-x11-2.0
[1G[J[34m 0:10.53(B[m configure: error: Library requirements (gtk+-2.0 >= 2.18.0 gtk+-unix-print-2.0 glib-2.0 >= 2.22 gobject-2.0 gdk-x11-2.0) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them.

I've committed and published the container after it fails:
https://registry.hub.docker.com/u/pmoore/failedmulet32

You can reproduce this issue in a local docker container with:

docker pull pmoore/failedmulet32:0.1
docker run -t -i pmoore/failedmulet32:0.1
cd ~/workspace/gecko/testing/taskcluster/scripts/builder
buildbot_step 'Build' ./build-mulet-linux.sh $HOME/workspace

The docker container was created with this Dockerfile:

https://hg.mozilla.org/try/file/a896d092829e/testing/docker/builder-i686/Dockerfile

which in turn is based on:

https://hg.mozilla.org/try/file/a896d092829e/testing/docker/b2g-build-i686/Dockerfile

These docker files show exactly how the CentOS environment is set up, and which packages are installed.

The mulet builds were added with:

I created the following mozconfig:
https://hg.mozilla.org/try/file/a896d092829e/b2g/dev/config/mozconfigs/linux32/mulet

... and the following configuration files:
  * https://hg.mozilla.org/try/file/a896d092829e/testing/taskcluster/tasks/build_i686.yml
  * https://hg.mozilla.org/try/file/a896d092829e/testing/taskcluster/tasks/builds/mulet_linux32.yml

The failing line seems to be coming from:
https://hg.mozilla.org/try/file/a896d092829e/configure.in#l4389

This may be related to https://bugzilla.mozilla.org/show_bug.cgi?id=1114898 where I think these dependencies changed.

Please note https://developer.mozilla.org/en/docs/Compiling_32-bit_Firefox_on_a_Linux_64-bit_OS is also a useful reference since our docker containers necessarily run 64 bit linux, we need to build 32 bit builds from inside 64 bit environment.

Glandium, might you be able to help me resolve this library dependency issue?

Please note in the end, it is the intention that the Dockerfile will use ./mach bootstrap in order to install dependencies (see bug 1133877).

Thanks!
Pete

Comment 2

[Mike Hommey [:glandium]]

From the log it would seem that the docker image doesn't have the necessary packages installed for some reason. I can't use docker to download the image to check what's up. I've tried with https://github.com/samalba/docker-registry-debug, but I can't get anything from failedmulet32 (while i can for, eg builder-i686), presumably because there is no tag.

Comment 3

[Pete Moore [:pmoore][:pete]]

Apologies Glandium, my bad! The image was ~5Gb (probably due to mozilla-central checkout) and the upload to dockerhub over my DSL connection failed. :(

The steps to reproduce are therefore slightly more involved:

$ docker pull pmoore/builder-i686:0.6.2
$ docker run -t -i pmoore/builder-i686:0.6.2

... and then inside the docker container I already preset all environment variables, so just need to run:

bash-4.1# checkout-gecko workspace && cd ./workspace/gecko/testing/taskcluster/scripts/builder && buildbot_step 'Build' ./build-mulet-linux.sh $HOME/workspace

This will check out gecko, and then attempt to build, but hit same error as above:
https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/2YmSkeMxTfqqQrcMHPqo8g/0/public/logs/live_backing.log

Thanks!
Pete

Comment 4

[Pete Moore [:pmoore][:pete]]

Attachment: Log file from creation of builder-i686 docker image

Comment 5

[Pete Moore [:pmoore][:pete]]

Attachment: Log file from creation of b2g-build-i686 docker image

Comment 6

[Pete Moore [:pmoore][:pete]]

Attachment: Log file from creation of builder-i686 docker image

Comment 7

[Pete Moore [:pmoore][:pete]]

For reference, the attached log files of the docker image creation process were generated with:

(cd testing/docker && ./build.sh b2g-build-i686 2>&1 | tee b2g-build-i686.log && ./build.sh builder-i686 2>&1 | tee builder-i686.log)

from a checkout of https://hg.mozilla.org/try/rev/56aabc7a2c29.

Comment 8

[Pete Moore [:pmoore][:pete]]

Attachment: installed_packages.log

The list of installed packages, as determined by

rpm -qa

Comment 9

[Pete Moore [:pmoore][:pete]]

I'm afriad I've not been able to focus on this topic, so I'm going to have to park it for now (quarterly goals to work on).

The work to-date can be found in:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=cdaaf2b206d4

Comment 10

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

The simulators see roughly 10 - 20% usage from 32 bit Linux (compared to 64 bit Linux), so it would be good to keep this flowing if we can.

I hope you are able to come back to it at some point!

Comment 11

[Pete Moore [:pmoore][:pete]]

Hi J. Ryan,

Essentially the biggest outstanding part is getting a working docker image to build with, with correct dependencies installed. I hope to reuse the work to be done in bug 1154827.

I'm working on the generic worker for the next two weeks, and then we have Whistler all-hands, so in July I'm free to come back to this, but hopefully with bug 1154827 landed, this should be much easier, and a small amount of work.

Pete

Comment 12

[Pete Moore [:pmoore][:pete]]

Hey Catlee,

Is this an integration piece that could be done by RelEng? There are a bunch of TC priorities at the moment, but maybe this fits more naturally in the scope of RelEng work rather than TC team?

Pete

Comment 13

[Chris AtLee [:catlee]]

RelEng doesn't have bandwidth to handle this either. Best thing would be to get the B2G folks to do this on their own.