584424 - Deploy a QT development environment to linux build slaves

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

In order to get builds of Firefox and Fennec with QT, we need to have the QT libraries installed on our build machines.  I am not sure if we want to deploy a testing environment to the linux slaves at this time.

Comment 1

[John Ford [:jhford] CET/CEST Berlin Time]

I am going to do a test installation of the SDK available at http://qt.nokia.com/downloads/sdk-linux-x11-32bit-cpp using the archive http://get.qt.nokia.com/qtsdk/qt-sdk-linux-x86-opensource-2010.04.bin

If this version of the SDK is *not* acceptable please let me know.

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

Alternatively, I can build the libraries at http://qt.nokia.com/downloads/linux-x11-cpp in the source tarball http://get.qt.nokia.com/qt/source/qt-everywhere-opensource-src-4.6.3.tar.gz

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

I used the sdk with a mozconfig of 

ac_add_options --with-qtdir=/home/cltbld/qtsdk-2010.04/qt
 
mk_add_options MOZ_MAKE_FLAGS=-j4
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir
 
ac_add_options --enable-application=browser
ac_add_options --enable-default-toolkit=cairo-qt
ac_add_options --disable-crashreporter

and errored out with

/tools/gcc/bin/gcc   -Wall -W -Wno-unused -Wpointer-arith -Wcast-align -W -pedantic -Wno-long-long -fno-strict-aliasing -pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fomit-frame-pointer -finline-limit=50 -o xpt_dump xpt_dump.o  -lpthread   -Wl,-rpath-link,/builds/qt-testing/mozilla-central/objdir/dist/bin -Wl,-rpath-link,/usr/local/lib  -L../../../../dist/bin -L../../../../dist/lib ../../../../dist/lib/libxpt.a -ldl -lm  -L/home/cltbld/qtsdk-2010.04/qt/lib/ -lQtGui -lQtNetwork -lQtCore -lQtDBus -lQtXml -lQtOpenGL   
//home/cltbld/qtsdk-2010.04/qt/lib//libQtGui.so: undefined reference to `FcFreeTypeQueryFace'
collect2: ld returned 1 exit status
home/cltbld/qtsdk-2010.04/qt/lib//libQtGui.so: undefined reference to `FcFreeTypeQueryFace'
collect2: make[8]: *** [xpt_link] Error 1
ld returned 1 exit statusmake[8]: *** Waiting for unfinished jobs....

make[8]: *** [xpt_dump] Error 1
make[8]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom/typelib/xpt/tools'
make[7]: *** [export] Error 2
make[7]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom/typelib/xpt/tools'
make[6]: *** [export] Error 2
make[6]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom/typelib/xpt'
make[5]: *** [export] Error 2
make[5]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom/typelib'
make[4]: *** [export] Error 2
make[4]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom'
make[3]: *** [export_tier_platform] Error 2
make[3]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make[2]: *** [tier_platform] Error 2
make[2]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make: *** [build] Error 2

Which looks like it is an issue with the fontconfig package

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

We have fontconfig-2.4.1 but according to http://fontconfig.org/release/ChangeLog-2.4.2 was added in version 2.4.2.

Looks like we need to upgrade fontconfig-2.4.2 to use this QT library.

Comment 5

[John Ford [:jhford] CET/CEST Berlin Time]

Adding bug 586037 as a dependency.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

I tested a build with the test installation of fontconfig-2.4.2 and got this output

/builds/qt-testing/mozilla-central/objdir/dist/bin/xpidl -m header -w -I/builds/qt-testing/mozilla-central/xpcom/base -I../../dist/idl -o _xpidlgen/nsIConsoleService /builds/qt-testing/mozilla-central/xpcom/base/nsIConsoleService.idl
make[5]: *** [_xpidlgen/nsIConsoleListener.h] Error 127
make[5]: *** Waiting for unfinished jobs....
make[5]: *** [_xpidlgen/nsIConsoleMessage.h] Error 127
/builds/qt-testing/mozilla-central/objdir/dist/bin/xpidl -m header -w -I/builds/qt-testing/mozilla-central/xpcom/base -I../../dist/idl -o _xpidlgen/nsIDebug2 /builds/qt-testing/mozilla-central/xpcom/base/nsIDebug2.idl
/builds/qt-testing/mozilla-central/objdir/dist/bin/xpidl: error while loading shared libraries: libQtGui.so.4: cannot open shared object file: No such file or directory
/builds/qt-testing/mozilla-central/objdir/dist/bin/xpidl: error while loading shared libraries: libQtGui.so.4: cannot open shared object file: No such file or directory
make[5]: *** [_xpidlgen/nsIConsoleService.h] Error 127
make[5]: *** [_xpidlgen/nsIDebug2.h] Error 127
make[5]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom/base'
make[4]: *** [export] Error 2
make[4]: Leaving directory `/builds/qt-testing/mozilla-central/objdir/xpcom'
make[3]: *** [export_tier_platform] Error 2
make[3]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make[2]: *** [tier_platform] Error 2
make[2]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/builds/qt-testing/mozilla-central/objdir'
make: *** [build] Error 2

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

It looks like adding a file 

[root@mv-moz2-linux-ix-slave01 ~]# cat /etc/ld.so.conf.d/qt.conf 
/home/cltbld/qtsdk-2010.04/qt/lib

allowed the build system to find the shared libraries.  The build still broke, but it looks like a code issue.  is there something that I can build to ensure that this is a working installation of qt4?

I am going to clean up my objdir and try again.

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

After speaking with Oleg, it looks like updating to m-c default-tip has the required fixes.

the unattended installer for the qt sdk is so broken, see http://bugreports.qt.nokia.com/browse/QTCREATORBUG-472

Comment 9

[Oleg Romashin (:romaxa)]

I think on latest m-c: it should compile fine, I have checked build and it looks like our firefox with broken qt themeing ;)

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: qt4.6.3 v1

this repackages the qt sdk for use on the build slaves.  This patch won't have the corresponding srpm uploaded as that is >400MB.  The SRPM is in the ~cltbld/rpmbuild folder on mv-moz2-linux-ix-slave01

Comment 11

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: puppet manfiests

Having trouble with this patch.  In the puppet client logs, I am seeing:

err: //Node[build]/base/centos5/Package[qt-sdk]/ensure: change from absent to present failed: Execution of '/bin/rpm -i --oldpackage http://staging-puppet.build.mozilla.org/staging/centos5-i686/build/RPMs/qt-sdk-4.6.3-2010.04moz1.i686.rpm' returned 1: error: Failed dependencies:
	fontconfig >= 2.4.2 is needed by qt-sdk-4.6.3-2010.04moz1.i686
.

As I understand my patch, the fontconfig-2.4.2 installation should have already happened because of the requirement line.

Comment 12

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: puppet manifest changes needed for deploying the QT SDK

Tested to work

Dec  6 15:00:35 linux-ix-slave01 puppetd[8064]: (//Node[build]/base/centos5/Package[fontconfig-devel]/ensure) ensure changed '2.4.1-6.el5' to '2.4.2-2'
Dec  6 15:00:40 linux-ix-slave01 puppetd[8064]: (//Node[build]/base/centos5/Package[fontconfig]/ensure) ensure changed '2.4.1-6.el5' to '2.4.2-2'
Dec  6 15:05:42 linux-ix-slave01 puppetd[8064]: (//Node[build]/base/centos5/Package[qt-sdk]/ensure) created


This needs a small change to the spec file to allow the fontconfig packages to get around a circular dependency.

Comment 13

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: fontconfig 2.4.2-2 diff

This is a change to make deployment through puppet work.  The RPM isn't really valid for general distribution because it could result in an invalid state (fontconfig 2.4.1 with fontconfig-devel 2.4.2).

Comment 14

[Chris Cooper [:coop] (he/him)]

Comment on attachment 495653 [details] [diff] [review]
fontconfig 2.4.2-2 diff

Aren't the version requirements meant to save us from this sort of thing? Is there something I'm missing here?

Comment 15

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #14)
> Comment on attachment 495653 [details] [diff] [review]
> fontconfig 2.4.2-2 diff
> 
> Aren't the version requirements meant to save us from this sort of thing? Is
> there something I'm missing here?

yes, but the way the Requires lines are, we have a circular dependency.  The only way to satisfy a circular dependency is to either use 

# rpm -Uvh --nodeps fontconfig-2.4.2.rpm
# rpm -Uvh fontconfig-devel-2.4.2.rpm

or 

# rpm -Uvh fontconfig-2.4.2.rpm fontconfig-devel-2.4.2.rpm

Using --nodeps is a very bad idea because it ignores all dependencies, not just the known to be ignorable ones.  If puppet supported multiple packages in one Package[] instance, we could use that instead.

I could optionally put the Requires: line back in and use an Exec[] type that does an rpm -Uvh using the puppet file store hrefs, but i don't know if that is any worse than changing the spec and rebuilding.

Comment 16

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #15) 
> Using --nodeps is a very bad idea because it ignores all dependencies, not just
> the known to be ignorable ones.  If puppet supported multiple packages in one
> Package[] instance, we could use that instead.

forgot to mention that neither of these are currently supported in puppet.

Comment 17

[Chris Cooper [:coop] (he/him)]

Comment on attachment 495653 [details] [diff] [review]
fontconfig 2.4.2-2 diff

All the proposed solution seem hacky, but at least this one is in hand. Switching to r+.

Comment 18

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: debug info update

also update the debuginfo package so that backtraces stay valid

Comment 19

[John Ford [:jhford] CET/CEST Berlin Time]

deployed