586037 - Upgrade fontconfig from 2.4.1 to 2.4.2 on Linux builders

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

I am trying to install a QT 4.6.3 SDK so we can produce Linux QT builds of Firefox and Fennec.  When I try to do a test build, i get errors about not having FcFreeTypeQueryFace.  According to the fontconfig changelog [1] this API was added in version 2.4.2.

My current plan is to manually rebuild the CentOS 5.0 fontconfig-2.4.1 rpm [2] using the upstream fontconfig-2.4.2 tarball.

From IRC:
jhford - where is the right place to ask if we are ok to upgrade fontconfig on linux to 2.4.2?
jhford - from 2.4.1
blassey - jhford: on the builders?
jhford - assuming it doesn't break builds, should I speak to anyone to ok the rollout before doing a rollout?
blassey - probably, but who is a good question
blassey - daggett, roc, vlad are all good choices
jhford - cool, thanks
roc - I don't know of any problem with that
roc - karlt would know best, but it's probably fine

If we are not ok to roll this out globally to all 32bit and 64bit linux slaves, does anyone have any suggestions to be able to build only QT builds with an alternative version of fontconfig.

[1]http://fontconfig.org/release/ChangeLog-2.4.2
[2]http://mirror.centos.org/centos/5/os/SRPMS/fontconfig-2.4.1-7.el5.src.rpm

Comment 1

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: fontconfig 2.4.2 srpm

All I did was modify the upstream RPM to use v2.4.2 instead of 2.4.1 and grabbed the tarball from the upstream distribution site.

Comment 2

[Karl Tomlinson (:karlt)]

Is it Qt that wants FcFreeTypeQueryFace?

This sounds fine with me.
I don't expect building against fontconfig 2.4.2 to change our minimum required version.
It just means that we won't notice should we change the code to link against this new function.
Even if we did that, IMHO it wouldn't be a bit deal to move our minimum system requirements from September 2006 to December 2006 levels.
(I don't know who the official person to ask would be though.)

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #2)
> Is it Qt that wants FcFreeTypeQueryFace?

Yes.  It is currently not building even with fontconfig 2.4.2, but that is something to do with the QT installation and/or our build system.

> This sounds fine with me.
> I don't expect building against fontconfig 2.4.2 to change our minimum required
> version.
> It just means that we won't notice should we change the code to link against
> this new function.
> Even if we did that, IMHO it wouldn't be a bit deal to move our minimum system
> requirements from September 2006 to December 2006 levels.
> (I don't know who the official person to ask would be though.)

a quick list of distributions that ship with fontconfig and the first version that I could find 2.4.2 or newer on and last version of 2.4.1 or lower if I could find it.
debian stable  - 2.6.0
ubuntu 6.06LTS - 2.3.2
ubuntu 8.04    - 2.5.0
fedora6        - 2.4.1
fedora7        - 2.4.2
RHEL 5.X       - 2.4.1
RHEL 6 Beta    - 2.8.0
Suse 10        - 2.3.2
opensuse 11.0  - 2.4.2
Mandr* 2007.0  - 2.4.1
Mandr* 2007.1  - 2.4.2

I don't know if it is worth doing a test GTK build against 2.4.2 and running the resulting build on a machine with 2.4.1 installed.

Comment 4

[Karl Tomlinson (:karlt)]

(In reply to comment #3)
> I don't know if it is worth doing a test GTK build against 2.4.2 and running
> the resulting build on a machine with 2.4.1 installed.

I can't see any issues there, but for peace of mind, may be worth doing.

Comment 5

[John Ford [:jhford] CET/CEST Berlin Time]

I have done a build with 2.4.2 and uploaded it to http://people.mozilla.com/~jford/fontconfig-2.4.2/

I will run the unittests on it when I get a chance.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

I ran this build through our unit test systems and got the following result on our reference test machine (Fedora 12):

mochitest 1/5 - green
mochitest 2/5 - green
mochitest 3/5 - green
mochitest 4/5 - green
mochitest 5/5 - green
mochitest other - green
crashtest - green
xpcshell - green
reftest - green
jsreftest - green

I will run these through on a Centos machine when I get a chance.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

XPCShell:
INFO | Result summary:
INFO | Passed: 836
INFO | Failed: 0

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

Crashtests:
REFTEST INFO | Loading a blank page
REFTEST FINISHED: Slowest test took 81790ms (file:///home/cltbld/fontconfig-2.4.2/reftest/tests/content/base/crashtests/340733-1.html)
REFTEST INFO | Result summary:
REFTEST INFO | Successful: 1639 (2 pass, 1637 load only)
REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 unexpected fixed asserts, 0 failed load, 0 exception)
REFTEST INFO | Known problems: 10 (0 known fail, 0 known asserts, 0 random, 10 skipped, 0 slow)
REFTEST INFO | Total canvas count = 0
REFTEST TEST-START | Shutdown
NOTE: child process received `Goodbye', closing down

Comment 9

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: full reftest log from build

Reftests

REFTEST INFO | Loading a blank page
REFTEST FINISHED: Slowest test took 5782ms (http://localhost:4444/1282759243334/122/font-matching/weightmapping-12579.html)
REFTEST INFO | Result summary:
REFTEST INFO | Successful: 4538 (4520 pass, 18 load only)
REFTEST INFO | Unexpected: 11 (5 unexpected fail, 6 unexpected pass, 0 unexpected asserts, 0 unexpected fixed asserts, 0 failed load, 0 exception)
REFTEST INFO | Known problems: 250 (158 known fail, 0 known asserts, 64 random, 28 skipped, 0 slow)
REFTEST INFO | Total canvas count = 14
REFTEST TEST-START | Shutdown
INFO | automation.py | Application ran for: 0:36:54.438029
INFO | automation.py | Reading PID log: /tmp/tmpPIlXQspidlog
==> process 22597 launched child process 23010
==> process 22597 launched child process 23431
INFO | automation.py | Checking for orphan process with PID: 23010
INFO | automation.py | Checking for orphan process with PID: 23431
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

REFTEST INFO | runreftest.py | Running tests: end.

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

there are a bunch of unexpected results and it looks like most of them are in the fonts area

Is this safe to proceed with?

Comment 11

[Karl Tomlinson (:karlt)]

John, are you running these reftests on CentOS?
Are all our production tests run on Fedora 12 machines, now?

If we are running the tests on CentOS,
do the current machines have fontconfig-2.4.1-6.el5?
That's what I would have guessed from http://vault.centos.org/5.0/os/SRPMS/

According to the changelog, the difference between -6 and -7 is:
* Sat Aug 25 2007 Behdad Esfahbod <[e-mail]> - 2.4.1-7
- Add Liberation fonts to 30-aliases-fedora.conf
- Resolves: #243049

Changes from 2.4.1 to 2.4.2 that might affect things are:

commit 0f963b0d3ec417a39f6aa2ba22ba56c2a79d05aa
Author: Keith Packard <[e-mail]>
Date:   Sat Dec 2 13:57:45 2006 -0800

    Segfault scanning non-font files. Disallow scan edit of user
    vars. (#8767)

    Missing NULL font check before attempting to edit scanned pattern.
    Also, <match target="scan"> rules are now checked to ensure all
    edited variables are in the predefined set; otherwise, the resulting
    cache files will not be stable.

 src/fcdir.c |    2 +-
 src/fcint.h |    1 +
 src/fcxml.c |    6 ++++++
 3 files changed, 8 insertions(+), 1 deletions(-)

commit 72ffe6536a6825a32095c8185aff836a12326ac5
Author: Keith Packard <[e-mail]>
Date:   Sat Dec 2 13:22:27 2006 -0800

    Add FcFreeTypeQueryFace external API. Bug #7311.

    Expose ability to build an FcPattern directly from an FT_Face
    object.

 configure.in            |    4 +-
 doc/fcfreetype.fncs     |   17 +++++++++++++++-
 fontconfig/fcfreetype.h |    6 +++++
 src/fcfreetype.c        |   50
 ++++++++++++++++++++++++++--------------------
 4 files changed, 52 insertions(+), 25 deletions(-)

This might affect tests with downloaded fonts, hopefully in a positive way, but there is unexpected fail with downloaded fonts in font-features-turkish.html.

ommit 253ec7609c13b46c717c801206ebb1a6c7f06e27
Author: Keith Packard <[e-mail]>
Date:   Sat Dec 2 11:47:07 2006 -0800

    Use explicit platform/nameid order when scanning ttf files.

    Instead of accepting whatever order names appear in the font file,
    use an explicit ordering for both platform and nameid.

    Platforms are high precedence than nameids.

    The platform order is:

        microsoft, apple unicode, macintosh, (other)

    The family nameid order is:

        preferred family, font family

    The fullname nameid order is:

        mac full name, full name

    The style nameid order is

        preferred subfamily, font subfamily

    This will change the names visible to users in various application
    UIs, but
    should not change how existing font names are matched as all names
    remain
    present in the resulting database. The hope is that family names
    will, in
    general, be less ambiguous. Testing here shows that commercial fonts
    have longer names now while DejaVu has a shorter family name, and
    moves more
    of the font description to the style name.

Comment 12

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #11)
> John, are you running these reftests on CentOS?
> Are all our production tests run on Fedora 12 machines, now?

Yes, and they all pass as green on Fedora 12
 
> If we are running the tests on CentOS,
> do the current machines have fontconfig-2.4.1-6.el5?
> That's what I would have guessed from http://vault.centos.org/5.0/os/SRPMS/

As I understand it, we only run 1.9.2/1.9.1 tests on the build machines.  We do have -6 installed currently.
 
> According to the changelog, the difference between -6 and -7 is:
> * Sat Aug 25 2007 Behdad Esfahbod <[e-mail]> - 2.4.1-7
> - Add Liberation fonts to 30-aliases-fedora.conf
> - Resolves: #243049
> 

> commit 72ffe6536a6825a32095c8185aff836a12326ac5
> Author: Keith Packard <[e-mail]>
> Date:   Sat Dec 2 13:22:27 2006 -0800
> 
>     Add FcFreeTypeQueryFace external API. Bug #7311.
> 
>     Expose ability to build an FcPattern directly from an FT_Face
>     object.
> 
>  configure.in            |    4 +-
>  doc/fcfreetype.fncs     |   17 +++++++++++++++-
>  fontconfig/fcfreetype.h |    6 +++++
>  src/fcfreetype.c        |   50
>  ++++++++++++++++++++++++++--------------------
>  4 files changed, 52 insertions(+), 25 deletions(-)
>
>This might affect tests with downloaded fonts, hopefully in a positive way, but
>there is unexpected fail with downloaded fonts in font-features-turkish.html.

This commit is the reason why we need fontconfig-2.4.2.  I am not too sure about the other commits as I don't really know all that much about fontconfig.

It would be great if I could build a second copy of fontconfig to use only in QT 4 builds.  I tried altering the specfile to do this but found that the rpm does a lot of setup and configuration.  When I get a chance, I am going to build the library without any of the configuration steps and see if adding the pkg-config file to the relevant env. variable will work.

Comment 13

[Karl Tomlinson (:karlt)]

John, I suspect that these results would be similar when running the same build
against fontconfig-2.4.1.

The bidi-04 tests are marked as failing on 1.9.1 and 1.9.2.
mirroring-02 and the font-features tests are not present on 1.9.2.

Comment 14

[John Ford [:jhford] CET/CEST Berlin Time]

i could build the same revision against 2.4.1 and compare the logs.

Comment 15

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: fontconfig 2.4.2 specfile

spec file to rebuild fontconfig for 2.4.2

Comment 16

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: fontconfig 2.4.2-1 srpm

srpm, can be rebuilt with 

rpmbuild --target i686 --rebuild fontconfig-2.4.2-1.src.rpm

Comment 17

[Chris Cooper [:coop] (he/him)]

Comment on attachment 494640 [details] [diff] [review]
fontconfig 2.4.2 specfile

Looks fine. Is there much of a diff between this spec file and the one for 2.4.1?

Comment 18

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #17)
> Comment on attachment 494640 [details] [diff] [review]
> fontconfig 2.4.2 specfile
> 
> Looks fine. Is there much of a diff between this spec file and the one for
> 2.4.1?

I can (and should) do a diff between them, but no, I remember the only changes were version numbers and my adding to the changelog.

I will need to make a tiny change (fontconfig-devel no longer depending on the same version of fontconfig) to make puppet deployment much easier.  I will post that at the same time that I upload puppet manifests.

Comment 19

[John Ford [:jhford] CET/CEST Berlin Time]

This package is going to be deployed as part of bug 584424.

Comment 20

[John Ford [:jhford] CET/CEST Berlin Time]

deployed