661306 - Upgrade freetype to >= 2.3.x on Linux builders

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Joe Drew (not getting mail)]

We need a symbol introduced in Freetype 2.3 in order to fix the rainbowy text problem introduced with the Cairo update.

We have some worry that this will cause runtime requirements for previous versions to increase, since we only have one pool of builders, so this should be explicitly tested before rolling the new Freetype out to all the builders.

Also, this will definitely increase runtime requirements for Firefox 7; how do we handle this during update?

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

> Also, this will definitely increase runtime requirements for Firefox 7; how
> do we handle this during update?

To block updates from old versions we'd have to send the Freetype version in the update query, just like we do for GTK version. That change has to be made on all versions we'd want to update to Firefox 7, so you'd want it in at least 6 and possibly more depending what our update strategy will be. Christian may be able to comment on that (ie minor updates from all old versions to latest rapid release ?)

Bug 655597 is going to add the libsdtc++ version.

Comment 2

[Karl Tomlinson (:karlt)]

Even Debian Lenny has FreeType 2.3.7, so RHEL derived distros are probably the only major supported distros with older FreeType.

Comment 3

[Mike Taylor [:bear]]

nthomas,

Can bug 655597 be extended to cover this bug also?  Do we need to file a bug to work out the issues for what will happen when we upgrade to the older RHEL builders?

Comment 4

[Joe Drew (not getting mail)]

OK, how can we get this unblocked? Right now the experience of Nightly on Linux is pretty terrible.

We block certain updates on Enterprise Linux 5 and derivatives, as of bug 655917 - is that enough?

Comment 5

[Joe Drew (not getting mail)]

I don't know if Zandr or John can help unblock this. We need this to be done prior to Fx7 branching.

Comment 6

[Joe Drew (not getting mail)]

We are down to less than 2 weeks before Fx7 branching. How can this be unblocked?

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #6)
> We are down to less than 2 weeks before Fx7 branching. How can this be
> unblocked?

Joe, can you verify that upgrading to this new version of freetype does not break older versions of Firefox or increase system requirements?  We can loan you a slave if that helps the investigation.

Building and deploying the upgraded freetype shouldn't be too difficult, unless it has a ton of unsatisfied dependencies.

Comment 8

[Joe Drew (not getting mail)]

Rob, we need to make sure we don't offer upgrades to people on distributions with old versions of Freetype. It's possible that bug 655917 is enough to ensure that, but if not, how can we go about sending the version of Freetype?

Chris AtLee is also worried about sending all these library versions in the AUS ping. He specifically said "maybe the snippet could describe minimum library versions and the client could determine if it's got them or not, instead of making aus do the heavy lifting there."

Thoughts?

Comment 9

[Joe Drew (not getting mail)]

(In reply to comment #7)
> Joe, can you verify that upgrading to this new version of freetype does not
> break older versions of Firefox or increase system requirements?  We can
> loan you a slave if that helps the investigation.

I believe it increases system requirements, and we want that. (It's possible that linking against newer Freetype wouldn't break anything if we didn't use any of the new symbols, but I don't want to rely upon that.) So apparently upgrading Freetype is a non-starter, since the same pool of slaves builds 3.6.

We already have a dynamic lookup of the relevant symbol (FT_Library_SetLcdFilter), so what we have to do to fix this is make sure we don't update people who have a too-old Freetype.

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

if freetype uses pkgconfig, I wonder if we could build a second version of freetype in /tools and use that to build newer branches.

Comment 11

[John Ford [:jhford] CET/CEST Berlin Time]

(this is a case where having chroots for each train would be great)

Comment 12

[Robert Strong (they/them - no direct email)]

(In reply to comment #8)
> Rob, we need to make sure we don't offer upgrades to people on distributions
> with old versions of Freetype. It's possible that bug 655917 is enough to
> ensure that, but if not, how can we go about sending the version of Freetype?
> 
> Chris AtLee is also worried about sending all these library versions in the
> AUS ping. He specifically said "maybe the snippet could describe minimum
> library versions and the client could determine if it's got them or not,
> instead of making aus do the heavy lifting there."
> 
> Thoughts?
Perhaps this could be done though this would mean that we have logic to accomplish this on the client and the server and I think it is better to keep it at the server instead of adding complexity to the client which is more difficult to update (e.g. if a problem occurs on the client side we're stuck whereas if a problem occurs on the server side we can fix it).

Comment 13

[Chris AtLee [:catlee]]

(In reply to comment #12)
> (In reply to comment #8)
> > Rob, we need to make sure we don't offer upgrades to people on distributions
> > with old versions of Freetype. It's possible that bug 655917 is enough to
> > ensure that, but if not, how can we go about sending the version of Freetype?
> > 
> > Chris AtLee is also worried about sending all these library versions in the
> > AUS ping. He specifically said "maybe the snippet could describe minimum
> > library versions and the client could determine if it's got them or not,
> > instead of making aus do the heavy lifting there."
> > 
> > Thoughts?
> Perhaps this could be done though this would mean that we have logic to
> accomplish this on the client and the server and I think it is better to
> keep it at the server instead of adding complexity to the client which is
> more difficult to update (e.g. if a problem occurs on the client side we're
> stuck whereas if a problem occurs on the server side we can fix it).

I was thinking that the user experience would be better if firefox could explain _why_ it couldn't download and apply the update. Maybe we need support from AUS to give explanatory messages like this?

Comment 14

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

From irc with joe:

Joe's request here is actually for 3 things:

1) install new software on linux builders, and start using that for linux builds for FF7. This is covered by this bug#661306.

2) change FF client string for linux users to transmit info so we can tell if linux users meet new requirement. Getting that fix into a shipped dot-releases of still supported versions of FF is going to take a while, and will also need to wait while users upgrade to it. This needs a separate bug to track, joe will file and also notify release-drivers.

3) change AUS to detect and handle those updates correctly. This needs a separate bug to track, joe will file.

Comment 15

[Robert Strong (they/them - no direct email)]

I've thought about adding canned messages for these cases... something like a simple message stating that the OS is not supported by the latest release along with a link to a page that details the system requirements. Is that what you are thinking of?

Comment 16

[Joe Drew (not getting mail)]

(In reply to comment #14)
> 2) change FF client string for linux users to transmit info so we can tell
> if linux users meet new requirement. Getting that fix into a shipped
> dot-releases of still supported versions of FF is going to take a while, and
> will also need to wait while users upgrade to it. This needs a separate bug
> to track, joe will file and also notify release-drivers.

This is bug 666732.

> 3) change AUS to detect and handle those updates correctly. This needs a
> separate bug to track, joe will file.

This is bug 666735.

At some point we will probably need to update Freetype on the build machines, but it's *probably* OK if that happens later than those two bugs are fixed.

Comment 17

[Chris AtLee [:catlee]]

(In reply to comment #15)
> I've thought about adding canned messages for these cases... something like
> a simple message stating that the OS is not supported by the latest release
> along with a link to a page that details the system requirements. Is that
> what you are thinking of?

I was thinking of something more specific, like "No updates available because your libfreetype is too old."

Comment 18

[Robert Strong (they/them - no direct email)]

Trying to get those strings to be generic enough that we can say that with a previous release across locales would probably be more pain than it is worth whereas stating a generic message with a link to a page that can provide instructions would be much easier to accomplish.

Comment 19

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

found in triage.

Comment 20

[Chris Cooper [:coop] (he/him)]

After 11 months, what's the action item here for releng? 

Are we still waiting on dependencies? Does a dev need a loaner slave (or two: 32/64) to verify the upgrade?

Comment 21

[JP Rosevear [:jpr]]

Didn't we end up resolving this with bug 666735 ?

Comment 22

[Karl Tomlinson (:karlt)]

The action item is 1) of comment 14.

If we had this bug fixed, then we wouldn't need to carry along cairo patches to access symbols through dlopen.  That is much less of a priority than bug 662417 and bug 740690, but involves the same process decisions for releng.

There are plenty of distros building against newer FreeType, so I don't think a dev needs a loaner slave.

Comment 23

[Chris Cooper [:coop] (he/him)]

Should get a new version as a by-product of bug 772446.

Comment 24

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to Chris Cooper [:coop] from comment #23)
> Should get a new version as a by-product of bug 772446.


Found in triage. Anything left to do here?

Comment 25

[Karl Tomlinson (:karlt)]

Should be good here now.