Closed Bug 1634204 Opened 4 years ago Closed 4 years ago

Investigate upgrading runtime requirements on Linux

Categories

(Firefox Build System :: General, task)

task

Tracking

(relnote-firefox 78+, firefox78 fixed)

RESOLVED FIXED
mozilla78
Tracking Status
relnote-firefox --- 78+
firefox78 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(5 files)

Last upgrade was bug 1536307, in time for esr68. With esr78 being close, it's about the right time to start thinking about this. This was initially brought up in bug 1633778 for Thunderbird, where I noted the following:

The Linux compat table would need to be refreshed and more investigation done, but at quick glance, it seems that bumping to 5.0 /might/ be a reasonable thing to do, even for Firefox.

Then

(...) if we're going to bump, we might as well explore reasonable options. It seems the only distro that has libstdc++ < 5 and that is still currently supported by Firefox and not EOL is Debian 8, which EOLs in June, so before 78 is actually released. Bumping to 5 would remove the need for _GLIBCXX_USE_CXX11_ABI=0.

So, one thing that I missed is that RHEL7 is listed as having libstdc++ 4.8, so 5.0 wouldn't work there.

4.8.5, exactly, which is up to GLIBCXX_3.4.19.

One data point: we bumped the runtime requirement to libstdc++ 4.6 with Firefox 50, in November 2016, 6 years after the release of RHEL6, which is, actually not even EOL yet. 78 would release 6 years after the release of RHLE7. However, November 2016 was also 2 years after the release of RHEL7, while we're only 1 year after the release of RHEL8.

Now, something else that is interesting to look at is, eliminating distros that are EOL or soon EOL (i.e. Debian 8):

  • the smallest version of glibc is 2.17 if we stick to supporting RHEL7, or 2.22 otherwise (we currently require 2.12)
  • the smallest version of glib is 2.42 if we stick to supporting RHEL7, or 2.48 otherwise (we currently require 2.22)
  • the smallest version of gtk+3 is 3.14 if we stick to supporting RHEL7, or 3.18 otherwise (we currently require 3.4)
  • the smallest version of pixman is 0.32 if we stick to supporting RHEL7, or 0.33 otherwise (we currently require 0.19.2)

In any case, we might be able to switch the builds to a Debian 8 baseline rather than the current Debian 7 (although Debian 8 comes with glibc 2.19, so there may be some problem there, depending on the symbols we use from glibc). Time to revive my patches to build on Debian 8.

Summary: Investigate upgrading libstdc++ requirement → Investigate upgrading runtime requirements on Linux

20190228 is the last snapshot of Debian wheezy available on dockerhub.

We currently check that the binaries we ship are not using symbol
versions of system libraries that would not be available on some older
systems. In some cases, however, we may get dependencies on newer symbol
versions in the form of weak symbols, that are checked for at runtime.
This happens with __cxa_thread_atexit_impl when building against a glibc
newer than 2.18, and the supporting code in Rust libstd actually checks
at runtime whether the weak symbol is resolved before using it.

This gets rid for the need of a number of local packages (mostly related
to Gtk+3). One exception is that we now need a 32-bits version of the
xz-utils package, some -dev package depends on it, and that dependency
can't be fulfilled in the 32-bits image because we already have the
64-bits backport installed, which conflicts with it (we need both
32-bits and 64-bits package to be at the same version when installed).

The system binutils fails to link clang-7 for some reason, so we now use
our toolchain binutils instead, like we already do for newer versions of
clang.

The debian-packages docker image now needs an explicit installation of
git, because it's not pulled in via the recommends of some other
package.

For some reason, snapshot.debian.org doesn't contain the jessie-backports
archive at the same location as others, and only has a few snapshots of
the archive.

While we're going to rebuild all the Debian-based docker images, we
might as well add one corrective thing that is going to be required
later on, and would require another rebuild, while doing it now would
skip that.

On Debian 10, the python3 distutils module is in a separate package,
python3-distutils, which didn't exist in previous Debian releases,
because back then the distutils module was part of the python3 package.

Fortunately, the python3-distutils-extra package existed on all the
Debian versions we use and depends on the distutils module, so it is a
convenient way to have the python3-distutils package pulled in without
altering the list of packages to install based on the Debian version.

Attachment #9145696 - Attachment description: Bug 1634204 - Bump glibc and libstdc++ runtime requirement to resp. 2.17 and 3.4.19 (4.8). → Bug 1634204 - Bump glibc and libstdc++ runtime requirement to resp. 2.17 and 3.4.19 (4.8.1).
Blocks: 1633778

I ran a Thunderbird try build with these changes today, no immediate problems.

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/e6a1b861522b
Update Debian-based docker images to a newer snapshot. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/71b00ec52717
Allow newer versions of symbols if they are weakly linked. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/cf742676d316
Bump glibc and libstdc++ runtime requirement to resp. 2.17 and 3.4.19 (4.8.1). r=froydnj
https://hg.mozilla.org/integration/autoland/rev/2bf3343d2994
Upgrade Linux build docker images to Debian 8. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/9d6acea5d432
Drive-by adjustment to Debian docker images. r=froydnj
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/b4d73278792e
Upgrade Linux build docker images to Debian 8. r=froydnj
Backout by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/09e8c2bcfc5f
Backed out changeset b4d73278792e as per glandium req.
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/0a1edd15b210
Upgrade Linux build docker images to Debian 8. r=froydnj
Depends on: 1635990
Depends on: 1636003
Status: RESOLVED → REOPENED
Flags: needinfo?(mh+mozilla)
Resolution: FIXED → ---
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/8621fde19c31
Upgrade Linux build docker images to Debian 8. r=froydnj
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED

Not sure if this needs a mention in the release notes, but this bug does need an update of the system requirements for 78: glibc -> 2.17, libstdc++ -> 4.8.1, Gtk+3 -> 3.14 (the latter may not be entirely true, as in, older versions might work, but meh, see commit message in https://hg.mozilla.org/mozilla-central/rev/cf742676d316).

Flags: needinfo?(jcristau)

Hi Mike. Do the improvements below make sense for your patches?
== Change summary for alert #25856 (as of Thu, 07 May 2020 09:10:23 GMT) ==

Improvements:

13% build times osx-shippable opt nightly taskcluster-c5d.4xlarge 3,240.26 -> 2,813.73
11% build times osx-shippable opt nightly taskcluster-c5.4xlarge 3,284.32 -> 2,912.96
8% build times linux64-shippable opt nightly taskcluster-c5d.4xlarge 3,003.31 -> 2,769.97
8% build times osx-shippable opt nightly taskcluster-m5.4xlarge 3,323.22 -> 3,069.82
7% build times linux64-shippable opt nightly taskcluster-m5.4xlarge 3,116.36 -> 2,912.62

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25856

Flags: needinfo?(mh+mozilla)

System requirements updated for 78.0a1.

Flags: needinfo?(jcristau)

And flagging this for relnotes, too...

Release Note Request (optional, but appreciated)
[Why is this notable]: system requirements bumped on linux
[Affects Firefox for Android]: no
[Suggested wording]: The minimal system requirements on Linux have been updated. Firefox now needs GNU libc 2.17, libstdc++ 4.8.1 and GTK+ 3.14 or newer versions.
[Links (documentation, blog post, etc)]:

relnote-firefox: --- → ?

(In reply to Alexandru Ionescu (needinfo me) :alexandrui from comment #20)

Hi Mike. Do the improvements below make sense for your patches?
== Change summary for alert #25856 (as of Thu, 07 May 2020 09:10:23 GMT) ==

Improvements:

13% build times osx-shippable opt nightly taskcluster-c5d.4xlarge 3,240.26 -> 2,813.73
11% build times osx-shippable opt nightly taskcluster-c5.4xlarge 3,284.32 -> 2,912.96
8% build times linux64-shippable opt nightly taskcluster-c5d.4xlarge 3,003.31 -> 2,769.97
8% build times osx-shippable opt nightly taskcluster-m5.4xlarge 3,323.22 -> 3,069.82
7% build times linux64-shippable opt nightly taskcluster-m5.4xlarge 3,116.36 -> 2,912.62

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25856

It's plausible (especially since the first backout showed a corresponding regression), although somewhat unexpected. My guess would be that the upgrade of glibc brought some perf improvements (possibly malloc, possibly in the form of SSEn or AVX-optimized routines).

Flags: needinfo?(mh+mozilla)
Regressions: 1637140

(In reply to Mike Hommey [:glandium] from comment #24)

It's plausible (especially since the first backout showed a corresponding regression), although somewhat unexpected. My guess would be that the upgrade of glibc brought some perf improvements (possibly malloc, possibly in the form of SSEn or AVX-optimized routines).

I am asking this about an improvement usually because there are multiple possible culprits and, for improvements, it is not feasible to do bisections. So is faster (actually often less time and resource consuming) to ask in the bug. The other possible culprits looking at their product/components are Bug 1636003 or Bug 1635990.

None of those changed the code in a way that would explain this improvement. It's more likely that it is this one.

No longer regressions: 1637140
Regressions: 1643258
You need to log in before you can comment on or make changes to this bug.