Investigate upgrading runtime requirements on Linux
Categories
(Firefox Build System :: General, task)
Tracking
(relnote-firefox 78+, firefox78 fixed)
People
(Reporter: glandium, Assigned: glandium)
References
Details
Attachments
(5 files)
Last upgrade was bug 1536307, in time for esr68. With esr78 being close, it's about the right time to start thinking about this. This was initially brought up in bug 1633778 for Thunderbird, where I noted the following:
The Linux compat table would need to be refreshed and more investigation done, but at quick glance, it seems that bumping to 5.0 /might/ be a reasonable thing to do, even for Firefox.
Then
(...) if we're going to bump, we might as well explore reasonable options. It seems the only distro that has libstdc++ < 5 and that is still currently supported by Firefox and not EOL is Debian 8, which EOLs in June, so before 78 is actually released. Bumping to 5 would remove the need for _GLIBCXX_USE_CXX11_ABI=0.
Assignee | ||
Comment 1•5 years ago
|
||
So, one thing that I missed is that RHEL7 is listed as having libstdc++ 4.8, so 5.0 wouldn't work there.
Assignee | ||
Comment 2•5 years ago
|
||
4.8.5, exactly, which is up to GLIBCXX_3.4.19.
Assignee | ||
Comment 3•5 years ago
|
||
One data point: we bumped the runtime requirement to libstdc++ 4.6 with Firefox 50, in November 2016, 6 years after the release of RHEL6, which is, actually not even EOL yet. 78 would release 6 years after the release of RHLE7. However, November 2016 was also 2 years after the release of RHEL7, while we're only 1 year after the release of RHEL8.
Now, something else that is interesting to look at is, eliminating distros that are EOL or soon EOL (i.e. Debian 8):
- the smallest version of glibc is 2.17 if we stick to supporting RHEL7, or 2.22 otherwise (we currently require 2.12)
- the smallest version of glib is 2.42 if we stick to supporting RHEL7, or 2.48 otherwise (we currently require 2.22)
- the smallest version of gtk+3 is 3.14 if we stick to supporting RHEL7, or 3.18 otherwise (we currently require 3.4)
- the smallest version of pixman is 0.32 if we stick to supporting RHEL7, or 0.33 otherwise (we currently require 0.19.2)
In any case, we might be able to switch the builds to a Debian 8 baseline rather than the current Debian 7 (although Debian 8 comes with glibc 2.19, so there may be some problem there, depending on the symbols we use from glibc). Time to revive my patches to build on Debian 8.
Assignee | ||
Comment 4•5 years ago
|
||
20190228 is the last snapshot of Debian wheezy available on dockerhub.
Assignee | ||
Comment 5•5 years ago
|
||
We currently check that the binaries we ship are not using symbol
versions of system libraries that would not be available on some older
systems. In some cases, however, we may get dependencies on newer symbol
versions in the form of weak symbols, that are checked for at runtime.
This happens with __cxa_thread_atexit_impl when building against a glibc
newer than 2.18, and the supporting code in Rust libstd actually checks
at runtime whether the weak symbol is resolved before using it.
Assignee | ||
Comment 6•5 years ago
|
||
Assignee | ||
Comment 7•5 years ago
|
||
This gets rid for the need of a number of local packages (mostly related
to Gtk+3). One exception is that we now need a 32-bits version of the
xz-utils package, some -dev package depends on it, and that dependency
can't be fulfilled in the 32-bits image because we already have the
64-bits backport installed, which conflicts with it (we need both
32-bits and 64-bits package to be at the same version when installed).
The system binutils fails to link clang-7 for some reason, so we now use
our toolchain binutils instead, like we already do for newer versions of
clang.
The debian-packages docker image now needs an explicit installation of
git, because it's not pulled in via the recommends of some other
package.
For some reason, snapshot.debian.org doesn't contain the jessie-backports
archive at the same location as others, and only has a few snapshots of
the archive.
Assignee | ||
Comment 8•5 years ago
|
||
While we're going to rebuild all the Debian-based docker images, we
might as well add one corrective thing that is going to be required
later on, and would require another rebuild, while doing it now would
skip that.
On Debian 10, the python3 distutils module is in a separate package,
python3-distutils, which didn't exist in previous Debian releases,
because back then the distutils module was part of the python3 package.
Fortunately, the python3-distutils-extra package existed on all the
Debian versions we use and depends on the distutils module, so it is a
convenient way to have the python3-distutils package pulled in without
altering the list of packages to install based on the Debian version.
Updated•5 years ago
|
Comment 9•5 years ago
|
||
I ran a Thunderbird try build with these changes today, no immediate problems.
Comment 10•5 years ago
|
||
Comment 11•5 years ago
|
||
Comment 12•5 years ago
|
||
Comment 13•5 years ago
|
||
Comment 14•5 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/e6a1b861522b
https://hg.mozilla.org/mozilla-central/rev/71b00ec52717
https://hg.mozilla.org/mozilla-central/rev/cf742676d316
https://hg.mozilla.org/mozilla-central/rev/9d6acea5d432
Comment 15•5 years ago
|
||
Comment 16•5 years ago
|
||
Backed out for valgrind bustages
Backout link: https://hg.mozilla.org/integration/autoland/rev/590613078c743f089f01b3f43118524e9553dd0b
Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301009949&repo=autoland&lineNumber=57506
Assignee | ||
Updated•5 years ago
|
Comment 17•5 years ago
|
||
Updated•5 years ago
|
Comment 18•5 years ago
|
||
bugherder |
Assignee | ||
Comment 19•5 years ago
|
||
Not sure if this needs a mention in the release notes, but this bug does need an update of the system requirements for 78: glibc -> 2.17, libstdc++ -> 4.8.1, Gtk+3 -> 3.14 (the latter may not be entirely true, as in, older versions might work, but meh, see commit message in https://hg.mozilla.org/mozilla-central/rev/cf742676d316).
Comment 20•5 years ago
|
||
Hi Mike. Do the improvements below make sense for your patches?
== Change summary for alert #25856 (as of Thu, 07 May 2020 09:10:23 GMT) ==
Improvements:
13% build times osx-shippable opt nightly taskcluster-c5d.4xlarge 3,240.26 -> 2,813.73
11% build times osx-shippable opt nightly taskcluster-c5.4xlarge 3,284.32 -> 2,912.96
8% build times linux64-shippable opt nightly taskcluster-c5d.4xlarge 3,003.31 -> 2,769.97
8% build times osx-shippable opt nightly taskcluster-m5.4xlarge 3,323.22 -> 3,069.82
7% build times linux64-shippable opt nightly taskcluster-m5.4xlarge 3,116.36 -> 2,912.62
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25856
Comment hidden (Intermittent Failures Robot) |
Comment 23•5 years ago
|
||
And flagging this for relnotes, too...
Release Note Request (optional, but appreciated)
[Why is this notable]: system requirements bumped on linux
[Affects Firefox for Android]: no
[Suggested wording]: The minimal system requirements on Linux have been updated. Firefox now needs GNU libc 2.17, libstdc++ 4.8.1 and GTK+ 3.14 or newer versions.
[Links (documentation, blog post, etc)]:
Assignee | ||
Comment 24•5 years ago
|
||
(In reply to Alexandru Ionescu (needinfo me) :alexandrui from comment #20)
Hi Mike. Do the improvements below make sense for your patches?
== Change summary for alert #25856 (as of Thu, 07 May 2020 09:10:23 GMT) ==Improvements:
13% build times osx-shippable opt nightly taskcluster-c5d.4xlarge 3,240.26 -> 2,813.73
11% build times osx-shippable opt nightly taskcluster-c5.4xlarge 3,284.32 -> 2,912.96
8% build times linux64-shippable opt nightly taskcluster-c5d.4xlarge 3,003.31 -> 2,769.97
8% build times osx-shippable opt nightly taskcluster-m5.4xlarge 3,323.22 -> 3,069.82
7% build times linux64-shippable opt nightly taskcluster-m5.4xlarge 3,116.36 -> 2,912.62For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25856
It's plausible (especially since the first backout showed a corresponding regression), although somewhat unexpected. My guess would be that the upgrade of glibc brought some perf improvements (possibly malloc, possibly in the form of SSEn or AVX-optimized routines).
Comment 25•5 years ago
•
|
||
(In reply to Mike Hommey [:glandium] from comment #24)
It's plausible (especially since the first backout showed a corresponding regression), although somewhat unexpected. My guess would be that the upgrade of glibc brought some perf improvements (possibly malloc, possibly in the form of SSEn or AVX-optimized routines).
I am asking this about an improvement usually because there are multiple possible culprits and, for improvements, it is not feasible to do bisections. So is faster (actually often less time and resource consuming) to ask in the bug. The other possible culprits looking at their product/components are Bug 1636003 or Bug 1635990.
Assignee | ||
Comment 26•5 years ago
|
||
None of those changed the code in a way that would explain this improvement. It's more likely that it is this one.
Updated•5 years ago
|
Description
•