Closed Bug 1643258 Opened 4 months ago Closed 3 months ago

Firefox beta 78.0b1 requires glibc 2.18 on Linux, not 2.17 as announced

Categories

(Firefox Build System :: General, defect)

78 Branch
Unspecified
Linux
defect

Tracking

(firefox-esr68 unaffected, firefox-esr78 fixed, firefox77 unaffected, firefox78+ fixed, firefox79 fixed)

RESOLVED FIXED
mozilla79
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- fixed
firefox77 --- unaffected
firefox78 + fixed
firefox79 --- fixed

People

(Reporter: steven.common, Assigned: glandium)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files, 1 obsolete file)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Updated Firefox beta to 78.0b1 on Redhat Enterprise Linux 7 and tried to run it

Actual results:

Firefox no longer starts due to unannounced GLIBC 2.18 dependancy. GLIBC 2.18 is unavailable on Rehdat Enterprise linux 7 (2.17 max).

https://www.mozilla.org/en-US/firefox/78.0beta/system-requirements/

Requirement is given as GLIBC 2.17. Error messages when starting Firefox:
XPCOMGlueLoad error for file /usr/local/firefox/libxul.so: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /usr/local/firefox/libxul.so) Couldn't load XPCOM.

Expected results:

Firefox should have started normaly, as it did before updating to 78.0b1

Version: 68 Branch → 78 Branch

Hi,

We do not have right environment set up in order to reproduce this issue but I will set the component for it and maybe one of our developers will be able to reproduce it on their end.

Thanks for the report.

Best regards,

Clara.

Component: Untriaged → Widget: Gtk
Product: Firefox → Core

I have the exact same problem on an up-to-date Centos 7 machine, which is probably easier to set up, since it's free.
This bug also applies to FF 79 Nightly.

Is there any chance that you could run mozregression to figure out what regressed this?

Flags: needinfo?(steven.common)

$ pip3 install --upgrade mozregression
$ ~/.local/bin/mozregression --good 77 --bad 78

Ok then, this is the result of mozregression:

6:08.87 INFO: Narrowed integration regression window from [6f03149b, 52cf30bf] (3 builds) to [6f03149b, 8621fde1] (2 builds) (~1 steps left)
6:08.87 INFO: No more integration revisions, bisection finished.
6:08.87 INFO: Last good revision: 6f03149b52094f6af4d842bfc40ce1ddf5a3b157
6:08.87 INFO: First bad revision: 8621fde19c3154f11758d8f9e0d2a22cfff1efed
6:08.87 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=6f03149b52094f6af4d842bfc40ce1ddf5a3b157&tochange=8621fde19c3154f11758d8f9e0d2a22cfff1efed

Thanks!

Flags: needinfo?(steven.common) → needinfo?(mh+mozilla)
Keywords: regression
OS: Unspecified → Linux
Regressed by: 1634204
Component: Widget: Gtk → General
Product: Core → Firefox Build System

Can you attach the full log when running with LD_DEBUG=all set in the environment?

Flags: needinfo?(mh+mozilla) → needinfo?(steven.common)
Attached file ld-debug.zip

log from LD_DEBUG=all on Centos 7

do you still "needinfo" from me or is Poitr's info sufficient (I think it is)?
I won't have access to the run environment on Red-hot Linux until Moday 15 June

Flags: needinfo?(steven.common)
Flags: needinfo?(mh+mozilla)

When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary get a weak versioned
symbol reference.

It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.

Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.

This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).

We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.

Assignee: nobody → mh+mozilla
Flags: needinfo?(mh+mozilla)

This bug would be a lot less frustrating if I could revert to v77 and halt all updates. However, even after setting "Ask before updating", firefox updates to v78 in the next start and dies on CentOS7.

As others report:

XPCOMGlueLoad error for file /opt/firefox-developer/firefox/libxul.so:
/lib64/libc.so.6: version `GLIBC_2.18' not found (required by /opt/firefox-developer/firefox/libxul.so)
Couldn't load XPCOM.

Thanks for working on this - I so miss firefox developer edition!

I can imagine that many users are just seeing Firefox fail to start (since starting from GUI does not show this error message)... and sadly moving on to other browsers. Perhaps consider a GUI popup, a death rattle, in the event of a system level fail like this.

Even better, automatically revert to a prior version of firefox(!!!) and transmit a message to mozilla proactively.

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/5c7f4c2de55b
Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/439eee50f354
Disable stdc++ compat on base-toolchain builds. r=firefox-build-system-reviewers,rstewart
https://hg.mozilla.org/integration/autoland/rev/4c359d743fb2
Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
Status: UNCONFIRMED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla79
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: FIXED → ---
Attachment #9157185 - Attachment is obsolete: true
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/b707f591bef5
Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/98efceb86ec5
Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
Status: REOPENED → RESOLVED
Closed: 3 months ago3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla79
Blocks: 1646993
Duplicate of this bug: 1646993

Comment on attachment 9156875 [details]
Bug 1643258 - Avoid the use of the __cxa_thread_atexit_impl symbol.

Beta/Release Uplift Approval Request

  • User impact if declined: Cannot start Firefox on some older Linux systems
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Trick to avoid using some symbols, which changes no functionality. If it broke anything, it would have been detected already. While there were several landings/backouts, none of them involved the trick breaking things, but some non-shipped builds using another symbol that wasn't expected to be used, for which another trick was added.
  • String changes made/needed: N/A
Flags: needinfo?(mh+mozilla)
Attachment #9156875 - Flags: approval-mozilla-beta?

Comment on attachment 9156875 [details]
Bug 1643258 - Avoid the use of the __cxa_thread_atexit_impl symbol.

approved for 78 rc1

Attachment #9156875 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Duplicate of this bug: 1645099
You need to log in before you can comment on or make changes to this bug.