1874059 - undefined hidden symbols in std::basic_string with libc++ 18

Status: UNCONFIRMED

(Firefox Build System :: General, defect, P3)

Description

[Violet Purcell]

I'm currently testing libc++ 18, and after fixing the bug mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1873379, linking libxul fails with a bunch of undefined hidden symbol errors from LLD.

12:29.66 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::assign(char const*)
12:29.66 >>> referenced by Unified_cpp_linux_launch0.cpp
12:29.66 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../security/sandbox/linux/launch/Unified_cpp_linux_launch0.o:(mozilla::SandboxLaunchPrepare(GeckoProcessType, base::LaunchOptions*, mozilla::ipc::SandboxingKind))
12:29.66 >>> referenced by Unified_cpp_linux_launch0.cpp
12:29.66 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../security/sandbox/linux/launch/Unified_cpp_linux_launch0.o:(mozilla::SandboxLaunchPrepare(GeckoProcessType, base::LaunchOptions*, mozilla::ipc::SandboxingKind))
12:29.66 >>> referenced by Unified_cpp_linux_launch0.cpp
12:29.66 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../security/sandbox/linux/launch/Unified_cpp_linux_launch0.o:(mozilla::SandboxLaunchPrepare(GeckoProcessType, base::LaunchOptions*, mozilla::ipc::SandboxingKind))
12:29.66 >>> referenced 295 more times
12:29.66 >>> did you mean: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::assign(char const*, unsigned long)
12:29.66 >>> defined in: /lib/libc++_shared.so
12:29.66 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::append(char const*, unsigned long)
12:29.66 >>> referenced by Unified_cpp_linux_launch0.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../security/sandbox/linux/launch/Unified_cpp_linux_launch0.o:(mozilla::SandboxLaunchPrepare(GeckoProcessType, base::LaunchOptions*, mozilla::ipc::SandboxingKind))
12:29.67 >>> referenced by Unified_cpp_gfx_ots_src2.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/ots/src/Unified_cpp_gfx_ots_src2.o:(ots::OpenTypeNAME::Serialize(ots::OTSStream*))
12:29.67 >>> referenced by Unified_cpp_gfx_ots_src2.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/ots/src/Unified_cpp_gfx_ots_src2.o:(ots::OpenTypeNAME::Serialize(ots::OTSStream*))
12:29.67 >>> referenced 336 more times
12:29.67 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::resize(unsigned long, char)
12:29.67 >>> referenced by glslang_lex_autogen.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/angle/targets/translator/glslang_lex_autogen.o:(floatsuffix_check(sh::TParseContext*))
12:29.67 >>> referenced by Unified_cpp_gfx_2d1.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/2d/Unified_cpp_gfx_2d1.o:(bool mozilla::gfx::RecordedEvent::DoWithEvent<mozilla::gfx::EventStream>(mozilla::gfx::EventStream&, mozilla::gfx::RecordedEvent::EventType, std::__1::function<bool (mozilla::gfx::RecordedEvent*)> const&))
12:29.67 >>> referenced by Unified_cpp_gfx_2d1.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/2d/Unified_cpp_gfx_2d1.o:(bool mozilla::gfx::RecordedEvent::DoWithEvent<mozilla::gfx::EventStream>(mozilla::gfx::EventStream&, mozilla::gfx::RecordedEvent::EventType, std::__1::function<bool (mozilla::gfx::RecordedEvent*)> const&))
12:29.67 >>> referenced 65 more times
12:29.67 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
12:29.67 >>> referenced by DirectiveParser.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/angle/targets/preprocessor/DirectiveParser.o:(angle::pp::DirectiveParser::parseDefine(angle::pp::Token*))
12:29.67 >>> referenced by DirectiveParser.cpp
12:29.67 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/angle/targets/preprocessor/DirectiveParser.o:(angle::pp::DirectiveParser::parsePragma(angle::pp::Token*))
12:29.68 >>> referenced by DirectiveParser.cpp
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/angle/targets/preprocessor/DirectiveParser.o:(angle::pp::DirectiveParser::parsePragma(angle::pp::Token*))
12:29.68 >>> referenced 304 more times
12:29.68 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::push_back(char)
12:29.68 >>> referenced by Unified_cpp_xpcom_base1.cpp
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../xpcom/base/Unified_cpp_xpcom_base1.o:(std::__1::basic_istream<char, std::__1::char_traits<char>>& std::__1::getline[abi:nn180000]<char, std::__1::char_traits<char>, std::__1::allocator<char>>(std::__1::basic_istream<char, std::__1::char_traits<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, char))
12:29.68 >>> referenced by Telemetry.cpp
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../components/telemetry/Telemetry.o:(std::__1::basic_istream<char, std::__1::char_traits<char>>& std::__1::operator>>[abi:nn180000]<char, std::__1::char_traits<char>, std::__1::allocator<char>>(std::__1::basic_istream<char, std::__1::char_traits<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&))
12:29.68 >>> referenced by parse_context.cc
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../components/protobuf/parse_context.o:(google::protobuf::internal::WriteVarint(unsigned int, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*))
12:29.68 >>> referenced 224 more times
12:29.68 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::append(char const*)
12:29.68 >>> referenced by Unified_cpp_audio_processing_gn0.cpp
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/modules/audio_processing/audio_processing_gn/Unified_cpp_audio_processing_gn0.o:(webrtc::AudioProcessingImpl::WriteAecDumpConfigMessage(bool))
12:29.68 >>> referenced by Unified_cpp_audio_processing_gn0.cpp
12:29.68 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/modules/audio_processing/audio_processing_gn/Unified_cpp_audio_processing_gn0.o:(webrtc::AudioProcessingImpl::WriteAecDumpConfigMessage(bool))
12:29.68 >>> referenced by Unified_cpp_audio_processing_gn0.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/modules/audio_processing/audio_processing_gn/Unified_cpp_audio_processing_gn0.o:(webrtc::AudioProcessingImpl::WriteAecDumpConfigMessage(bool))
12:29.69 >>> referenced 1382 more times
12:29.69 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::insert(unsigned long, char const*)
12:29.69 >>> referenced by Unified_cpp_rtc_base_logging_gn0.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/rtc_base/logging_gn/Unified_cpp_rtc_base_logging_gn0.o:(rtc::LogSink::OnLogMessage(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, rtc::LoggingSeverity, char const*))
12:29.69 >>> referenced by Unified_cpp_rtc_base_logging_gn0.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/rtc_base/logging_gn/Unified_cpp_rtc_base_logging_gn0.o:(rtc::LogSink::OnLogMessage(std::__1::basic_string_view<char, std::__1::char_traits<char>>, rtc::LoggingSeverity, char const*))
12:29.69 >>> referenced by Unified_cpp_rtc_base_logging_gn0.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../third_party/libwebrtc/rtc_base/logging_gn/Unified_cpp_rtc_base_logging_gn0.o:(rtc::LogSink::OnLogMessage(std::__1::basic_string_view<char, std::__1::char_traits<char>>, rtc::LoggingSeverity, char const*))
12:29.69 >>> referenced 15 more times
12:29.69 ld.lld: error: undefined hidden symbol: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::insert(unsigned long, char const*, unsigned long)
12:29.69 >>> referenced by Unified_cpp_hunspell_src0.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../extensions/spellcheck/hunspell/src/Unified_cpp_hunspell_src0.o:(strlinecat(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&))
12:29.69 >>> referenced by VariablePacker.cpp
12:29.69 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../gfx/angle/targets/translator/VariablePacker.o:(sh::(anonymous namespace)::ExpandStructArrayVariable(sh::ShaderVariable const&, unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<sh::ShaderVariable, std::__1::allocator<sh::ShaderVariable>>*))
12:29.69 >>> referenced by Unified_cpp_dom_canvas0.cpp
12:29.70 >>>               /var/tmp/portage/www-client/firefox-121.0/work/firefox_build/toolkit/library/build/../../../dom/canvas/Unified_cpp_dom_canvas0.o:(mozilla::webgl::SanitizeRenderer(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&))

I don't think this is related to my small patch, but please tell me off if it is. I'm not sure what could have caused this upstream in libc++, but I got a tip that it may be related to how Firefox wraps the C++ STL headers.
Full build log: https://0x0.st/HIXW.log
Tarball produced by LLD's --reproduce: https://0x0.st/HI2S.tar.xz
For the reproduce tarball, use by running ld.lld @response.txt.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:ahochheiden, could you have a look please?

For more information, please visit BugBot documentation.

Comment 3

[Dimitry Andric]

I reported the same issue in https://github.com/llvm/llvm-project/issues/79027, to get upstream's opinion. It looks like upstream is not yet decided whether the approach with gcc_hidden.h and forcing a #pragma GCC visibility push(hidden) in front of every system header is supported at all.

For the FreeBSD port www/firefox, I submitted https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276746 (which also has the fix for bug 1873379), that effectively stops using gcc_hidden.h in front of every file, and switches to the more usual -fvisibility=hidden and -fvisibility-inlines-hidden compiler flags:

--- build/moz.configure/toolchain.configure.orig        2024-01-18 21:41:20.000000000 +0100
+++ build/moz.configure/toolchain.configure     2024-01-31 11:18:04.902430000 +0100
@@ -2150,7 +2150,7 @@ def visibility_flags(target, env):
 @depends(target, build_environment)
 def visibility_flags(target, env):
     if target.os != "WINNT":
-        if target.kernel == "Darwin":
+        if target.kernel == "Darwin" or target.kernel == "FreeBSD":
             return ("-fvisibility=hidden", "-fvisibility-inlines-hidden")
         return (
             "-I%s/system_wrappers" % os.path.join(env.dist),

However, this is really a bit of a quick hack to make it work on FreeBSD. Instead, I would suggest checking via the regular "does this compiler flag work" mechanism whether the -fvisibility=hidden and -fvisibility-inlines-hidden flags are supported, and to always use them over the -include gcc_hidden.h approach.