Closed Bug 1827095 Opened 2 years ago Closed 2 years ago

Windows 11 PGO build fails with undefined symbol: __llvm_profile_instrument_memop

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86_64
Windows 11
Type:
defect

Tracking

(firefox114 fixed)

Status:
RESOLVED FIXED
Milestone:
114 Branch
Tracking Flags:
Tracking Status
firefox114 --- fixed

People

(Reporter: gzyfqh, Assigned: glandium)

Details

Attachments

(8 files)

output
148.16 KB, text/plain
Details
error
456.04 KB, text/plain
Details
config.status
38.60 KB, text/plain
Details
Snipaste_2023-04-14_10-10-44.png
71.25 KB, image/png
Details
Snipaste_2023-04-14_10-13-01.png
133.37 KB, image/png
Details
new output
550.68 KB, text/plain
Details
Bug 1827095 - Put the clang runtime directory first in LIB.
48 bytes, text/x-phabricator-request
Details | Review
new build error
44.20 KB, text/plain
Details

+++ This bug was initially created as a clone of Bug #1718171 +++

Steps to reproduce:

Building Firefox on Windows 11 fails after turning on PGO with ac_add_options MOZ_PGO=1

Actual results:

security/manager/ssl/ipcclientcerts/force-cargo-library-build
0:11.51 security/nss/lib/crmf/crmf.lib
0:11.52 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:11.52 >>> referenced by pbproxy.obj:(void __cdecl raiseError(unsigned long, class std::basic_string<wchar_t, struct std::char_traits<wchar_t>, class std::allocator<wchar_t>>))
0:11.53 >>> referenced by pbproxy.obj:(void __cdecl raiseError(unsigned long, class std::basic_string<wchar_t, struct std::char_traits<wchar_t>, class std::allocator<wchar_t>>))
0:11.55 >>> referenced by pbproxy.obj:(void __cdecl raiseError(unsigned long, class std::basic_string<wchar_t, struct std::char_traits<wchar_t>, class std::allocator<wchar_t>>))
0:11.57 >>> referenced 7 more times
0:11.58 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:421: ../../../dist/bin/private_browsing.exe] Error 1
0:11.59 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: browser/app/pbproxy/target] Error 2
0:11.59 mozmake[3]: *** Waiting for unfinished jobs....
0:11.61 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:11.63 >>> referenced by ....\memory\build\Unified_cpp_memory_build0.obj:(private: void * __cdecl arena_t::PallocHuge(unsigned __int64, unsigned __int64, bool))
0:11.63 >>> referenced by ....\memory\build\Unified_cpp_memory_build0.obj:(private: void * __cdecl arena_t::RallocSmallOrLarge(void *, unsigned __int64, unsigned __int64))
0:11.63 >>> referenced by ....\memory\build\Unified_cpp_memory_build0.obj:(private: void * __cdecl arena_t::RallocSmallOrLarge(void *, unsigned __int64, unsigned __int64))
0:11.64 >>> referenced 382 more times
0:11.66 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:535: mozglue.dll] Error 1
0:11.68 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: mozglue/build/target] Error 2
0:11.93 dom/media/fake-cdm/fake.dll.res
0:11.93 toolkit/mozapps/notificationserver/notificationserver.dll.res
0:11.94 dom/media/gmp-plugin-openh264/fakeopenh264.dll.res
0:11.94 memory/replace/logalloc/replay/logalloc-replay.exe.res
0:11.96 media/ffvpx/libavutil/mozavutil.dll.res
0:12.00 accessible/ipc/win/handler/AccessibleHandler.dll.res
0:12.46 toolkit/mozapps/notificationserver/notificationserver.dll

0:12.52 dom/media/fake-cdm/fake.dll
0:12.52 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:12.54 >>> referenced by Unified_cpp_notificationserver0.obj:(private: class mozilla::Maybe<struct ToastArgs> __cdecl NotificationCallback::ParseToastArguments(wchar_t const *))
0:12.58 >>> referenced by Unified_cpp_notificationserver0.obj:(private: class mozilla::Maybe<struct ToastArgs> __cdecl NotificationCallback::ParseToastArguments(wchar_t const *))
0:12.60 >>> referenced by Unified_cpp_notificationserver0.obj:(private: class mozilla::Maybe<struct ToastArgs> __cdecl NotificationCallback::ParseToastArguments(wchar_t const *))
0:12.61 >>> referenced 61 more times
0:12.63 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:535: notificationserver.dll] Error 1
0:12.65 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: toolkit/mozapps/notificationserver/target] Error 2
0:12.65 dom/media/gmp-plugin-openh264/fakeopenh264.dll
0:12.67 media/ffvpx/libavutil/mozavutil.dll
0:12.67 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.69 lld-link: warning: ignoring unknown argument '-mavx'
0:12.69 lld-link: warning: ignoring unknown argument '-maes'
0:12.71 lld-link: warning: ignoring unknown argument '-O3'
0:12.73 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.73 lld-link: warning: ignoring unknown argument '-mavx'
0:12.73 lld-link: warning: ignoring unknown argument '-maes'
0:12.74 lld-link: warning: ignoring unknown argument '-O3'
0:12.76 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.76 lld-link: warning: ignoring unknown argument '-mavx'
0:12.78 lld-link: warning: ignoring unknown argument '-maes'
0:12.78 lld-link: warning: ignoring unknown argument '-O3'
0:12.78 accessible/ipc/win/handler/AccessibleHandler.dll
0:12.80 memory/replace/logalloc/replay/logalloc-replay.exe
0:12.81 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.82 lld-link: warning: ignoring unknown argument '-mavx'
0:12.83 lld-link: warning: ignoring unknown argument '-maes'
0:12.83 lld-link: warning: ignoring unknown argument '-O3'
0:12.85 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:12.85 >>> referenced by avsscanf.obj:(ffstring_read)
0:12.87 >>> referenced by avstring.obj:(av_strnstr)
0:12.87 >>> referenced by bprint.obj:(av_bprint_alloc)
0:12.88 >>> referenced 47 more times
0:12.89 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:535: mozavutil.dll] Error 1
0:12.91 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: media/ffvpx/libavutil/target] Error 2
0:12.93 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.94 lld-link: warning: ignoring unknown argument '-mavx'
0:12.96 lld-link: warning: ignoring unknown argument '-maes'
0:12.96 lld-link: warning: ignoring unknown argument '-O3'
0:12.96 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:12.97 >>> referenced by Replay.obj:(public: class Buffer __cdecl FdReader::ReadLine(void))
0:12.97 >>> referenced by Replay.obj:(public: void __cdecl Replay::malloc(class Buffer &, class Buffer &))
0:12.97 >>> referenced by Replay.obj:(public: void __cdecl Replay::posix_memalign(class Buffer &, class Buffer &))
0:12.99 >>> referenced 35 more times
0:12.99 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:421: ../../../../dist/bin/logalloc-replay.exe] Error 1
0:13.01 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: memory/replace/logalloc/replay/target] Error 2
0:13.03 lld-link: error: undefined symbol: __llvm_profile_instrument_memop
0:13.03 >>> referenced by HandlerChildEnumerator.obj:(private: __cdecl mozilla::a11y::HandlerChildEnumerator::HandlerChildEnumerator(class mozilla::a11y::HandlerChildEnumerator const &))
0:13.04 >>> referenced by HandlerChildEnumerator.obj:(private: __cdecl mozilla::a11y::HandlerChildEnumerator::HandlerChildEnumerator(class mozilla::a11y::HandlerChildEnumerator const &))
0:13.06 >>> referenced by HandlerChildEnumerator.obj:(private: long __cdecl mozilla::a11y::HandlerChildEnumerator::MaybeCacheChildren(void))
0:13.07 mozmake[4]: *** [E:/work/mozilla-unified/config/rules.mk:535: AccessibleHandler.dll] Error 1
0:13.07 mozmake[3]: *** [E:/work/mozilla-unified/config/recurse.mk:72: accessible/ipc/win/handler/target] Error 2
0:16.03 Compiling nom v7.1.3
0:17.53 Compiling cexpr v0.6.0
0:17.84 Compiling bindgen v0.64.0
0:26.34 Compiling pkcs11-bindings v0.1.5
0:27.26 Compiling rsclientcerts v0.1.0 (E:\work\mozilla-unified\security\manager\ssl\rsclientcerts)
0:28.28 Compiling ipcclientcerts-static v0.1.0 (E:\work\mozilla-unified\security\manager\ssl\ipcclientcerts)
0:32.85 Finished release [optimized] target(s) in 21.30s
0:33.02 mozmake[2]: *** [E:/work/mozilla-unified/config/recurse.mk:34: compile] Error 2
0:33.02 mozmake[1]: *** [E:/work/mozilla-unified/config/rules.mk:361: default] Error 2
0:33.02 mozmake: *** [client.mk:60: build] Error 2
0:33.03 316 compiler warnings present.

The compiler has some errors related to PGO during the linking phase.

What revision of mozilla-unified are you building from?

Flags: needinfo?(gzyfqh)
Flags: needinfo?(gzyfqh)
Version: Firefox 113 → Trunk

(In reply to Mike Hommey [:glandium] from comment #2)

What revision of mozilla-unified are you building from?

changeset: 659415:25045b498bff
tag: tip
fxtree: central
user: Iulian Moraru <imoraru@mozilla.com>
date: Tue Apr 11 10:10:41 2023 +0300
summary: Backed out 6 changesets (bug 1821512) for causing multiple failures. CLOSED TREE

I don't reproduce, and there's no reason that the behavior would be different between windows 10 and windows 11.

These lines in the output you pasted:

0:12.93 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.94 lld-link: warning: ignoring unknown argument '-mavx'
0:12.96 lld-link: warning: ignoring unknown argument '-maes'
0:12.96 lld-link: warning: ignoring unknown argument '-O3'

suggest that you have an unconventional configuration. I would suggest you take a step back and start afresh with just ac_add_options MOZ_PGO=1. If you can still reproduce in that configuration, I'd ask you to attach the full output from ./mach build -v after ./mach clobber.

Attached file output —
(In reply to Mike Hommey [:glandium] from comment #4) > I don't reproduce, and there's no reason that the behavior would be different between windows 10 and windows 11. > > These lines in the output you pasted: > ``` > 0:12.93 lld-link: warning: ignoring unknown argument '-Wl,-O3' > 0:12.94 lld-link: warning: ignoring unknown argument '-mavx' > 0:12.96 lld-link: warning: ignoring unknown argument '-maes' > 0:12.96 lld-link: warning: ignoring unknown argument '-O3' > ``` > > suggest that you have an unconventional configuration. I would suggest you take a step back and start afresh with just `ac_add_options MOZ_PGO=1`. If you can still reproduce in that configuration, I'd ask you to attach the full output from `./mach build -v` after `./mach clobber`. 我调整了mozconfig文件里面的内容，仍然出现前面提及到的错误。 然后，我把mozconfig文件中除了`ac_add_options MOZ_PGO=1`之外的内容全部清除，发现出现了以下错误。

(In reply to Mike Hommey [:glandium] from comment #4)

I don't reproduce, and there's no reason that the behavior would be different between windows 10 and windows 11.

These lines in the output you pasted:

0:12.93 lld-link: warning: ignoring unknown argument '-Wl,-O3'
0:12.94 lld-link: warning: ignoring unknown argument '-mavx'
0:12.96 lld-link: warning: ignoring unknown argument '-maes'
0:12.96 lld-link: warning: ignoring unknown argument '-O3'

suggest that you have an unconventional configuration. I would suggest you take a step back and start afresh with just ac_add_options MOZ_PGO=1. If you can still reproduce in that configuration, I'd ask you to attach the full output from ./mach build -v after ./mach clobber.

I adjusted the contents inside the mozconfig file and still got the error mentioned earlier.

Then, I cleared all the contents of the mozconfig file except ac_add_options MOZ_PGO=1 and found the following error.

The error message has been appended to the attachment

Try removing E:/work/mozilla-unified/mozilla-central/obj-x86_64-pc-windows-msvc entirely and start again.

Attached file error —
(In reply to Mike Hommey [:glandium] from comment #8) > Try removing E:/work/mozilla-unified/mozilla-central/obj-x86_64-pc-windows-msvc entirely and start again. There are still some compilation errors, the relevant information has been appended to the attachment

What is the output from C:/Users/Ho/.mozbuild/clang/bin/clang.exe --version? can you also attach the config.status file?

Flags: needinfo?(gzyfqh)
Attached file config.status —
(In reply to Mike Hommey [:glandium] from comment #10) > What is the output from `C:/Users/Ho/.mozbuild/clang/bin/clang.exe --version`? can you also attach the config.status file? C:/Users/Ho/.mozbuild/clang/bin/clang.exe --version clang version 16.0.0 (taskcluster-PylPJ8RqQ--l63r3tY-1_g) Target: x86_64-pc-windows-msvc Thread model: posix InstalledDir: C:\Users\Ho\.mozbuild\clang\bin
Flags: needinfo?(gzyfqh)

What does C:/Users/Ho/.mozbuild/indices/clang contain?

Flags: needinfo?(gzyfqh)

Also, what is in C:\Users\Ho\.mozbuild\clang\lib\clang\16\lib\windows

Flags: needinfo?(gzyfqh)

(In reply to Mike Hommey [:glandium] from comment #13)

Also, what is in C:\Users\Ho\.mozbuild\clang\lib\clang\16\lib\windows

Screenshots of the above two folders have been uploaded

Let's try something different. Remove E:/work/mozilla-unified/mozilla-central/obj-x86_64-pc-windows-msvc entirely, create the following mozconfig:

ac_add_options --enable-profile-generate
export LDFLAGS=-verbose

and then run the following commands:

./mach configure
./mach build pre-export export
./mach build -v -C . mozglue/build/target

and please attach the full output from these commands.

Flags: needinfo?(gzyfqh)
Attached file new output —

(In reply to Mike Hommey [:glandium] from comment #17)

Let's try something different. Remove E:/work/mozilla-unified/mozilla-central/obj-x86_64-pc-windows-msvc entirely, create the following mozconfig:

ac_add_options --enable-profile-generate
export LDFLAGS=-verbose

and then run the following commands:

./mach configure
./mach build pre-export export
./mach build -v -C . mozglue/build/target

and please attach the full output from these commands.

Still encountering errors, the output has been appended to the attachment

Flags: needinfo?(gzyfqh)

Ooooh, this is starting to make sense:

0:23.20 lld-link: Reading C:/PROGRA~1/MICROS~3/2022/COMMUN~1/VC/Tools/MSVC/1435~1.322/lib/x64\clang_rt.profile-x86_64.lib

There's a clang_rt in MSVC.

Try this patch:

--- a/build/moz.configure/windows.configure
+++ b/build/moz.configure/windows.configure
@@ -386,7 +386,7 @@ def lib_path_for(host_or_target):
                 onerror=lambda: None
             ).strip()
             if runtime_dir and os.path.exists(runtime_dir):
-                libs.append(runtime_dir)
+                libs.insert(0, runtime_dir)
         return libs
 
     return lib_path

(In reply to Mike Hommey [:glandium] from comment #20)

Ooooh, this is starting to make sense:

0:23.20 lld-link: Reading C:/PROGRA~1/MICROS~3/2022/COMMUN~1/VC/Tools/MSVC/1435~1.322/lib/x64\clang_rt.profile-x86_64.lib

There's a clang_rt in MSVC.

Try this patch:

--- a/build/moz.configure/windows.configure
+++ b/build/moz.configure/windows.configure
@@ -386,7 +386,7 @@ def lib_path_for(host_or_target):
                 onerror=lambda: None
             ).strip()
             if runtime_dir and os.path.exists(runtime_dir):
-                libs.append(runtime_dir)
+                libs.insert(0, runtime_dir)
         return libs
 
     return lib_path

Now that there are no more errors, I will try . /mach build and add the mozconfig parameter I used before

Assignee: nobody → mh+mozilla
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

3:34.95 build/clang-plugin/clang-plugin.dll
3:34.98 lld-link: warning: ignoring unknown argument '-IC:/Users/Ho/.mozbuild/clang/include'
3:34.98 lld-link: warning: ignoring unknown argument '-Xclang'
3:34.98 lld-link: warning: ignoring unknown argument '-std=c++17'
3:34.99 lld-link: warning: ignoring unknown argument '-EHs-c-'
3:34.99 lld-link: warning: ignoring unknown argument '-GR-'
3:34.99 lld-link: warning: ignoring unknown argument '-D_CRT_SECURE_NO_DEPRECATE'
3:34.99 lld-link: warning: ignoring unknown argument '-D_CRT_SECURE_NO_WARNINGS'
3:34.99 lld-link: warning: ignoring unknown argument '-D_CRT_NONSTDC_NO_DEPRECATE'
3:34.99 lld-link: warning: ignoring unknown argument '-D_CRT_NONSTDC_NO_WARNINGS'
3:34.99 lld-link: warning: ignoring unknown argument '-D_SCL_SECURE_NO_DEPRECATE'
3:34.99 lld-link: warning: ignoring unknown argument '-D_SCL_SECURE_NO_WARNINGS'
3:35.00 lld-link: warning: ignoring unknown argument '-DUNICODE'
3:35.00 lld-link: warning: ignoring unknown argument '-D_UNICODE'
3:35.00 lld-link: warning: ignoring unknown argument '-D__STDC_CONSTANT_MACROS'
3:35.00 lld-link: warning: ignoring unknown argument '-D__STDC_FORMAT_MACROS'
3:35.00 lld-link: warning: ignoring unknown argument '-D__STDC_LIMIT_MACROS'
3:35.01 lld-link: warning: ignoring unknown argument '-GR-'
3:35.01 lld-link: warning: ignoring unknown argument '-EHsc'
3:35.01 lld-link: warning: ignoring unknown argument '-O3'
3:35.01 lld-link: warning: ignoring unknown argument '-mavx'
3:35.01 lld-link: warning: ignoring unknown argument '-maes'
3:35.15 lld-link: error: undefined symbol: protected: __cdecl llvm::SmallVectorBase<unsigned int>::SmallVectorBase<unsigned int>(void *, unsigned __int64)
3:35.15 >>> referenced by host_ArithmeticArgChecker.obj:(protected: __cdecl llvm::SmallVectorTemplateCommon<class clang::CharSourceRange, void>::SmallVectorTemplateCommon<class clang::CharSourceRange, void>(unsigned __int64))
3:35.15 >>> referenced by host_ArithmeticArgChecker.obj:(protected: __cdecl llvm::SmallVectorTemplateCommon<class clang::FixItHint, void>::SmallVectorTemplateCommon<class clang::FixItHint, void>(unsigned __int64))
3:35.15 >>> referenced by host_CanRunScriptChecker.obj:(protected: __cdecl llvm::SmallVectorTemplateCommon<class clang::DynTypedNode, void>::SmallVectorTemplateCommon<class clang::DynTypedNode, void>(unsigned __int64))
3:35.15 >>> referenced 3 more times
3:35.15 lld-link: error: undefined symbol: protected: __cdecl llvm::SmallVectorBase<unsigned __int64>::SmallVectorBase<unsigned __int64>(void *, unsigned __int64)
3:35.15 >>> referenced by host_DanglingOnTemporaryChecker.obj:(protected: __cdecl llvm::SmallVectorTemplateCommon<char, void>::SmallVectorTemplateCommon<char, void>(unsigned __int64))
3:35.15 >>> referenced by host_ExplicitImplicitChecker.obj
3:35.16 >>> referenced by host_ExplicitOperatorBoolChecker.obj
3:35.16 mozmake[4]: *** [E:/work/mozilla-unified/mozilla-central/config/rules.mk:521: clang-plugin.dll] Error 1
3:35.16 mozmake[3]: *** [E:/work/mozilla-unified/mozilla-central/config/recurse.mk:72: build/clang-plugin/host] Error 2
3:35.17 mozmake[2]: *** [E:/work/mozilla-unified/mozilla-central/config/recurse.mk:34: compile] Error 2
3:35.17 mozmake[1]: *** [E:/work/mozilla-unified/mozilla-central/config/rules.mk:361: default] Error 2
3:35.17 mozmake: *** [client.mk:60: build] Error 2
3:35.20 7 compiler warnings present.

new error

(In reply to Mike Hommey [:glandium] from comment #20)

Ooooh, this is starting to make sense:

0:23.20 lld-link: Reading C:/PROGRA~1/MICROS~3/2022/COMMUN~1/VC/Tools/MSVC/1435~1.322/lib/x64\clang_rt.profile-x86_64.lib

There's a clang_rt in MSVC.

Try this patch:

--- a/build/moz.configure/windows.configure
+++ b/build/moz.configure/windows.configure
@@ -386,7 +386,7 @@ def lib_path_for(host_or_target):
                 onerror=lambda: None
             ).strip()
             if runtime_dir and os.path.exists(runtime_dir):
-                libs.append(runtime_dir)
+                libs.insert(0, runtime_dir)
         return libs
 
     return lib_path

ping Mike@

Attached file new build error —

Remove --enable-clang-plugin from your mozconfig.

Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/4adeab79414b Put the clang runtime directory first in LIB. r=firefox-build-system-reviewers,andi

(In reply to Mike Hommey [:glandium] from comment #26)

Remove --enable-clang-plugin from your mozconfig.

The problem has been solved, thank you very much

https://hg.mozilla.org/mozilla-central/rev/4adeab79414b

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
status-firefox114: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 114 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: