1757378 - PGO build failed with "LLVM ERROR: Cannot select: intrinsic %llvm.x86.aesni.aesenc"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P3)

Description

[smsxgli]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0

Steps to reproduce:

build firefox locally on arch Linux. The first build succeed, but the second build with PGO and thin LTO failed.
In fact, this also happens on firefox 96, and even more older versions.
OS: arch linux, x86-64
rust version: tried both 1.58 and 1.59
clang, lld, llvm version: 13.0.1
CPU: Intel(R) Core(TM) i7-4790 CPU, haswell
CFLAGS: -march=native -mtune=native -O2 -pipe -fno-plt -fexceptions -fasynchronous-unwind-tables -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-protector-strong -fstack-clash-protection -fcf-protection
CXXFLAGS: $CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS
LDFLAGS: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now,-z,noexecstack
RUSTFLAGS: -C opt-level=2 -C target-cpu=native

Actual results:

The build failed with "LLVM ERROR: Cannot select: intrinsic %llvm.x86.aesni.aesenc"

Expected results:

I have post this error to both LLVM and rust project.
LLVM issue: https://github.com/llvm/llvm-project/issues/54055
rust issue: https://github.com/rust-lang/rust/issues/94326
Please take a look for more details.
In https://github.com/rust-lang/rust/issues/94326#issuecomment-1050688836, the rust developer want to get an "expansion" of what -C target-cpu=native is considered to be in compiling. Since I have no idea about firefox building system, I came here to ask for help. Please give me some idea about how to achieve it.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::General' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Hey, thanks for this report and the great follow-up with upstream LLVM and Rust, despite being relatively unfamiliar. Nice work!

the rust developer want to get an "expansion" of what -C target-cpu=native is considered to be in compiling.

Good question - I think the best technique here will be to discover which cargo command is being run right before the failure, then trying to repro it outside of the build system. Once that's possible, then tweaking cargo according to what the Rust folks are requesting should be easy.

To discover this, do the following:

1. (Set up PGO as you already have)
2. ./mach build (will fail due to the LLVM/Rust bug)
3. ./mach build -v -j 1. The -v flag tells the build system to, among other things, print each command that it runs (including clang and cargo with all of their options). The -j 1 flag tells the build system to run without parallelism, which will avoid overlapping of output, and will also guarantee that the last cargo command that's shown will be the one associated with your failure.
   • OTOH I can't remember where environment variables are shown, you might need to manually export [C|CXX|RUST|LD]FLAGS.
   • We're doing this after the regular ./mach build call to ensure that we're re-using as much work as possible, because building with -j 1 is very slow!
4. Manually run the last cargo command right before the failure, and see if it reproduces the issue. Hopefully yes.
   • If not, then please share the ./mach build -v -j 1 log here and we can dig into it.

Finally, this technique should work with PGO, but it might be a lot slower due to the lack of parallelism on the final build call.
Good luck!

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

I'm not able to repro this, so I'm going to WORKSFORME. :smsxgli, if you're able to provide more info, I'll reopen this.

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Hmm, wait, mixed up my bugs, my mistake.
Either way, it looks like this should be fixed upstream.
I'll keep this open until we update our Rust to include the fix.

Carry on 👍

Comment 5

[[:sergesanspaille]]

Both the llvm and rust bug mentioned in this thread have been fixed upstream, I consider this fixed too.