1640982 - rust 1.45 error: options `-C embed-bitcode=no` and `-C lto` are incompatible

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect, P3)

Description

[(Away)]

$ rustc --version -v
rustc 1.45.0-nightly (9310e3bd4 2020-05-21)
binary: rustc
commit-hash: 9310e3bd4f425f84fc27878ebf2bda1f30935a63
commit-date: 2020-05-21
host: x86_64-pc-windows-msvc
release: 1.45.0-nightly
LLVM version: 10.0

$ cat $MOZCONFIG
ac_add_options --enable-js-shell
ac_add_options --enable-warnings-as-errors
ac_add_options --enable-release

I'm pretty sure the -C lto is explicitly set by our build system, but I don't think we set embed-bitcode in either direction. Is this a new default from the recent bitcode optimizations?

Comment 1

[(Away)]

The error message:

 6:43.34 error: options `-C embed-bitcode=no` and `-C lto` are incompatible
 6:43.49 error: could not compile `gkrust`.

Comment 2

[(Away)]

I tried a naive fix, adding -Cembed-bitcode=yes at the same place in rust.mk where we add -Clto, but I got:

11:50.57 error: failed to get bitcode from object file for LTO (Bitcode section not found in object file)
11:50.57 error: aborting due to previous error
11:50.73 error: could not compile `gkrust`.

Comment 3

[Mike Hommey [:glandium]]

Try removing -Clto, we may not need it anymore.

Comment 4

[(Away)]

That worked, but could you explain why?

Comment 5

[Mike Hommey [:glandium]]

(In reply to :dmajor from comment #4)

That worked, but could you explain why?

Well comment 1 implies -Clto is conflicting somehow, if you remove it, you remove the conflict. The question though is whether it's still necessary or not. On PGO+LTO builds, I suspect it's not, because of cross-LTO, but there might be something subtle with libstd and libcore that could require it. On opt builds, though, it's necessary for https://searchfox.org/mozilla-central/rev/7dafc35406b9c945189c617d427f5458933fd3fb/config/makefiles/rust.mk#297-310.

Comment 6

[(Away)]

On opt builds, though, it's necessary for https://searchfox.org/mozilla-central/rev/7dafc35406b9c945189c617d427f5458933fd3fb/config/makefiles/rust.mk#297-310.

It sounds like opt builds are still stuck, then.

Comment 7

[Nicholas Nethercote [inactive]]

Relevant docs:

• embed-bitcode -- note especially the "Note"
• lto

.o files produced by Rust can take one of three forms:

• Just object code. You get this with -Cembed-bitcode=no.
• Object code with LLVM bitcode embedded in a special section: -Cembed-bitcode=yes
• Just LLVM bitcode. You get this with -Clinker-plugin-lto=<whatever>.

rustc defaults to -Cembed-bitcode=yes, but Cargo specifies -Cembed-bitcode=no whenever it can.

I admit I am hazy on exactly what -Clto does. Maybe Alex can explain.

The force-cargo-library-build thing you linked to in comment 5 says "The check only works when rust code is built with -Clto." Do you know why it only works like that? I wonder if whether using -Clinker-plugin-lto instead would work.

Comment 8

[Mike Hommey [:glandium]]

The check only works with -Clto and no cross-lto, because it relies on the .a containing actual compiled code, rather than LLVM IR, which cross-lto would produce. And it doesn't work without -Clto because in that case, unused code from libstd is not compiled out, and libstd contains all sort of networking code that the check doesn't want to see.

Comment 9

[Alex Crichton [:acrichto]]

Is -Clto set via RUSTFLAGS? If so this is likely https://github.com/rust-lang/rust/issues/71848. Cargo's logic about how to improve build times with bitcode generation has improved, so to set LTO the best way to do that is CARGO_PROFILE_<name>_LTO instead of RUSTFLAGS, so Cargo knows what's going on.

Comment 10

[(Away)]

I believe we're passing -Clto on the command line rather than via environment variables.

Comment 11

[Mike Hommey [:glandium]]

We pass it via cargo rustc -- -Clto, which I think is equivalent to RUSTFLAGS.

The cargo repo suggests CARGO_PROFILE_<name>_LTO was stabilized in cargo 0.44.0, which... I'm not sure what version of rust that corresponds to (1.43? 1.45?), but I don't think we can rely on that, because we currently have a minimum supported version of rust of 1.41, and we can't bump that until at least bug 1632735 is fixed.

Comment 12

[Mike Hommey [:glandium]]

However, we could choose to use CARGO_PROFILE_<name>_LTO instead of -Clto based on the cargo version we see during configure.

Comment 14

[Sebastien Marie [:semarie]]

rustc 1.45.0 is about to be released this week. I am hiting it with the prelease version and firefox78.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Rust 1.45 is out now and this is breaking builds. Can we please prioritize fixing this?

Comment 16

[Jan Alexander Steffens [:heftig]]

This has started hitting me. I tried just building with CARGO_PROFILE_RELEASE_LTO=true in the environment to work around it (assuming the duplicate -C lto flags won't conflict), but this failed the second build (using profile-use=cross and lto=cross) with:

 3:03.82 error: linking with `/build/firefox-nightly/src/mozilla-central/build/cargo-linker` failed: exit code: 1
 3:03.82   |
 3:03.82   = note: "/build/firefox-nightly/src/mozilla-central/build/cargo-linker" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.0.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.1.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.2.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.3.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.4.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.5.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.6.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.7.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.8.rcgu.o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.build_script_build.copei7wm-cgu.9.rcgu.o" "-o" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/build/libsqlite3-sys-b9ad497afc8f2c40/build_script_build-b9ad497afc8f2c40.49w4z2lhnqhieuz4.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps/libpkg_config-2de1420d96fe23b0.rlib" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps/libcc-1c49a2bb0e9412fd.rlib" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps/libnum_cpus-73c82eb139191966.rlib" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps/libjobserver-90819db86a8327c8.rlib" "/build/firefox-nightly/src/mozilla-central/obj-x86_64-pc-linux-gnu/release/deps/liblibc-198d52a6c562ea2c.rlib" "-Wl,--start-group" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-7755ccd2463efe47.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-8d74f0baed814295.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-50b49635f8455ad4.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-d24539552f34d87c.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-1dbacc51469ad140.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-6d2e39abb6026501.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-3faa150a9ff25d74.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-58aadcbe12b96605.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-716482fbaf990b48.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-5553c12cd349fcba.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-f98ead51d7c23e02.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-ccca25a27a62275a.rlib" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-15768ce9c8391919.rlib" "-Wl,--end-group" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d419cc7130b27c28.rlib" "-Wl,-Bdynamic" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-ldl" "-lutil"
 3:03.82   = note: /usr/bin/ld: /tmp/lto-llvm-a3864c.o: in function `std::sys::unix::fs::copy':
 3:03.82           /build/rust/src/rustc-1.45.0-src/src/libstd/sys/unix/fs.rs:1172: undefined reference to `__rust_probestack'
 3:03.82           clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
 3:03.83 error: aborting due to previous error
 3:03.85 error: could not compile `libsqlite3-sys`.

Comment 18

[Thomas Deutschmann]

Just dropping "-Clto" seems not work for ESR. With

--- old/config/makefiles/rust.mk
+++ new/config/makefiles/rust.mk
@@ -47,7 +47,7 @@ cargo_rustc_flags = $(CARGO_RUSTCFLAGS)
 ifndef DEVELOPER_OPTIONS
 ifndef MOZ_DEBUG_RUST
 # Enable link-time optimization for release builds.
-cargo_rustc_flags += -C lto
+cargo_rustc_flags += 
 endif
 endif

firefox-68.10.0 will fail with

2:02.73 TEST-UNEXPECTED-FAIL | check_networking | libgkrust.a | Identified 13 networking function(s) being imported in the rust static library (accept,bind,connect,getaddrinfo,getsockname,getsockopt,listen,recv,recvfrom,send,sendto,setsockopt,socket)

for me.

Comment 19

[Andrew Sutherland [:asuth] (he/him)]

I was able to make things happy for beta (soon to be release) by building with rust 1.44.0. Note that for older releases, you may need to choose a different version. For example, for ESR68 in https://bugzilla.mozilla.org/show_bug.cgi?id=1597864#c4 I had to use 1.34.0. (Is this something mach should know how to do?)

Workaround

# get pre 1.45-rust
rustup install 1.44.0
# use old rust in the current directory, assuming you're already in your source dir 
# (and your objdir lives under your source dir?)
rustup override set 1.44.0

And when you want to go back to using the default:

# in the directory you previously overrode
rustup override unset
# If you forgot which directory, list directories with overrides:
rustup override list

Comment 20

[Emerson Bernier]

Keep in mind that various build environments will have only one rust version available (perhaps latest) so choosing different rust version isn't an option. This is typical situation for linux distros.

Comment 21

[Mike Hommey [:glandium]]

Attachment: Bug 1640982 - Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO.

Comment 22

[Alan Robertson]

I can confirm that on Linux Mint 19.3 that Firefox 79 Beta 9 will compile fine with Rust 1.46 providing you change this line:

cargo_rustc_flags += -Clto

To:

cargo_rustc_flags +=

In the file:

config/makefiles/rust.mk

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bb79da633a2e
Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO. r=firefox-build-system-reviewers,rstewart

Comment 24

[Mike Hommey [:glandium]]

Comment on attachment 9164579 [details]
Bug 1640982 - Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO.

Beta/Release Uplift Approval Request

• User impact if declined: Build failure with latest rust
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The patch has no effect at all when using versions of rust < 1.43, and no practical effect on more recent versions. On rust >= 1.45, it enables the compiler to know about the -Clto that was already passed.
• String changes made/needed: N/A

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: See Beta/Release uplift approval request.
• User impact if declined:
• Fix Landed on Version:
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky):
• String or UUID changes made by this patch:

Comment 25

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/bb79da633a2e

Comment 26

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9164579 [details]
Bug 1640982 - Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO.

Approved for all the relevant release branches. Note that we've already built the RC builds for 79.0 and 68.11esr, so unless we need to respin, this fix won't be included in the tagged release commit.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr68/rev/fb557eeb5977

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr78/rev/e5dfd0f87e3b

Comment 29

[aidan]

Attachment: Error after this patch

Not entirely sure if it's related to this issue, but I'm now getting the attached error.

Comment 30

[Jan Alexander Steffens [:heftig]]

I can replicate the error with rustup and the rusqlite crate (https://github.com/rusqlite/rusqlite) like this:

rusqlite> CARGO_PROFILE_RELEASE_LTO=true RUSTFLAGS="-Clinker-plugin-lto=LLVMgold.so" cargo +stable build --release                
    Updating crates.io index
   Compiling pkg-config v0.3.18
   Compiling libc v0.2.73
   Compiling bitflags v1.2.1
   Compiling memchr v2.3.3
   Compiling linked-hash-map v0.5.3
   Compiling fallible-streaming-iterator v0.1.9
   Compiling smallvec v1.4.1
   Compiling fallible-iterator v0.2.0
   Compiling lru-cache v0.1.2
   Compiling libsqlite3-sys v0.19.0 (/home/jan/vc/rusqlite/libsqlite3-sys)
   Compiling time v0.1.43
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,-plugin=LLVMgold.so" "-Wl,-plugin-opt=O3" "-Wl,-plugin-opt=mcpu=x86-64" "-L" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.0.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.1.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.10.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.11.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.12.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.13.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.14.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.15.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.2.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.3.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.4.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.5.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.6.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.7.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.8.rcgu.o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.build_script_build.7mwl8gcz-cgu.9.rcgu.o" "-o" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a" "/home/jan/vc/rusqlite/target/release/build/libsqlite3-sys-f447ce3cb2731b7a/build_script_build-f447ce3cb2731b7a.497gn0w79xbcp3uz.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/jan/vc/rusqlite/target/release/deps" "-L" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/jan/vc/rusqlite/target/release/deps/libpkg_config-800dea52ae78bc9e.rlib" "-Wl,--start-group" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-c147cd9c030850ef.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-7e62a3a07bb85bc1.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-000f77165d4d2d36.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-7dc0cb59ed386ac6.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-9248bfbd7273ac3d.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-b75363fb938de39d.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7bbe96f555da4ad6.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-a145493c64eeb044.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9577436fc6fce6bc.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-b3376c0a2b35415c.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-5708f6b2b59b6e0f.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-e9fd09201d99d6f4.rlib" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-914c6ef6f5cf354a.rlib" "-Wl,--end-group" "/home/jan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-1445b6c7903692a2.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-ldl" "-lutil"
  = note: /usr/bin/ld: /tmp/lto-llvm-7519f5.o: in function `std::sys::unix::fs::copy':
          /rustc/5c1f21c3b82297671ad3ae1e8c942d2ca92e84f2//src/libstd/sys/unix/fs.rs:1172: undefined reference to `__rust_probestack'
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `libsqlite3-sys`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

This works with 1.44.1:

rusqlite> CARGO_PROFILE_RELEASE_LTO=true RUSTFLAGS="-Clinker-plugin-lto=LLVMgold.so" cargo +1.44.1 build --release
   Compiling libc v0.2.73
   Compiling pkg-config v0.3.18
   Compiling memchr v2.3.3
   Compiling bitflags v1.2.1
   Compiling linked-hash-map v0.5.3
   Compiling smallvec v1.4.1
   Compiling fallible-streaming-iterator v0.1.9
   Compiling fallible-iterator v0.2.0
   Compiling lru-cache v0.1.2
   Compiling libsqlite3-sys v0.19.0 (/home/jan/vc/rusqlite/libsqlite3-sys)
   Compiling time v0.1.43
   Compiling rusqlite v0.23.1 (/home/jan/vc/rusqlite)
    Finished release [optimized] target(s) in 3.56s

---

Using clang -flto as the linker instead of GCC with explicit LLVMgold.so behaves analogously:

fails:    CARGO_PROFILE_RELEASE_LTO=true CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang RUSTFLAGS="-Clinker-plugin-lto -Clink-arg=-flto" cargo +stable build --release
succeeds: CARGO_PROFILE_RELEASE_LTO=true CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang RUSTFLAGS="-Clinker-plugin-lto -Clink-arg=-flto" cargo +1.44.1 build --release

Using LLD or gold instead of the BFD linker also makes it work:

CARGO_PROFILE_RELEASE_LTO=true RUSTFLAGS="-Clinker-plugin-lto=LLVMgold.so -Clink-arg=-fuse-ld=gold" cargo +stable build --release
CARGO_PROFILE_RELEASE_LTO=true RUSTFLAGS="-Clinker-plugin-lto=LLVMgold.so -Clink-arg=-fuse-ld=lld" cargo +stable build --release
CARGO_PROFILE_RELEASE_LTO=true CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang RUSTFLAGS="-Clinker-plugin-lto -Clink-arg=-flto -Clink-arg=-fuse-ld=gold" cargo +stable build --release
CARGO_PROFILE_RELEASE_LTO=true CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang RUSTFLAGS="-Clinker-plugin-lto -Clink-arg=-flto -Clink-arg=-fuse-ld=lld" cargo +stable build --release

Comment 31

[Mike Hommey [:glandium]]

Please file a new bug, but if you can reproduce with a third party crate, that would be a bug in the rust compiler or cargo.

Comment 32

[Jan Alexander Steffens [:heftig]]

(In reply to Mike Hommey [:glandium] from comment #31)

Please file a new bug, but if you can reproduce with a third party crate, that would be a bug in the rust compiler or cargo.

I filed an issue at https://github.com/rust-lang/rust/issues/74657.

Comment 33

[aidan]

I am very much not an expert with the rust build tools. What would be the easiest way to work around this issue when building firefox?

Comment 34

[Mike Hommey [:glandium]]

Wait for the fix in bug 1654465

Comment 35

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:glandium, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Comment 36

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9164579 [details]
Bug 1640982 - Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO.

Per comment 34. This was already uplifted to the ESR branches, however.

Comment 37

[Mike Hommey [:glandium]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #36)

Comment on attachment 9164579 [details]
Bug 1640982 - Set CARGO_PROFILE_RELEASE_LTO=true when enabling rust LTO.

Per comment 34. This was already uplifted to the ESR branches, however.

But not beta?

Comment 38

[Ryan VanderMeulen [:RyanVM]]

79 had already merged to release by the time approval was requested and Beta is closed until Monday's merge from m-c. Does this need backing out from the ESR branches?

Comment 39

[Mike Hommey [:glandium]]

I think it's fine, but we'll want to uplift bug 1654465, although it's okay if it doesn't make it in time.