Closed Bug 1954359 Opened 1 year ago Closed 1 year ago

Build failure with GCC and LTO

Categories

(Firefox Build System :: General, defect)

Firefox 137
defect

Tracking

(firefox-esr128 unaffected, firefox136 unaffected, firefox137 fixed, firefox138 fixed)

RESOLVED FIXED
138 Branch
Tracking Status
firefox-esr128 --- unaffected
firefox136 --- unaffected
firefox137 --- fixed
firefox138 --- fixed

People

(Reporter: kocelfc, Assigned: glandium)

References

(Regression)

Details

(Keywords: regression)

Attachments

(3 files)

Attached file complete build log

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:137.0) Gecko/20100101 Firefox/137.0

Steps to reproduce:

Try to build Firefox 137 beta 1-6 with GCC and LTO
This is the script I use to build firefox:

#!/bin/bash
set -x

FF_BLD="/home/kostadin/firefox-build"
FF_SRC="/home/kostadin/mozilla-unified"
FF_CFG="/home/kostadin/mozilla-unified/.mozconfig"

export MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE=none
export PIP_NETWORK_INSTALL_RESTRICTED_VIRTUALENVS=mach

export MOZBUILD_STATE_PATH="${FF_BLD}"

mkdir -p "${FF_BLD}"

export CC=gcc
export CXX=g++
export NM=gcc-nm
export RANLIB=gcc-ranlib

export MAKEOPTS="-j16 -l16"

export CFLAGS="-march=native"
export CXXFLAGS="-march=native"

export MOZ_MAKE_FLAGS="${MAKEOPTS}"
export MOZCONFIG="${FF_CFG}"

export RUSTC="/opt/rust-bin-1.84.1/bin/rustc"
export CARGO="/opt/rust-bin-1.84.1/bin/cargo"

export RUSTFLAGS="-C opt-level=3 -C codegen-units=1"

cat <<-EOF > ${FF_CFG}
ac_add_options --disable-bootstrap
ac_add_options --enable-application=browser #
ac_add_options --enable-project=browser #
ac_add_options --allow-addon-sideload # Gentoo default
ac_add_options --disable-cargo-incremental # Gentoo default
ac_add_options --disable-crashreporter # Gentoo default
ac_add_options --disable-disk-remnant-avoidance # Gentoo default
ac_add_options --disable-geckodriver # Gentoo default
ac_add_options --disable-install-strip # Gentoo default
ac_add_options --disable-legacy-profile-creation # Gentoo default
ac_add_options --disable-parental-controls # Gentoo default
ac_add_options --disable-strip # Gentoo default
ac_add_options --disable-tests # Gentoo default
ac_add_options --disable-updater # Gentoo default
ac_add_options --disable-wmf # Gentoo default
ac_add_options --enable-negotiateauth # Gentoo default
ac_add_options --enable-new-pass-manager # Gentoo default
ac_add_options --enable-official-branding # Gentoo default
ac_add_options --enable-release # Gentoo default
ac_add_options --enable-system-ffi # Gentoo default
ac_add_options --enable-system-pixman # Gentoo default
ac_add_options --enable-system-policies # Gentoo default
ac_add_options --host=x86_64-pc-linux-gnu # Gentoo default
ac_add_options --libdir=/usr/lib64 # Gentoo default
ac_add_options --prefix=/usr # Gentoo default
ac_add_options --target=x86_64-pc-linux-gnu # Gentoo default
ac_add_options --without-ccache # Gentoo default
ac_add_options --without-wasm-sandboxed-libraries # Gentoo default
ac_add_options --with-intl-api # Gentoo default
ac_add_options --with-libclang-path=/usr/lib/llvm/19/lib64 # Gentoo default
#ac_add_options --with-system-nspr # Gentoo default
#ac_add_options --with-system-nss # Gentoo default
ac_add_options --with-system-zlib # Gentoo default
ac_add_options --with-toolchain-prefix=x86_64-pc-linux-gnu- # Gentoo default
ac_add_options --with-unsigned-addon-scopes=app,system # Gentoo default
ac_add_options --x-includes=/usr/include # Gentoo default
ac_add_options --x-libraries=/usr/lib64 # Gentoo default
ac_add_options --update-channel=release #
ac_add_options --enable-rust-simd #
ac_add_options --enable-sandbox #
# ac_add_options --with-system-av1 # +system-av1
# ac_add_options --with-system-harfbuzz # +system-harfbuzz
# ac_add_options --with-system-graphite2 # +system-harfbuzz
# ac_add_options --with-system-icu # +system-icu
ac_add_options --with-system-jpeg # +system-jpeg
ac_add_options --with-system-libevent # +system-libevent
ac_add_options --with-system-libvpx # +system-libvpx
ac_add_options --without-system-png # -system-png
ac_add_options --with-system-webp # +system-webp
ac_add_options --enable-dbus # +dbus
ac_add_options --disable-libproxy # -libproxy
ac_add_options --disable-valgrind # -valgrind
ac_add_options --enable-audio-backends=pulseaudio # --enable-audio-backends
ac_add_options --disable-necko-wifi # -wifi
ac_add_options --enable-default-toolkit=cairo-gtk3-wayland-only # +wayland
ac_add_options --enable-lto=full # +lto
ac_add_options --enable-linker=bfd # linker is set to bfd
ac_add_options --disable-debug # -debug
ac_add_options --disable-real-time-tracing # Gentoo defaults
ac_add_options --disable-debug-symbols # from CFLAGS
ac_add_options --enable-optimize=-O3 # from CFLAGS
ac_add_options --enable-elf-hack=relr # relr elf-hack
ac_add_options XARGS=/usr/bin/xargs # Gentoo default
mk_add_options MOZ_OBJDIR="${FF_BLD}" # Gentoo default
EOF

cd "${FF_BLD}" || exit 1
"${FF_SRC}"/configure || exit 1
exec "${FF_SRC}"/mach build

Actual results:

While building the betas for 137, I noticed that when building with GCC and LTO, the build failed because of undefined references in glsl optimizer. Building with gcc without LTO or Clang with and without LTO is fine.

GCC version used is 15.0.1 20250315
Rust version used is 1.84.1

Bisect led to:

d75afbb5ec7767503bc7d88f97773930610b56d2 is the first bad commit
commit d75afbb5ec7767503bc7d88f97773930610b56d2
Author: Mike Hommey <mh+mozilla@glandium.org>
Date:   Wed Feb 12 03:46:11 2025 +0000

    Bug 1946006 - Pass PGO and LTO flags when compiling C/C++ sources from rust. r=firefox-build-system-reviewers,sergesanspaille
    
    This depends on bug 1945694 for all the relevant flags to make it to the
    compiler, but works even before that lands, which is fortunate because
    landing bug 1945694 fail without this change.
    
    Differential Revision: https://phabricator.services.mozilla.com/D236966

 config/makefiles/rust.mk | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
1:51.76 error: linking with `/home/kostadin/mozilla-unified/build/cargo-linker` failed: exit status: 1
 1:51.76   |
 1:51.76   = note: LC_ALL="C" PATH="/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/kostadin/firefox-build/srcdirs/mozilla-unified-528b8cbe7422/_virtualenvs/build/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/19/bin:/etc/eselect/wine/bin:/home/kostadin/.cargo/bin" VSLANG="1033" "/home/kostadin/mozilla-unified/build/cargo-linker" "-m64" "/tmp/rustci96vXz/symbols.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.00.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.01.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.02.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.03.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.04.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.05.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.06.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.07.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.08.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.09.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.10.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.11.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.12.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.13.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.14.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.build_script_build.f2dd3ae19cb2f7f-cgu.15.rcgu.o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce.dgstdcmhxyhhi53dpgojle4hn.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib" "/home/kostadin/firefox-build/release/deps/libbuild_parallel-a047f2e26e4edb64.rlib" "/home/kostadin/firefox-build/release/deps/libnum_cpus-0c9b1fc93803f903.rlib" "/home/kostadin/firefox-build/release/deps/libjobserver-4c3fa32380d52508.rlib" "/home/kostadin/firefox-build/release/deps/liblibc-355f06ead1c0e24e.rlib" "/home/kostadin/firefox-build/release/deps/libcrossbeam_utils-8538eec2e8ec77ef.rlib" "/home/kostadin/firefox-build/release/deps/libwebrender_build-268c60db47d184b2.rlib" "/home/kostadin/firefox-build/release/deps/libserde-4dcf581a9f531d52.rlib" "/home/kostadin/firefox-build/release/deps/liblazy_static-734f0677264680c0.rlib" "/home/kostadin/firefox-build/release/deps/libbitflags-d26add2e5010500e.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-6f060101dda10b7a.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-70f71d9a6d284c89.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-3dc783e4d6e04b19.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-4fffc92fe380873a.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-c572ac12fe8cba29.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-fc5479163f049547.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-abc6fce37da6e535.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-454117f0927a89af.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-302800adacdaa375.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-0b365bbc25bb52b1.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-ee3a5cb2982a2a9f.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-521236c0a42ef1a3.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-541d85b154f80ded.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-380aa0852ecf2621.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-099427f186ab3f66.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-1c29bac7d035f466.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-d193bce3b6600c65.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-75c6c8b4d83dd0d3.rlib" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-ad7808ee138565ae.rlib" "-Wl,-Bdynamic" "-lstdc++" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/kostadin/firefox-build/release/build/glslopt-84ec9fe0fe1e8dd0/out" "-L" "/home/kostadin/firefox-build/release/build/glslopt-84ec9fe0fe1e8dd0/out" "-L" "/home/kostadin/firefox-build/release/build/glslopt-84ec9fe0fe1e8dd0/out" "-L" "/opt/rust-bin-1.84.1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/kostadin/firefox-build/release/build/webrender-fd58a135af9f72ce/build_script_build-fd58a135af9f72ce" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-debug" "-nodefaultlibs"
 1:51.76   = note: /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `glslopt::Context::new':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN7glslopt7Context3new17h2377adab379fffd0E+0x6): undefined reference to `glslopt_initialize'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `glslopt::Context::optimize':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN7glslopt7Context8optimize17hb43ee81da6851493E+0x3c): undefined reference to `glslopt_optimize'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `<glslopt::Context as core::ops::drop::Drop>::drop':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN58_$LT$glslopt..Context$u20$as$u20$core..ops..drop..Drop$GT$4drop17hea1c2a23b9db6f14E+0x5): undefined reference to `glslopt_cleanup'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `glslopt::Shader::get_status':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN7glslopt6Shader10get_status17h8e4aa5aaf1b93198E+0x5): undefined reference to `glslopt_get_status'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `glslopt::Shader::get_output':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN7glslopt6Shader10get_output17hef189a8ad41f9869E+0xa): undefined reference to `glslopt_get_output'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `glslopt::Shader::get_log':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN7glslopt6Shader7get_log17h74b43fad0990270dE+0xa): undefined reference to `glslopt_get_log'
 1:51.76           /usr/lib/gcc/x86_64-pc-linux-gnu/15/../../../../x86_64-pc-linux-gnu/bin/ld.bfd: /home/kostadin/firefox-build/release/deps/libglslopt-a7219b9f7db66c19.rlib(glslopt-a7219b9f7db66c19.glslopt.466691a160b0a324-cgu.0.rcgu.o): in function `<glslopt::Shader as core::ops::drop::Drop>::drop':
 1:51.76           glslopt.466691a160b0a324-cgu.0:(.text._ZN57_$LT$glslopt..Shader$u20$as$u20$core..ops..drop..Drop$GT$4drop17h5ec548ac216030e5E+0x5): undefined reference to `glslopt_shader_delete'
 1:51.76           collect2: error: ld returned 1 exit status
 1:51.76   = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
 1:51.76   = note: use the `-l` flag to specify native libraries to link
 1:51.76   = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)
 1:51.77 error: could not compile `webrender` (build script) due to 1 previous error

Expected results:

The build finishes fine

The Bugbug bot thinks this bug should belong to the 'Core::Graphics: WebRender' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Graphics: WebRender
Product: Firefox → Core
Component: Graphics: WebRender → General
Keywords: regression
Product: Core → Firefox Build System
Regressed by: 1946006

:glandium, since you are the author of the regressor, bug 1946006, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Flags: needinfo?(mh+mozilla)
Assignee: nobody → mh+mozilla
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Set release status flags based on info from the regressing bug 1946006

Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/6790f80a80be Don't pass LTO flags when compiling C/C++ sources from rust with GCC. r=firefox-build-system-reviewers,sergesanspaille
Flags: needinfo?(mh+mozilla)
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 138 Branch

The patch landed in nightly and beta is affected.
:glandium, is this bug important enough to require an uplift?

  • If yes, please nominate the patch for beta approval.
  • If no, please set status-firefox137 to wontfix.

For more information, please visit BugBot documentation.

Flags: needinfo?(mh+mozilla)
Severity: -- → S3
Attachment #9473114 - Flags: approval-mozilla-beta?

beta Uplift Approval Request

  • User impact if declined: Build failure for downstreams using GCC with LTO
  • Code covered by automated testing: no
  • Fix verified in Nightly: yes
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing: N/A
  • Risk associated with taking this patch: Low
  • Explanation of risk level: NPOTB
  • String changes made/needed: N/A
  • Is Android affected?: no
Attachment #9473114 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Flags: needinfo?(mh+mozilla)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: