1802095 - error: linking with `/builds/mozilla-central-build/build/cargo-linker` failed: exit status: 1

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect)

Description

[Petr Sumbera]

Steps to reproduce:

Build on Solaris fails with:

23:28.88    Compiling unicode-normalization v0.1.21
23:29.09    Compiling env_logger v0.9.0
23:29.24    Compiling mio v0.6.23 (/builds/psumbera/mozilla-central-build/third_party/rust/mio-0.6.23)
23:29.34 error: linking with `/builds/psumbera/mozilla-central-build/build/cargo-linker` failed: exit status: 1
23:29.34   |
23:29.35   = note: "/builds/psumbera/mozilla-central-build/build/cargo-linker" "-m64" "/tmp/rustcsA3pZU/symbols.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.0.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.1.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.10.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.11.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.12.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.13.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.14.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.15.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.2.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.3.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.4.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.5.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.6.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.7.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.8.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.build_script_build.077e1d52-cgu.9.rcgu.o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1.34g95vckzuewzt9k.rcgu.o" "-Wl,-z,ignore" "-L" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/deps" "-L" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libstd-aa0182df625918f7.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libpanic_unwind-89aee586b8c36aeb.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libobject-b6acc4caf6740c9c.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libmemchr-8b4494dbb5c259f4.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libaddr2line-f42e8e95660ad206.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libgimli-889d4cba112c5fbb.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/librustc_demangle-fa50caeba308b6c6.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libstd_detect-39a5c4192deb3d30.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libhashbrown-d9ccbcd3dfe1f3be.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libminiz_oxide-e026e6a62148ac14.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libadler-c65806df40ec6ed7.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/librustc_std_workspace_alloc-a0ffa3c181a0b607.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libunwind-9e6e1ade1cd6873d.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libcfg_if-1dad653efbada068.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/liblibc-c5d3b4087849fd2e.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/liballoc-04e474a53a74dabf.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/librustc_std_workspace_core-2627cd144f4b48b5.rlib" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libcore-bea228e8c308139a.rlib" "-Wl,--end-group" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib/libcompiler_builtins-5418f936eba3f440.rlib" "-Wl,-Bdynamic" "-lsocket" "-lposix4" "-lpthread" "-lresolv" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lsendfile" "-llgrp" "-L" "/builds/psumbera/rustc-1.63.0/lib/rustlib/x86_64-pc-solaris/lib" "-o" "/builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1" "-nodefaultlibs"
23:29.35   = note: ld: fatal: file /builds/psumbera/mozilla-central-build/obj-x86_64-pc-solaris2.11/debug/build/libc-c9d4fffdbed5e5f1/build_script_build-c9d4fffdbed5e5f1: creation interrupted: Bad address
23:29.35           collect2: error: ld returned 1 exit status
23:29.39 error: could not compile `libc` due to previous error
23:29.39 warning: build failed, waiting for other jobs to finish...
23:30.63    Compiling idna v0.2.3
23:30.99    Compiling nom v7.1.1
23:31.80    Compiling clang-sys v1.3.3
23:32.03 toolkit/library/build/libxul.so
23:33.72    Compiling mio-extras v2.0.6
23:33.92    Compiling url v2.1.0
23:34.50    Compiling chrono v0.4.19
23:35.35 error: failed to build archive: No such file or directory
23:35.39 error: could not compile `memchr` due to previous error
23:35.39 warning: build failed, waiting for other jobs to finish...

The first bad revision is:
changeset: 643235:6b0210e159c4
user: Arthur Carcano <[e-mail]>
date: Tue Nov 22 11:55:44 2022 -0800
description:
Bug 1798291 - Use a common CARGO_TARGET_DIR for all crates.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Toolchains' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

:arthur.carcano, since you are the author of the regressor, bug 1798291, could you take a look?

For more information, please visit auto_nag documentation.

Comment 3

[Mike Hommey [:glandium]]

Petr, can you try this?

diff --git a/python/mozbuild/mozbuild/backend/recursivemake.py b/python/mozbuild/mozbuild/backend/recursivemake.py
index f9f788f55c3d7..0743b81f8ea4a 100644
--- a/python/mozbuild/mozbuild/backend/recursivemake.py
+++ b/python/mozbuild/mozbuild/backend/recursivemake.py
@@ -786,7 +786,7 @@ class RecursiveMakeBackend(MakeBackend):
                 # builds.
                 for prior_target, target in pairwise(
                     sorted(
-                        [t for t in rust_libs], key=lambda t: t != self._gkrust_target
+                        [t for t in rust_roots], key=lambda t: t != self._gkrust_target
                     )
                 ):
                     r = root_deps_mk.create_rule([target])

Comment 4

[Petr Sumbera]

Yes, this one makes it work again. Thank you!

Comment 5

[Mike Hommey [:glandium]]

(In reply to Petr Sumbera from comment #4)

Yes, this one makes it work again. Thank you!

You should open an issue on https://github.com/rust-lang/cargo about the fact that locking doesn't work on Solaris. (normally, running cargo twice in the same workspace will make one of them wait for the other)

Comment 6

[Mike Hommey [:glandium]]

Attachment: Bug 1802095 - Serialize all calls to cargo.

Before bug 1798291, all rust libraries were built in the same target
directory, and rust programs were built in a separate target directory
each. Cargo would be allowed to run in parallel for each target
directory. After bug 1798291, everything rust hits the same target
directory, so the normal behavior for Cargo is to block itself if run
multiple times. What this means in terms of the build is that now those
Cargos that are waiting are taking a make token, preventing other work
from happening.

It also turns out that on Solaris, Cargo doesn't block, and those Cargos
running in parallel are stepping on each other's toes.

By serializing, we ensure this doesn't happen. This should have been
done at the same time as bug 1798291 but I overlooked this.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d1e0375c436
Serialize all calls to cargo. r=firefox-build-system-reviewers,ahochheiden

Comment 8

[Petr Sumbera]

(In reply to Mike Hommey [:glandium] from comment #5)

You should open an issue on https://github.com/rust-lang/cargo about the fact that locking doesn't work on Solaris. (normally, running cargo twice in the same workspace will make one of them wait for the other)

https://github.com/rust-lang/cargo/issues/11421

Comment 9

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/0d1e0375c436

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1798291