1561907 - Build system fails to execute Rust

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Paul Menzel]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

Try to build Firefox 67.0.4 or Thunderbird 60.7.2.

/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/build> env   RUSTFLAGS='-C opt-level=2 -C debuginfo=2 '  CARGO_TARGET_DIR=/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/build/toolkit/library RUSTC=/usr/local/package/bin/rustc MOZ_SRC=/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/source MOZ_DIST=/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/build/dist LIBCLANG_PATH="/usr/lib" CLANG_PATH="/usr/bin/clang" PKG_CONFIG_ALLOW_CROSS=1 RUST_BACKTRACE=full MOZ_TOPOBJDIR=/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/build  MOZ_CARGO_WRAP_LDFLAGS="-lpthread -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,--build-id -Wl,-rpath-link,/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/build/dist/bin -Wl,-rpath-link,/usr/lib" MOZ_CARGO_WRAP_LD=" /usr/bin/gcc -std=gnu99" CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=/dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/source/build/cargo-linker /usr/local/package/bin/cargo rustc  --release --frozen --manifest-path /dev/shm/bee-root/thunderbird/thunderbird-60.7.2-0/source/toolkit/library/rust/Cargo.toml --lib --target=x86_64-unknown-linux-gnu --features "servo bindgen cubeb_pulse_rust cubeb-remoting no-static-ideograph-encoder-tables" --  -C lto
error: process didn't exit successfully: `/usr/local/package/bin/rustc -vV` (exit code: 1)
--- stderr
/usr/local/package/bin/rustc : rustc not found

Actual results:

The rustc binary is not found. Here is the wrapper content.

$ more /usr/local/package/bin/rustc
#! /bin/bash
cmd=$(basename "$0")
if [ -n "$PWRAP_rustc_1_34_2_0" ]; then
  echo "$0 : $cmd not found" >&2;
  exit 1
fi
. /usr/local/package/lib/rustc-1.34.2-0.profile
export PWRAP_rustc_1_34_2_0=1
if ! /usr/sbin/hostconfig 'mxqd' ;then
  rcmd=$(/usr/bin/readlink -f $(/usr/bin/which "$cmd"))
  user=$(/usr/bin/id -nu)
  /usr/bin/logger --server log.molgen.mpg.de -t usrlocalpkg "rustc-1.34.2-0.pro
file ${cmd} (${rcmd}) ${user}@${HOSTNAME} mx64"
fi
exec "$cmd" "$@"

Expected results:

The wrapper script should have been executed normally.

$ /usr/local/package/bin/rustc -vV
rustc 1.34.2
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.34.2
LLVM version: 8.0

Sourcing the profile manually seems to work around the issue.

. /pkg/rustc-1.34.2-0/profile

$ more /pkg/rustc-1.34.2-0/profile
PATH=/pkg/rustc-1.34.2-0/bin:$PATH
LD_LIBRARY_PATH=/pkg/rustc-1.34.2-0/lib:$LD_LIBRARY_PATH
export PATH LD_LIBRARY_PATH
if [ -d /pkg/rustc-1.34.2-0/.compatlibs ]; then LD_LIBRARY_PATH=/pkg/rustc-1.34
.2-0/.compatlibs${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} ; fi

Comment 1

[Paul Menzel]

The problem is, that the build system sets the environment variable RUSTC is set to an absolute path. In our case it’s the wrapper script. cargo then executes that. RUSTC shouldn’t be set at all, and rustc just be run from the environment.

Comment 2

[Nathan Froyd [:froydnj]]

(In reply to Paul Menzel from comment #1)

The problem is, that the build system sets the environment variable RUSTC is set to an absolute path. In our case it’s the wrapper script. cargo then executes that. RUSTC shouldn’t be set at all, and rustc just be run from the environment.

It seems like the problem is that the wrapper script can't find rustc, and that doesn't seem like the fault of the Firefox build system? Where does the actual rustc live in this setup, and why can't the wrapper script find it?

Comment 3

[Paul Menzel]

(In reply to Nathan Froyd [:froydnj] from comment #2)

(In reply to Paul Menzel from comment #1)

The problem is, that the build system sets the environment variable RUSTC is set to an absolute path. In our case it’s the wrapper script. cargo then executes that. RUSTC shouldn’t be set at all, and rustc just be run from the environment.

It seems like the problem is that the wrapper script can't find rustc, and that doesn't seem like the fault of the Firefox build system? Where does the actual rustc live in this setup, and why can't the wrapper script find it?

The wrapper script tries to avoid loops to change the path needlessly by sourcing the profile several times.

if [ -n "$PWRAP_rustc_1_34_2_0" ]; then
  echo "$0 : $cmd not found" >&2;
  exit 1
fi
. /usr/local/package/lib/rustc-1.34.2-0.profile
export PWRAP_rustc_1_34_2_0=1

This is done by exporting the variable PWRAP_rustc_1_34_2_0 and setting it to 1.

We could change the wrapper script to allow a certain amount of loops, but would like to know, why the absolute path is needed by the build system in the first place?

Comment 4

[Nathan Froyd [:froydnj]]

(In reply to Paul Menzel from comment #3)

We could change the wrapper script to allow a certain amount of loops, but would like to know, why the absolute path is needed by the build system in the first place?

Did you specify RUSTC in your mozconfig? Can you attach the config.log that you get from a build?

Comment 5

[Paul Menzel]

Attachment: `config.log` from running `./configure …`

We didn’t configure anything for Rust. Please find the output config.log attached.

Comment 6

[Nathan Froyd [:froydnj]]

OK, so configure finds the wrapper script and returns an absolute path to that.

If the wrapper script can't handle being invoked as an absolute path, then I'd say that's a bug in the wrapper script.

Comment 7

[Paul Menzel]

But why is the absolute path needed?

The script can be called by an absolute path, but not several times.

Comment 8

[Paul Menzel]

(In reply to Paul Menzel from comment #5)

Created attachment 9074521 [details]
config.log from running ./configure …

Sorry, I uploaded the wrong config.log, where the real binary is referenced and not the wrapper script. I’ll upload the “wrong” one soon.

Comment 9

[Nathan Froyd [:froydnj]]

(In reply to Paul Menzel from comment #7)

But why is the absolute path needed?

I suppose it's not strictly needed, but it is what our program-finding bits in configure use. I don't think this behavior is particularly unusual; AC_PATH_PROG{,S} from autoconf does this as well.

It does have the virtue of insulating the build from changes in PATH. (You could reasonably argue that changes in PATH should trigger a reconfigure, but build systems are hard and there is only so much time in a day.)

The script can be called by an absolute path, but not several times.

This does not seem like a Firefox build system issue.