1545437 - Cannot detect valid Rust target on Yocto buid system

Status: NEW

(Firefox Build System :: General: Unsupported Platforms, enhancement, P5)

Description

[Takuro Ashie [:ashie]]

I got the following error when I try to build mozilla-central on Yocto build system.

 0:05.41 DEBUG: Executing: `/build/master/build/tmp/work/i586-poky-linux/firefox/68esr-r0/recipe-sysroot-native/usr/bin/rustc --crate-type staticlib --target=i686-unknown-linux-gnu -o /tmp/conftest5oS4YD.rlib /tmp/conftestD7xj2s.rs`
 0:05.41 DEBUG: The command returned non-zero exit status 1.
 0:05.41 DEBUG: Its error output was:
 0:05.41 DEBUG: | error[E0463]: can't find crate for `std`
 0:05.41 DEBUG: |   |
 0:05.41 DEBUG: |   = note: the `i686-unknown-linux-gnu` target may not be installed
 0:05.41 DEBUG: |
 0:05.42 DEBUG: | error: aborting due to previous error
 0:05.42 DEBUG: |
 0:05.42 DEBUG: | For more information about this error, try `rustc --explain E0463`.
 0:05.42 ERROR: Cannot compile for i586-poky-linux-gnu with /build/master/build/tmp/work/i586-poky-linux/firefox/68esr-r0/recipe-sysroot-native/usr/bin/rustc

The mozilla build system tries to guess Rust target but it may fail on certain platform such as Yocto.
Although Yocto can provide valid Rust target name, there is no chance to pass it to mozilla's build system.
It would be better that it has an option to do it.

Comment 1

[Takuro Ashie [:ashie]]

Attachment: Bug 1545437 - Add options to specify Rust target name r?glandium

Certain build systems such as Yocto know more suitable Rust target name, so it
would be better that there is a way to pass it to Mozilla's build system.

Comment 2

[Nathan Froyd [:froydnj]]

Please provide your mozconfig and a complete config.log file. Given that you're compiling for i586-poky-linux-gnu, I think we ought to be selecting the i586-unknown-linux-gnu target...but I also think that we don't support compiling Firefox for systems without SSE2 anymore.

Comment 3

[Takuro Ashie [:ashie]]

The above log is just for an example, our main target is x86_64, Arm and AArch64.
It fails to detect against such platforms as well.
I confirmed that I can build against them by the attached patch.

https://github.com/webdino/meta-browser/tree/firefox-68-wip

Comment 4

[Nathan Froyd [:froydnj]]

(In reply to Takuro Ashie [:ashie] from comment #3)

The above log is just for an example, our main target is x86_64, Arm and AArch64.
It fails to detect against such platforms as well.
I confirmed that I can build against them by the attached patch.

https://github.com/webdino/meta-browser/tree/firefox-68-wip

This looks more like a complete build configuration mechanism than a patch.

Please provide the information requested in comment 2. Thanks.

Comment 5

[Takuro Ashie [:ashie]]

Attachment: mozconfig

Comment 6

[Takuro Ashie [:ashie]]

Attachment: config.log

Comment 7

[Takuro Ashie [:ashie]]

Attachment: config.log

Comment 8

[Takuro Ashie [:ashie]]

Sorry, the first config.log is for the patched version.
The later one is for the non-patched version.

Comment 9

[Takuro Ashie [:ashie]]

FYI it seems that it can't find Rust stdlib on Yocto when performing test compile.
We apply the following patch for 60esr to avoid it:

https://github.com/OSSystems/meta-browser/blob/master/recipes-browser/firefox/firefox/fixes/rustc_cross_flags.patch

Comment 10

[Nathan Froyd [:froydnj]]

Ah, the problem is that we canonicalize i586-poky-linux-gnu to be "x86 cpu on linux", which selects the i686 runtime libraries. We'd have to somehow split things up so that i586 actually looks like a different CPU to the rust target selection process...but again, I don't think we support 32-bit x86 CPUs that don't support SSE2.

(In reply to Takuro Ashie [:ashie] from comment #9)

FYI it seems that it can't find Rust stdlib on Yocto when performing test compile.
We apply the following patch for 60esr to avoid it:

https://github.com/OSSystems/meta-browser/blob/master/recipes-browser/firefox/firefox/fixes/rustc_cross_flags.patch

This doesn't seem like a proper fix: this fixes the configure test, but surely rustc will be invoked later in the build process and need the exact same -L flag, otherwise linking Rust libraries will fail?

Comment 11

[Takuro Ashie [:ashie]]

Attachment: config.log for aarch64

Comment 12

[Takuro Ashie [:ashie]]

I attached a config.log for the case of AArch64.
It also fails with original code and success with the patch.

This doesn't seem like a proper fix: this fixes the configure test, but surely rustc will be invoked later in the build process and need the exact same -L flag, otherwise linking Rust libraries will fail?

This is the odd point. We success later build process without it.

Comment 13

[Takuro Ashie [:ashie]]

The most different thing against normal build is that we introduce Rust toolchain by Yocto (meta-rust), not using ./mach bootstrap & rustup.

Comment 14

[Simon McVittie]

A similar configure failure can also happen with the arm*-unknown-linux-gnueabi family of targets: there's at least armv4t-unknown-linux-gnueabi, armv5te-unknown-linux-gnueabi and armv7-unknown-linux-gnueabi, all of which are currently considered to be (CPU=ARM, endian=little, ABI=linux-gnueabi).

armv6-unknown-netbsd-eabihf and armv7-unknown-netbsd-eabihf would probably have the same problem. I think the current situation is that configure will only succeed if the rust target you have installed happens to be the last one in rustc --print target-list that the build system considers to match its simplified (CPU, endian, ABI) triple, which seems fragile: if rustc adds a new i686sse-unknown-linux-gnu or armv8-unknown-linux-gnueabi target, then people who are compiling for i686 or ARMv7 will find that their intended target is no longer the last one in the list.

Debian is in about the same situation as Yocto: our toolchain/infrastructure already knows which Rust target is the most appropriate, but there's currently no way to communicate that to the build system.

Comment 15

[Simon McVittie]

On Debian's ARM ports, unlike x86, there is not necessarily enough information in the GNU tuple to infer which rustc target is the most suitable: Debian armel configures with --target=arm-linux-gnueabi, which in fact is meant to map to Rust --target=armv5te-unknown-linux-gnueabi, but the build system doesn't have enough information to know that we didn't mean armv4t-unknown-linux-gnueabi or armv5te-unknown-linux-gnueabi.

Comment 16

[Simon McVittie]

(In reply to Simon McVittie from comment #15)

doesn't have enough information to know that we didn't mean armv4t-unknown-linux-gnueabi or armv5te-unknown-linux-gnueabi.

Sorry, that should have said: armv4t-unknown-linux-gnueabi or armv7-unknown-linux-gnueabi.

Comment 17

[Mike Hommey [:glandium]]

Configure does check what the C compiler targets by default, or with the flags the build system is given, and uses that as additional information to choose the rust target.
https://searchfox.org/mozilla-central/rev/202a285024f174c2d2bf2152d9cba90a03723eab/build/moz.configure/rust.configure#281