./mach bootstrap to install downstream rust package

RESOLVED FIXED in Firefox 53

Status

defect
RESOLVED FIXED
3 years ago
Last year

People

(Reporter: jbeich, Assigned: jbeich)

Tracking

Trunk
mozilla54
Dependency tree / graph

Firefox Tracking Flags

(firefox52 unaffected, firefox53 fixed, firefox54 fixed)

Details

Attachments

(1 attachment)

Tier3 platforms are poorly represented on rustup. For one, DragonFly and OpenBSD are currently missing[1] while i386 lacks Cargo. It'd be nice to try downstream package as well.

[1] https://static.rust-lang.org/dist/channel-rust-stable.toml
- MSYS2 packages are currently pure guess assuming they're similar to ArchLinux
- Homebrew (OS X) appears to have Rust + Cargo in one package[1]
- Gentoo is left as is because it relies on ebuild dependencies
- Debian (and openSUSE) use "rustc" for package with Rust compiler
- Rust and Cargo are installed explicitly as relying on whether Cargo pulls Rust is error-prone

[1] https://github.com/Homebrew/homebrew-core/blob/master/Formula/rust.rb
Rust+Cargo are installed for both Desktop Firefox and Mobile Firefox. While mozconfig.rust is currently not enabled for Mobile the patch in bug 1283898 doesn't explicitly exclude it.
(In reply to Jan Beich from comment #3)
> Rust+Cargo are installed for both Desktop Firefox and Mobile Firefox. While
> mozconfig.rust is currently not enabled for Mobile the patch in bug 1283898
> doesn't explicitly exclude it.

The intent is to enable it for mobile as well. It's enabled for official armv7-android builds starting with Firefox 52.

More of a bug is that we don't install the standard library for the android targets when bootstrapping for mobile development. Bug 1320741 adds a configure-time suggestion for how to handle that at least.
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review96514

::: python/mozboot/mozboot/centosfedora.py:23
(Diff revision 1)
>  
>          self.group_packages = []
>  
>          self.packages = [
>              'autoconf213',
> +            'cargo',

Thanks for the patch.

My concern with using system packages on Debian and Fedora/CentOS is that the packaged rust will almost always be too old. For many people, this is likely to just be an extra download, and an additional point of confusion between the system-package rust and the one in ~/.cargo/bin in the PATH order.

Also, until distro packaging of the rust cross libstd is better, rustup is a much easier path to cross-compilation.

What do you think about doing this only for platforms rustup doesn't support?

It would also help to add tier-3 platforms rustup *does* support to `python/mozboot/mozboot/rust` like arm and aarch64 linux. I added x86_64 FreeBSD there as an example, but didn't test it.
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review99846

I agree with rillian--I don't think we should do this for the platforms where rustup works. If people want to install their distro's Rust package and it works that's fine, but I think they'll just be annoyed later when we bump our Rust version requirement and their distro package hasn't kept up.

I'd be fine with taking this for the BSD bootstrap scripts (although it looks like rustup has freebsd and netbsd packages: https://github.com/rust-lang-nursery/rustup.rs/#other-installation-methods).
Attachment #8815259 - Flags: review?(ted) → review-
Attachment #8815259 - Flags: review?(nate.hak)
Sorry it took *way* too long to clear the review flag. I'm busy with school and probably shouldn't be reviewing patches without taking the time to make sure I'm doing it correctly.
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review96514

> Thanks for the patch.
> 
> My concern with using system packages on Debian and Fedora/CentOS is that the packaged rust will almost always be too old. For many people, this is likely to just be an extra download, and an additional point of confusion between the system-package rust and the one in ~/.cargo/bin in the PATH order.
> 
> Also, until distro packaging of the rust cross libstd is better, rustup is a much easier path to cross-compilation.
> 
> What do you think about doing this only for platforms rustup doesn't support?
> 
> It would also help to add tier-3 platforms rustup *does* support to `python/mozboot/mozboot/rust` like arm and aarch64 linux. I added x86_64 FreeBSD there as an example, but didn't test it.

> system packages on Debian and Fedora/CentOS ... too old

Bug 1284816 should force downstream to stay up to date at least for mozilla-release.

> until distro packaging of the rust cross libstd is better, rustup is a much easier path to cross-compilation.

On FreeBSD to cross-compile from amd64 to i386 it requires chroot/jail with cargo/rustc, not just libstd, as there're no lib32 packages to satisfy Gecko dependencies. As for non-x86 I'm not sure Linux is any better except when the target is Android or MinGW which have prepackaged sysroot.

> It would also help to add tier-3 platforms rustup does support ... like arm and aarch64 linux

I'm waiting for rustup to add support for i686 or aarch64 on freebsd, not interested otherwise.
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review96514

> > system packages on Debian and Fedora/CentOS ... too old
> 
> Bug 1284816 should force downstream to stay up to date at least for mozilla-release.
> 
> > until distro packaging of the rust cross libstd is better, rustup is a much easier path to cross-compilation.
> 
> On FreeBSD to cross-compile from amd64 to i386 it requires chroot/jail with cargo/rustc, not just libstd, as there're no lib32 packages to satisfy Gecko dependencies. As for non-x86 I'm not sure Linux is any better except when the target is Android or MinGW which have prepackaged sysroot.
> 
> > It would also help to add tier-3 platforms rustup does support ... like arm and aarch64 linux
> 
> I'm waiting for rustup to add support for i686 or aarch64 on freebsd, not interested otherwise.

> I added x86_64 FreeBSD there as an example, but didn't test it.

Thank you. It works as intended, tested in a 11.0 amd64 jail.
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review116508

Thanks for updating the patch. r=me for tier-3 platform-specific improvements.
Attachment #8815259 - Flags: review?(giles) → review+
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

https://reviewboard.mozilla.org/r/96272/#review96514

> > I added x86_64 FreeBSD there as an example, but didn't test it.
> 
> Thank you. It works as intended, tested in a 11.0 amd64 jail.

Great, thanks for confirming the x86_64 FreeBSD rustup install worked!

> Bug 1284816 should force downstream to stay up to date at least for mozilla-release.

Maybe Fedora. They're currently on rust 1.14.0, while m-c requires 1.15.1, but that could be available in updates by the time Firefox 54 is in release. Debian and CentOS ship the esr release, so they won't need to support Rust until 2018.

> As for non-x86 I'm not sure Linux is any better...

This has improved a lot lately. rustup currently has std library packages for linux arm, arm-hf, armv-hf, aarch64, mips, mipsel, mips64, mips64el, powerpc, powerpc64, powerpc64le, and s390x. About half of those have musl variants. Sparc is the only major architecture missing in the rust 1.15 release, and that's available on nightly now. Of course those are tier-2 or -3 builds, so they may be buggy, but I think the compiler team has been positive in supporting ports.
Pushed by rgiles@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/502b6c944ccf
rustup is N/A on Tier3 platforms, so prefer downstream Rust packages. r=rillian
https://hg.mozilla.org/mozilla-central/rev/502b6c944ccf
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

I'd like this backported to Firefox 53.0. The merge date is close, so one of approval* flags maybe redundant.

Approval Request Comment
[Feature/Bug causing the regression]: bug 1283898 regression
[User impact if declined]: Broken build on some Tier3 platforms due to missing dependencies unless --disable-rust is passed
[Is this code covered by automated tests?]: No
[Has the fix been verified in Nightly?]: Yes, by myself.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Can only break "./mach bootstrap" on FreeBSD and OpenBSD, nop elsewhere.
[String changes made/needed]: None
Attachment #8815259 - Flags: approval-mozilla-beta?
Attachment #8815259 - Flags: approval-mozilla-aurora?
Comment on attachment 8815259 [details]
Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages.

Another build fix for FreeBSD. This should make it into 53 before it moves to beta.
Attachment #8815259 - Flags: approval-mozilla-beta?
Attachment #8815259 - Flags: approval-mozilla-beta-
Attachment #8815259 - Flags: approval-mozilla-aurora?
Attachment #8815259 - Flags: approval-mozilla-aurora+
Assignee: nobody → jbeich
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.