Closed Bug 1284816 Opened 8 years ago Closed 7 years ago

Require Rust to build

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox50 affected, firefox54 fixed)

RESOLVED FIXED
mozilla54
Tracking Status
firefox50 --- affected
firefox54 --- fixed

People

(Reporter: ted, Assigned: rillian)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

At some point we're going to require Rust to build Firefox. This is a blocker to writing any critical components in Rust. Once we've fixed the blockers of bug 1283898 this is primarily just a political decision. It will mean that non-Tier-1 platforms may have a harder time building Firefox, but I think we're OK with that.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #0)
> It will
> mean that non-Tier-1 platforms may have a harder time building Firefox, but
> I think we're OK with that.
To give more explanations with my Debian packager hat, Rust becoming a mandatory requirement means:
* Rust has to be supported on the OS or arch
* As Rust is based on LLVM, LLVM has to be properly supported on the OS and arch
* As a nice side effect, as most of the GNU/Linux distro are still using gcc to build LLVM (and clang) and LLVM is not too afraid to require new C++ features, that means that, in some cases, gcc has to be backported (this is the case for some old Ubuntu LTS being still supported)
* Rust moves super fast and has a tight dependency on the latest LLVM stable version. If Firefox code uses the most recent features of Rust, that means that backporting Firefox involves backporting Rust + LLVM (and potentially gcc).

I am not saying we should not do it, just explaining the impact on non-tier-1 platforms
(In reply to Sylvestre Ledru [:sylvestre] from comment #1)
> * Rust moves super fast and has a tight dependency on the latest LLVM stable
> version. If Firefox code uses the most recent features of Rust, that means
> that backporting Firefox involves backporting Rust + LLVM (and potentially
> gcc).
> 
> I am not saying we should not do it, just explaining the impact on
> non-tier-1 platforms

As a side note, LTS distributions of any kind (RH/SLE/Ubuntu LTS) would be helped tremendously if Firefox ESR required no rustc upgrades during its lifetime (i.e. fixes in the Rust code not needing newer Rust features then on the day of fork from the rapid release).
> (In reply to Ted Mielczarek [:ted.mielczarek] from comment #0)
> I am not saying we should not do it, just explaining the impact on
> non-tier-1 platforms

I agree that this could be a significant burden. However, we've decided that the advantage of using Rust is too great here. If the outcome is that Firefox doesn't ship on non-Tier-1 platforms we are OK with that. We normally don't go out of our way to make life harder for people maintaining Firefox ports, but in this case we can't let lesser-used platforms restrict us from using Rust in Firefox.

I suspect that for the Debian case where it's just Linux on other CPU architectures, once someone does the work to port LLVM to that architecture and gets that upstreamed then the future maintenance burden should not be great. (This does assume that Rust converges on using upstream LLVM.)

(In reply to Petr Cerny [:hrosik] from comment #2)
> As a side note, LTS distributions of any kind (RH/SLE/Ubuntu LTS) would be
> helped tremendously if Firefox ESR required no rustc upgrades during its
> lifetime (i.e. fixes in the Rust code not needing newer Rust features then
> on the day of fork from the rapid release).

That is totally reasonable and we should codify that. I don't know if we have it written down anywhere, but we've generally avoided taking toolchain updates on stable branches.(In reply to Sylvestre Ledru [:sylvestre] from comment #1)
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #3)
> I suspect that for the Debian case where it's just Linux on other CPU
> architectures, once someone does the work to port LLVM to that architecture
> and gets that upstreamed then the future maintenance burden should not be
> great. (This does assume that Rust converges on using upstream LLVM.)

https://www.debian.org/distrib/netinst indicates that Debian runs on   

 * amd64
 * arm64
 * armel
 * armhf
 * i386
 * mips
 * mipsel
 * powerpc
 * ppc64el
 * s390x

The Rust team already ships rustc and cargo for amd64 aka. x86_64 and for the ARM variants:
https://github.com/rust-lang-nursery/rustup.rs/#other-installation-methods

Additionally, the Rust team ships rustc and cargo for i686. From forum posts I understand that it's possible to target x86 without SSE2 at least.

Additionally, https://forge.rust-lang.org/platform-support.html indicates that cross-compiling the standard library for mips and mipsel is already part of the Rust team's continuous integration, so one might assume mips and mipsel to work with reasonable effort.

powerpc is listed as tier 3, but it's unclear if someone is actively keeping it working.

That leaves only the IBM server architectures, ppc64el and s390x, as unsupported. Debian already has LLVM for those: https://packages.debian.org/search?keywords=llvm&searchon=names&suite=stable&section=all . It's unclear to me how relevant ppc64el and s390x are to desktop applications.

So:

 * amd64: Shipped by the Rust team both via the traditional installer and rustup.rs
 * arm64: Shipped by the Rust team both via rustup.rs
 * armel: Shipped by the Rust team both via rustup.rs
 * armhf: Shipped by the Rust team both via rustup.rs
 * i386: Needs a special target config (i686 shipped by the Rust team)
 * mips: Tier 2 for Rust
 * mipsel: Tier 2 for Rust
 * powerpc: Tier 3 for Rust
 * ppc64el: LLVM exists
 * s390x: LLVM exists
In conversations with Linux distro maintainers, it has come up that it would be nice if we could avoid *requiring* Rust until after the next ESR, Firefox 52. So far as I can tell, this would only mean that we should not add features that are Rust-only until 52 leaves m-c, which from https://wiki.mozilla.org/RapidRelease/Calendar seems to indicate early/mid-September.

Since most of the work we are doing right now has a C++ fallback (or is running both the Rust and C++ side-by-side for testing reasons), that seems like a reasonable request.
Fedora also wants s390, ppc64 and ppc64le, and Martin Stransky described the need as 'critical'. But it's not clear to me still whether the lack of support for any particular architecture is a showstopper for either Debian or Fedora, whether they can just drop Firefox from those architectures. It's also not clear to me that they need a compiler *hosted* for those platform - in theory they could cross-compile, but I imagine for simplicity all architectures are purely self-hosting, that is, they use build servers that correspond to that architecture. I'll try to clarify.

Also note that Rust's in-tree ARM definitions are a mess: https://github.com/rust-lang/rust/issues/35590. I wouldn't necessarily expect that to prevent packagers from creating their own target definitions that worked to their liking, but we're trying to get it all sorted out. Debian says this is blocking armhf support for them.

There's some indication that Rust will produce powerpc64le compilers in the (relatively) near future, but there is no movement on the others. It is quite difficult to maintain builds for all these platforms, but if we need to produce compilers for all these second-tier architectures then it's mostly just a matter of work.

It's also worth noting that Rust's support for most of these second-tier architecture amounts to just producing builds - except for arm/android we do not verify that they actually produce working code, and they often do not. Getting all these architectures under _test_ will truly be difficult. Just building them is mostly a matter of cross-compiling and uploading to S3. Testing requires hardware, emulators, etc.
(In reply to Brian Anderson from comment #6)
> Fedora also wants s390, ppc64 and ppc64le, and Martin Stransky described the
> need as 'critical'. But it's not clear to me still whether the lack of
> support for any particular architecture is a showstopper for either Debian
> or Fedora, whether they can just drop Firefox from those architectures. It's
> also not clear to me that they need a compiler *hosted* for those platform -
> in theory they could cross-compile, but I imagine for simplicity all
> architectures are purely self-hosting, that is, they use build servers that
> correspond to that architecture. I'll try to clarify.

For Debian that's the case. No cross-compilation is done for any package (which is actually quite painful in some cases, like Firefox, where building (which, incidentally, includes running some tests) on the slowest build slaves on the slowest architectures, can take more than 24 hours)

> It's also worth noting that Rust's support for most of these second-tier
> architecture amounts to just producing builds - except for arm/android we do
> not verify that they actually produce working code, and they often do not.
> Getting all these architectures under _test_ will truly be difficult. Just
> building them is mostly a matter of cross-compiling and uploading to S3.
> Testing requires hardware, emulators, etc.

I'm not at all convinced you need to do that on your end. If a second-tier port is essentially about triples and LLVM support, there are small chances they will break with Rust changes, right? So as long as people interested in second-tier ports build and test them, and you accept patches when something broke, I don't see why you'd need to provide (untested) cross-compiled versions of rustc for those platforms. Especially now that you can build rustc $version+1 with rustc $version.
(In reply to Brian Anderson from comment #6)
> Fedora also wants s390, ppc64 and ppc64le, and Martin Stransky described the
> need as 'critical'. 

Sorry, I was misleading here. Critical for Red Hat is that next ESR cycle (Firefox 52) will not require Rust so we have time to prepare build environment.
(In reply to Brian Anderson from comment #6)
> Fedora also wants s390, ppc64 and ppc64le, and Martin Stransky described the
> need as 'critical'. But it's not clear to me still whether the lack of
> support for any particular architecture is a showstopper for either Debian
> or Fedora, whether they can just drop Firefox from those architectures. It's
> also not clear to me that they need a compiler *hosted* for those platform -
> in theory they could cross-compile, but I imagine for simplicity all
> architectures are purely self-hosting, that is, they use build servers that
> correspond to that architecture. I'll try to clarify.

Fedora does not cross-compile any package and we use hosted compilers with all the issues Mike mentioned. And yes, Fedora can drop Firefox on specific arches and ship Firefox only on the working ones.
Landry & Martin, are OpenBSD and NetBSD prepared to build Firefox with --enable-rust at least on x86 architectures?

https://www.mail-archive.com/dev-platform@lists.mozilla.org/msg20261.html
(In reply to Jan Beich from comment #10)
> Landry & Martin, are OpenBSD and NetBSD prepared to build Firefox with
> --enable-rust at least on x86 architectures?
> 
> https://www.mail-archive.com/dev-platform@lists.mozilla.org/msg20261.html

Lol, as if we had any choice. amd64 will, i386 doesnt have a working rust yet.
(In reply to Landry Breuil (:gaston) from comment #11)
> (In reply to Jan Beich from comment #10)
> > Landry & Martin, are OpenBSD and NetBSD prepared to build Firefox with
> > --enable-rust at least on x86 architectures?
> > 
> > https://www.mail-archive.com/dev-platform@lists.mozilla.org/msg20261.html
> 
> Lol, as if we had any choice. amd64 will, i386 doesnt have a working rust
> yet.

No, we are not ready for that.
Firefox "portability" has become a bad joke, last working version on big endian machines for me is 44.
I am about to completely punt on the project and search for alternatives.
I'm using netsurf on powerpc, and midori on i386/low-end atom...
(In reply to Martin Husemann from comment #12)
> Firefox "portability" has become a bad joke, last working version on big
> endian machines for me is 44.

I got TenFourFox 45 off the ground with a couple endian hacks, but we don't use Skia at all in Tiger, which is where I suspect other major endian problems lurk (though not relevant to this bug specifically). Contact me if you want them -- they're in our changesets too.
(In reply to Martin Husemann from comment #12)
> Firefox "portability" has become a bad joke, last working version on big
> endian machines for me is 44.

I take that back and would like to honestly apologize.
Writing this comment with Firefox 50.0.2 on sparc64...

Another NetBSD developer is working on making rust more portable to other NetBSD platforms, including i386 and sparc64 (which is quite painful work).

So: we are not ready, but we are getting closer.

Still I think requiring an essentially non- (or at least hard-) portable language like Rust is not a good idea for a portable project, but only time will tell if this was a good decision or not.
As an update, I'm intending to resolve this bug soon.

If you're maintaining Firefox on a platform without good rust support you'll be able to continue for a bit, but starting with Firefox 55 I expect not having rust to require disabling features affecting user security. For Firefox 53 (release 2017 April 18) you can build with `--disable-rust`. For Firefox 54 you'll probably be able to revert changes from this bug and still get a working build. We're not testing non-rust configurations and intend to land code which makes it non-optional, so keeping a --disable-rust variant working will be increasingly difficult after that.
(In reply to Martin Husemann from comment #15)

> [...] only time will tell if this was a good decision or not.

Very true, and sorry for adding to the port burden. We believe it will be worthwhile. Thanks for your hard work keeping Firefox running!
Blocks: sb-audio
Comment on attachment 8832681 [details]
Bug 1284816 - Remove --disable-rust.

https://reviewboard.mozilla.org/r/108876/#review110126

::: build/moz.configure/rust.configure:7
(Diff revision 2)
>  # vim: set filetype=python:
>  # This Source Code Form is subject to the terms of the Mozilla Public
>  # License, v. 2.0. If a copy of the MPL was not distributed with this
>  # file, You can obtain one at http://mozilla.org/MPL/2.0/.
>  
> -option('--disable-rust', help='Don\'t include Rust language sources')
> +rustc = check_prog('RUSTC', ['rustc'], allow_missing=True)

Might want a comment indicating that `rust_compiler` below makes this required. Alternately, you could just remove `allow_missing`, but we would have a less-good error message.
Attachment #8832681 - Flags: review?(ted) → review+
Assignee: nobody → giles
As a followup we should remove the `MOZ_RUST` subst and define.
Comment added per review. Carrying r=ted.
Blocks: 1336153
https://hg.mozilla.org/mozilla-central/rev/eb1677e32680
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
Just as a heads-up: On Debian, rustc currently builds on i386 (which is actually i686), amd64 (x86_64) and arm64 [1]. According to Debian's Rust maintainers, maintaining Rust on non-x86 besides arm64 seems to be very troublesome and involved [2].

So, with the current situation, making rustc a requirement for Firefox will effectively kill Firefox on everything but i386, amd64 and arm64 for Debian unless someone is really willing to pick up all the extra burden to fix rustc on the non-x86 platforms.

I honestly think that killing the portability of Firefox is a bad decision and might lead to Debian or other projects forking Firefox again like it happened with Iceweasel to allow keep using the browser on targets like the Raspberry Pi or modern embedded MIPS machines.

> [1] https://buildd.debian.org/status/package.php?p=rustc&suite=sid
(In reply to John Paul Adrian Glaubitz from comment #26)
> According to Debian's Rust
> maintainers, maintaining Rust on non-x86 besides arm64 seems to be very
> troublesome and involved [2].

^ Missing link
I think the Rust team was planning on upping the breadth of the official builds, and this was gated on the CI system being overhauled (in progress, should be done soon). So that should not be the issue by then.


Note that you don't need to fork firefox; you can disable the rust dependency without a fork. Until Firefox 53 (last release of 2017), disabling Rust will not cause any loss of functionality. As :rillian mentioned this start causing features to go missing from Firefox 54, in which case a fork doesn't help unless you are going to implement the features yourself (at which point just fixing the rustc problems, if not already fixed by the Rust team, would be easier?).
(In reply to Botond Ballo [:botond] from comment #27)

> ^ Missing link


I suspect it's http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-20161226/000758.html


We had breakages on ARM recently.
(In reply to Manish Goregaokar [:manishearth] from comment #29)
> (In reply to Botond Ballo [:botond] from comment #27)
> 
> > ^ Missing link
> I suspect it's
> http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-
> 20161226/000758.html

Yes, this one. Sorry and thanks for helping me out :).
(In reply to Manish Goregaokar [:manishearth] from comment #28)
> I think the Rust team was planning on upping the breadth of the official
> builds, and this was gated on the CI system being overhauled (in progress,
> should be done soon). So that should not be the issue by then.

"Should" is a conjunctive which is the main problem I am seeing. It's been promised for a while but still hasn't happened.

> Note that you don't need to fork firefox; you can disable the rust
> dependency without a fork. Until Firefox 53 (last release of 2017),
> disabling Rust will not cause any loss of functionality.

Yes, I'm aware of that. But with this bug report being resolved, the direction has already been set.

> As :rillian mentioned this start causing features to go missing from Firefox 54,
> in which case a fork doesn't help unless you are going to implement the features yourself

Well, the fork would, of course, be based on a version prior the huge change 54 will bring.

> (at which point just fixing the rustc problems, if not already fixed by the Rust team, would be easier?).

I'm not sure if all that is going to happen soon. According to Sylvestre's message [1], LLVM - which is used by Rust - doesn't even run the testsuite on most architectures, so I'm not sure how much sense it makes to work with a compiler which doesn't pass its own testsuite.

I'm just skeptical that rustc will reach Tier1 status on non-x86 targets in the foreseeable future which is why I'm worried by this decision. 32-bit ARM is one of the biggest Linux markets and not having Firefox available there would be at least weird.

> [1] http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-20161226/000759.html
> It's been promised for a while but still hasn't happened.

Most of the work is done now. It's been going on for most of this month and the last. Moving release packaging to the new system is a bit more involved, but AIUI the work is going on.

This effort is a large one, I wouldn't expect it to happen immediately.


> Well, the fork would, of course, be based on a version prior the huge change 54 will bring.

There is literally no point in doing that. --disable-rust on 54 only means that you won't get some speed improvements. --disable-rust on 55 means that you may not get newer features from 55 (because they will be written in pure Rust), which is what happens when you fork anyway, except if you fork you may also lose out on other patches. Unless you're willing to rewrite these features in C++ in the fork, the fork will always be worse off than --disable-rust.


I'm not sure what the plan for the far future is with respect to dropping support completely for non-Rust builds (at which point I expect a stronger guarantee from Rust), but for now just disabling Rust is unambiguously a better solution than forking.

--------------


Could you list all of the platforms that you need?
(In reply to Manish Goregaokar [:manishearth] from comment #32)
> --disable-rust on 54 only means
> that you won't get some speed improvements. --disable-rust on 55 means that
> you may not get newer features from 55 (because they will be written in pure
> Rust)

I'm a bit confused. Isn't the patch that landed in this bug removing --disable-rust from 54 trunk, so that there's no such thing as "--disable-rust on 54" and "--disable-rust on 55"?
(In reply to John Paul Adrian Glaubitz from comment #31)
> Well, the fork would, of course, be based on a version prior the huge change
> 54 will bring.

You would be better served IMHO using 52ESR as your base, which doesn't require Rust either, and gets security patches. That gives you until Fx59 to get it right, if getting rustc up on your architectures is the desired goal. That's what I would have done with TenFourFox, except lots of breaking changes occurred between 45 and 52 that would require unmaintainable amounts of churn to revert, so we're forking after 45 (I'm a solitary port maintainer nowadays, and I don't have enough stamina to maintain a browser *and* LLVM *and* rustc on OS X Tiger PowerPC).
(In reply to Botond Ballo [:botond] from comment #33)
> (In reply to Manish Goregaokar [:manishearth] from comment #32)
> > --disable-rust on 54 only means
> > that you won't get some speed improvements. --disable-rust on 55 means that
> > you may not get newer features from 55 (because they will be written in pure
> > Rust)
> 
> I'm a bit confused. Isn't the patch that landed in this bug removing
> --disable-rust from 54 trunk, so that there's no such thing as
> "--disable-rust on 54" and "--disable-rust on 55"?

It is, and I misunderstood the situation (there's a different bug which added disable-rust) my apologies.


Yep, ESR makes sense here.


Does the rust package actually need the LLVM package to exist? AIUI rust uses a pinned LLVM version. I don't think it can be built with an LLVM release yet (this used to be sort of possible, but stopped being the case at some point). At least, that's how I understand the situation, this could have changed again. Is it acceptable to build the LLVM compiler  pinned as a submodule? If rust does run the testsuite for $target (doesn't right now, but should in the future), would that be sufficient, or is it the case that rust must use an LLVM release for packaging?
(In reply to Manish Goregaokar [:manishearth] from comment #35)
> Does the rust package actually need the LLVM package to exist? AIUI rust
> uses a pinned LLVM version. I don't think it can be built with an LLVM
> release yet (this used to be sort of possible, but stopped being the case at
> some point).

Rust does still support external LLVM 3.7+, and I'm pretty sure bors even has a 3.7 test to make sure this keeps working.  We build it this way on Fedora, currently using our packaged llvm-3.8 on F24/F25 and llvm-3.9 on rawhide.  Using a bundled LLVM would be permitted by our packaging policy, but not preferred.
That's great to hear!
Some clarifications about Rust platform support and CI.

The coverage improvements Rust has promised to this point to support Firefox on Linux are implemented: mips, ppc, and s90x builds of rustc [1] [2]. If there is more needed or expected from the Rust team at this time, I am not aware, and would like to discuss it (ideally not on this bug).

[1]: https://github.com/rust-lang/rust/issues/36006
[2]: https://github.com/rust-lang/rust/issues/36015

Today we don't test Rust on non-x86 Linux targets, but there is a PR open to add an ARM target tested via qemu [3].

[3]: https://github.com/rust-lang/rust/pull/39400

There are no plans presently to increase coverage beyond that.
I believe we've said this before, but while it's unfortunate that requiring Rust may mean that building Firefox on non-x86/arm CPU architectures may be hard or impossible we are investing heavily into Rust and we believe that the benefits to Firefox are worthwhile. The Rust team seems generally interested in broadening the number of platforms supported by the Rust toolchain, so I think the situation will improve over time. Regardless, we're sympathetic but we are still going to proceed with this plan.
(In reply to Brian Anderson from comment #38)
> The coverage improvements Rust has promised to this point to support Firefox
> on Linux are implemented: mips, ppc, and s90x builds of rustc [1] [2]. If
> there is more needed or expected from the Rust team at this time, I am not
> aware, and would like to discuss it (ideally not on this bug).

Implementation isn't enough though. The ports for these targets have to be actively maintained and tested, otherwise they aren't really something you want to rely on for production code. I have seen too many compiler bugs in gcc and others on more exotic targets for exactly this reason.

> Today we don't test Rust on non-x86 Linux targets, but there is a PR open to
> add an ARM target tested via qemu [3].

Running a compiler testsuite on QEMU is courageous, to say the least. In Debian, we don't allow builds for release architectures on emulated hardware for very good reasons. There have been multiple cases in the past where compilers generated wrong code when executed on QEMU. I have made plenty of experiences in this regard.

Really, you *need* a complete CI setup with real hardware for all the ports you actually want to support like Google does for golang [1]. Otherwise claiming to support any of such architectures is a bit dishonest.

What are all these fancy new memory safety and code quality of Rust worth if you can't actually guarantee that the compilers generates correct code? I have the impression that the Rust team takes this responsibility a bit too light-heartedly.

> [1] https://build.golang.org/
Like Brian already said, discussion of the suitableness of Rust's support for these architectures and/or their CI system should be taken someplace else.
Will any version of rustc be ok or will the dependencies be "recentish rustc" (for whatever value of recentish)? Asked the other way around: if we manage to have rustc setup on a build system once, will we be required to track rust developement?
(In reply to Martin Husemann from comment #42)
> Will any version of rustc be ok or will the dependencies be "recentish
> rustc" (for whatever value of recentish)? Asked the other way around: if we
> manage to have rustc setup on a build system once, will we be required to
> track rust developement?

I think you already know the sad answer to this :) I just hope the 'you need the very latest release of $whatever' trend doesnt reach release/beta channels....
I suspect there will be a little while here where we wind up requiring the latest stable version. Rust 1.15 added support for custom derive macros, which are needed for quite a few useful things (like auto-generated serialization code with serde). Rust is still a young language, and Servo/Gecko development is driving some feature work. I don't expect that this will be a long-term trend, however.
(In reply to Landry Breuil (:gaston) from comment #43)
> (In reply to Martin Husemann from comment #42)
> > Will any version of rustc be ok or will the dependencies be "recentish
> > rustc" (for whatever value of recentish)? Asked the other way around: if we
> > manage to have rustc setup on a build system once, will we be required to
> > track rust developement?
> 
> I think you already know the sad answer to this :) I just hope the 'you need
> the very latest release of $whatever' trend doesnt reach release/beta
> channels....

Is this some sort of joke? Do you really expect distributions to constantly update to the latest git snapshot of the Rust compiler just because they are updating their Firefox package?

I'm sorry, but this isn't maintainable in the long run. Because if we have to update the Rust compiler to be able to build the new Firefox version, we will most likely unrelated Rust packages and that is not acceptable.

Stable ABIs and APIs exist for very practical reasons.
John, we are aware of all that. However, in order to move fast, we made the call to follow rust's versions.
We are aware that this is going to be a pain for Linux distro (especially Debian with the long cycles) but as Ted said, this won't be a long term requirement.
Until then, in Debian, ESR52 doesn't require Rust to compile. Debian stable will have some time to figure out a solution
> Do you really expect distributions to constantly
> update to the latest git snapshot of the Rust compiler just because they are
> updating their Firefox package?


The latest stable distribution, not the latest git snapshot. Rust has a stable release every six weeks.

Rust does have a stable API. The point is that Firefox will wish to use newer APIs, at least for the near future (As ted says in the long-term this shouldn't be the case).
(In reply to Sylvestre Ledru [:sylvestre] from comment #46)
> John, we are aware of all that. However, in order to move fast, we made the
> call to follow rust's versions.

This is going to create lots of issues for the distributions.

> We are aware that this is going to be a pain for Linux distro (especially
> Debian with the long cycles) but as Ted said, this won't be a long term
> requirement.

I'm not sure why you can't make Rust stable first, then go ahead and move into production. You are basically using the distributions and their users as test guinea-pigs which isn't acceptable.

> Until then, in Debian, ESR52 doesn't require Rust to compile. Debian stable
> will have some time to figure out a solution

There isn't really any other way to resolve this than providing a stable API and language. Google is doing that with golang, why can't Mozilla do it for Rust?
(In reply to Manish Goregaokar [:manishearth] from comment #47)
> Rust does have a stable API. The point is that Firefox will wish to use
> newer APIs, at least for the near future (As ted says in the long-term this
> shouldn't be the case).

Then most distributions will not be able to ship the latest version of Firefox for the time being.
I don't think this is the place to continue the discussion. Please move the discussion to dev-platform or to rust packaging list.
Please see those threads with Rust upstream, Rust downstream and Firefox people:
http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-20160829/000548.html
http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-20160822/000541.html
http://lists.alioth.debian.org/pipermail/pkg-rust-maintainers/Week-of-Mon-20160822/000534.html
No longer blocks: sb-audio
Blocks: 1346665
Blocks: 1341967
FYI, I've tried to keep --disable-rust alive downstream but quickly gave up after bug 1261841 landed. It did buy some time for FreeBSD to package Rust on aarch64 and NetBSD to package Cargo.
https://github.com/freebsd/freebsd-ports/blob/7d67a39637dc/www/firefox/files/patch-rust-option
FYI: I tried to get rust working properly on NetBSD/sparc64 but did not get very far. Still trying that, but it will be slow progress. Rust build times do not help. It may be an interesting academic language, but is a complete disaster as implementation language of real world stuff.

IMHO Firefox should stop claiming to be open source. Rustc has no free bootstrap path so effectively is binary blob download only and we don't know what hidden gems it compiles into itself and the final products.

You may call me paranoid.
Martin, once you have a rust compiler in your repository, you can use "version-1" to build "version". No need of binary blobs anymore (we did that in Debian&Ubuntu to bootstrap it into the archive)
Yes, and how do you get the initial compiler without trusting some random binary blob?

There is no alternative rust compiler out there, and with no formal language spec it is not clear if any ever would appear.

Go does handle this correctly: they support an old version that can be bootstrapped from C code (and you can use various C compilers to do that). With that bootstrap version you can compile the real compiler, and then recompile it again with itself.
Note you /can/ build the rust compiler from the old Ocaml-based rust compiler. There's however a *lot* of intermediate steps for that.
I'm slowly working on getting a script for the bootstrap chain FWIW, so it might be possible to get clean blobs from a long, but automated process. The main issue is that the distribution mechanism changed a bunch of times, so while all the bootstrap points are well documented, the script for each step may be different.
(In reply to Manish Goregaokar [:manishearth] from comment #56)
> I'm slowly working on getting a script for the bootstrap chain FWIW, so it
> might be possible to get clean blobs from a long, but automated process. The
> main issue is that the distribution mechanism changed a bunch of times, so
> while all the bootstrap points are well documented, the script for each step
> may be different.

Come to think of it, is that actually very useful to script? I mean the reason people would want to bootstrap from the original non-rust compiler is because they don't trust the shipped binaries. So they would need to audit the Ocaml compiler code, and the source code of all the intermediate rust compilers... Considering the amount of work to verify there's no trusting trust issue in any of those steps, one might as well implement a new rust compiler (doesn't even need to do all the borrow checking, etc. just enough to produce a binary that can then self-compile)
Yes, there's still that trust issue, but a trusting trust attack is unlikely to have survived through the compilers of two different languages.

There probably are alternate ocaml compilers out there if you want to diverse double compile to be sure.

I mean, that was the point of the original trusting trust paper in the first place! You "have*  to trust _some_ level of the system. The OCaml compiler is a reasonable stopping point, and security folks I've talked to about this agree. But yes, if you don't trust that then it's a problem. You could follow ocamls bootstrap chain as well I guess.

The main reason I wish to do this is so once rustc has deterministic output (I hear we're close) it will be possible for independent folks to verify these builds from scratch.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.