1505040 - Add support for auto-bootstrapping some dependencies while running configure

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[(not currently active) Ted Mielczarek]

We have enough build dependencies that the cycle of "update your source checkout, try to build, realize you have to run `mach bootstrap` again" can get pretty tedious. It's even more pronounced for people that don't update and build on a daily basis, or for developers who have a build environment that they don't use regularly (like a Windows VM).

For build dependencies that we already pull from pre-built toolchains in bootstrap, or things like Rust installed via rustup that are easy to install and don't require invoking the system package manger, it would be great if we could teach configure to simply bootstrap a missing/outdated dependency and continue on. Even just supporting this for the set of things where it would be possible and on our Tier 1 platforms would make the build experience a lot nicer for many people.

Comment 1

[Kim Moir [:kmoir] ET]

Great idea Ted! I've personally run into this issue a number of times since I don't run builds as often as a regular developer.

Comment 2

[Mike Hommey [:glandium]]

Implementation note: this should be limited to local builds only (developer_options).

Comment 3

[Gregory Szorc [:gps]]

I want the default behavior for developer builds to be for the build system to manage toolchains for you automatically on tier 1 platforms. e.g. if you have a source checkout on a fresh machine, `mach build` will download the exact Clang, Rust, etc toolchains that CI uses for that revision and use them.

If you update your source revision and the toolchain changes, the new toolchain is downloaded and used automatically.

Whether this is implemented in configure or elsewhere, it doesn't matter.

Obviously we would need a way to disable this behavior, as it isn't appropriate everywhere.

FWIW we also need better support for multiple versions of toolchains in ~/.mozbuild. Right now, we e.g. install a single version of Clang to ~/.mozbuild/clang. If you have e.g. a mozilla-central and mozilla-beta checkout, they will compete with each other for toolchain management. We probably want some kind of LRU "cache" to hold toolchains. Artifact builds employ something like this.

Comment 4

[Mike Hommey [:glandium]]

`mach artifact toolchain` has a LRU cache of the downloaded bits.

Comment 5

[Gregory Szorc [:gps]]

So I guess it is just the extracted archive that doesn't support multiple "installs." I'd love to see an LRU cache there as well. But that is arguably for a separate bug.

Comment 6

[Alex Hochheiden [:ahochheiden]]

This has been done for various dependencies, recently bug 1686880 is notable.