1575760 - Make `mach vendor rust` create a .cargo/config and check it in the tree

Status: RESOLVED FIXED

(Firefox Build System :: General, task)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1575760 - Generate .cargo/config as part of the build rather than as a configure subst file.

We'll require preprocessing that configure subst files don't allow in
the next change, so prepare for that.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1575760 - Make `mach vendor rust` create a .cargo/config and check it in the tree.

Maybe back when .cargo/config.in was added, the directory indicated for
vendored crates needed to be absolute. That is at least not the case
with the current supported versions of rust.

The current setup has a few caveats:

• .cargo/config.in has shown to become stale (it currently contains
  multiple unused entries)
• non-gecko build tasks have to generate a .cargo/config on their own if
  they want to use vendored crates
• in turn, non-gecko build tasks that don't, may unknowingly get their
  dependencies from crates.io (see the recent attempt at moving
  geckodriver builds to a separate task).

By checking in a .cargo/config file, we can alleviate the last two, but
that comes at the price of cargo update not wanting to act when
.cargo/config exists, because of the source replacement configuration.

But rust vendor gently generates a suitable configuration on its own, so
we can use that to generate a .cargo/config automatically. Which
addresses the first caveat of the current setup. That leaves us with
cargo update not working out of the box, but that just requires people
running it to manually remove .cargo/config first. Which is arguably
what rust wants you to do in the first place. It's kind of incidental
that we started with a .cargo/config.in rather than .cargo/config.

Now, while a simple .cargo/config works, that's not enough for the case
where the objdir doesn't live inside the source directory. In that case
cargo looks for the configuration from the objdir, and fails to find it.
So we still need a .cargo/config.in, which we generate with a little
trick.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9e0c871919f0
Generate .cargo/config as part of the build rather than as a configure subst file. r=firefox-build-system-reviewers,chmanchester
https://hg.mozilla.org/integration/autoland/rev/80fa83b977a0
Make `mach vendor rust` create a .cargo/config and check it in the tree. r=nalexander

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/9e0c871919f0
https://hg.mozilla.org/mozilla-central/rev/80fa83b977a0

Comment 5

[Jeff Muizelaar [:jrmuizel]]

It seems like this broke running gfx/wr/wrench$ cargo run because the

[source.crates-io]
replace-with = "vendored-sources"

makes it so we can't find out of tree dependencies like mozangle

Comment 6

[Mike Hommey [:glandium]]

... which brings the question: why is it not vendored in the first place?

Comment 7

[Jeff Muizelaar [:jrmuizel]]

The dependencies are rather big. They include a separate copy of ANGLE, a copy of mesa (which looks to be about ~140MB) and a bunch of other crates for things like websockets and image decoding etc. It's very nice to be able to add dependencies for wrench without having to worry about them bloating the m-c repo.

Comment 8

[Dzmitry Malyshau [:kvark]]

AFAIK, we only want to vendor the dependencies that are needed to build Gecko. Mozangle is basically a dev-dependency for WebRender (since Wrench is only used during development).