1705376 - Investigate why the winapi crate is being rebuilt even when using the mozilla-central-workspace-hack

Status: RESOLVED FIXED

(Firefox Build System :: General, task, P3)

Description

[Gabriele Svelto [:gsvelto] (PTO)]

In bug 1704518 I've used the mozilla-central-workspace-hack crate to reduce the number of dependencies that were rebuilt just to build the mozwer-rust crate. However while all the other crates in the workspace-hack were now not being rebuild winapi still was. I'm filing this mostly to double-check why this is happening because it could provide measurable build time savings.

Comment 1

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: workspace-hack-hack.patch

I made this contraption to try and cut down the number of times the winapi crate was being rebuilt (and possibly other crates too) and to my great surprise it only cut down winapi rebuilds by one, but it also cut down the rebuilds of the following other crates by one: serde, serde_derive, serde_json, syn, thiserror, thiserror-impl, prost and prost-derive.

I'm probably doing it wrong :-/

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Hmm, thanks for the exploratory patch here.
I'm guessing that in cases like gfx/wr/direct-composition/Cargo.toml, you still want to specify the specific features that you want? Then, when Cargo runs, it'll see that it's included in the superset in workspace-hack, and only build winapi once?

I'll have more concrete answers when I dig into this.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Update: though not all the necessary features of the different projects existed in workspace-hack, adding them and re-building doesn't resolve this problem and avoid extra builds.

Interestingly, each extra build happens on a fresh cargo invocation against a different (in-tree) project.
This makes sense - for my personal projects, cargo doesn't re-use built dependencies, even if the features line up.

I'm going to dig in a little more (and apply some findings from this Rust thread), but I have a question for you, gsvelto: Which other crates were not being rebuilt? Do any of those exist between "projects" that we separately invoke cargo on?

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Clearing NI, I see where the gaps are (different out-dir, missing dependency on workspace-hack, missing feature in workspace-hack).

Comment 5

[Mike Hommey [:glandium]]

The ones in different out-dirs, we can't do anything about at the moment. It's a tradeoff. Build times actually get worse if we make everything use the same out-dir.

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1705376: Synchronize workspace-hack features and usage

Not all in-tree Rust libraries were using workspace-hack.
Additionally, some needed winapi features were missing from
workspace-hack's configuration.

Now, winapi is re-compiled less frequently on a full build.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1b643530229d
Synchronize workspace-hack features and usage r=firefox-build-system-reviewers,glandium

Comment 8

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/1b643530229d