Use "codegen-units = 4" for compiling Rust code in debug mode

RESOLVED FIXED in Firefox 55

Status

RESOLVED FIXED
2 years ago
10 months ago

People

(Reporter: SimonSapin, Assigned: SimonSapin)

Tracking

(Depends on: 1 bug)

Trunk
mozilla55

Firefox Tracking Flags

(firefox55 fixed)

Details

Attachments

(1 attachment)

(Assignee)

Description

2 years ago
rustc supports parallelizing code generation. This makes compilation faster, but also makes the compiler program slower (because LLVM’s optimizer only sees part of the code at a time, which reduces inlining opportunities.) Given this tread-off, we may want to enable this is dev/debug mode but not in release mode.

Previous discussion:

https://internals.rust-lang.org/t/default-settings-for-parallel-codegen/519
https://internals.rust-lang.org/t/lets-talk-about-parallel-codegen/2759
http://www.ncameron.org/blog/how-fast-can-i-build-rust/
https://github.com/servo/servo/pull/14995

About hard-coding the number 4, Matt Brubeck wrote in https://github.com/servo/servo/pull/14995#issuecomment-272525504

> Using the number of CPUs may not even be the best choice on all systems,
> because codegen-units has decreasing returns
> [http://www.ncameron.org/blog/how-fast-can-i-build-rust/],
> and because on an n-core machine parts of the build would have up to
> n rustc processes running, for a total of up to n² codegen threads.
> (Ideally, Cargo would have a feature like Make's “job server”
> to coordinate parallel builds.)
Comment hidden (mozreview-request)
(In reply to Simon Sapin (:SimonSapin) from comment #0)
> > Using the number of CPUs may not even be the best choice on all systems,
> > because codegen-units has decreasing returns
> > [http://www.ncameron.org/blog/how-fast-can-i-build-rust/],
> > and because on an n-core machine parts of the build would have up to
> > n rustc processes running, for a total of up to n² codegen threads.
> > (Ideally, Cargo would have a feature like Make's “job server”
> > to coordinate parallel builds.)

Mike, ISTR you were working on this...did you get anywhere with that?
Flags: needinfo?(mh+mozilla)
Making debug builds faster certainly sounds like a good idea!

I think the only concern would be that Cargo's processes are going to compete for CPU time with other processes started up by `make`, resulting in more processes than the user might have expected running at once, and *maybe* resulting in a slower build?  But there's not much we can do about that unless we get jobserver support in Cargo.
We're already in that situation since we're invoking cargo and it's spawning however many rustc processes it wants in parallel with whatever else make is doing. Making rustc use more cores will certainly make cpu contention worse from that aspect, but I've heard complaints about Rust build times in Gecko already so it might be a net win for people actively working on Rust code.
(Assignee)

Comment 5

2 years ago
This is probably a niche use case, but I have icecream configured to compile C/C++ code on many CPU cores across multiple machines in the office. So on a full build, C/C++ files are done compiling relatively quickly and the rest of the time is spent compiling Rust code on one core (because the end of the crate dependency graph tends to be narrow).

(I might help to tweak the build’s schedule to start building Rust things as early as possible, but that’s another story.)
Yeah, there are basically two use-cases:
(1) Building the whole browser. In this case, we want cargo to play well with other processes.
(2) Explicit ./mach build toolkit/library/rust (or the equivalent when building something else in the tree that subsequently causes toolkit/library to be rebuilt). In this case, we currently just use one core to rebuild most of the complicated rust stuff, which sucks.

I wonder if there's a way to separate (1) and (2).

Comment 7

2 years ago
mozreview-review
Comment on attachment 8853750 [details]
Bug 1352815 - "codegen-units = 4" for Rust code in debug mode

https://reviewboard.mozilla.org/r/125816/#review130418

I think we do want this, so let's land this to  make life easier for some people.
Attachment #8853750 - Flags: review+
Assignee: nobody → simon.sapin

Comment 8

2 years ago
Pushed by ecoal95@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/760d66e6c82e
"codegen-units = 4" for Rust code in debug mode r=froydnj

Comment 9

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/760d66e6c82e
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox55: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Flags: needinfo?(mh+mozilla)
Depends on: 1375507

Updated

10 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.