59 bytes, text/x-review-board-request
rustc supports parallelizing code generation. This makes compilation faster, but also makes the compiler program slower (because LLVM’s optimizer only sees part of the code at a time, which reduces inlining opportunities.) Given this tread-off, we may want to enable this is dev/debug mode but not in release mode. Previous discussion: https://internals.rust-lang.org/t/default-settings-for-parallel-codegen/519 https://internals.rust-lang.org/t/lets-talk-about-parallel-codegen/2759 http://www.ncameron.org/blog/how-fast-can-i-build-rust/ https://github.com/servo/servo/pull/14995 About hard-coding the number 4, Matt Brubeck wrote in https://github.com/servo/servo/pull/14995#issuecomment-272525504 > Using the number of CPUs may not even be the best choice on all systems, > because codegen-units has decreasing returns > [http://www.ncameron.org/blog/how-fast-can-i-build-rust/], > and because on an n-core machine parts of the build would have up to > n rustc processes running, for a total of up to n² codegen threads. > (Ideally, Cargo would have a feature like Make's “job server” > to coordinate parallel builds.)
(In reply to Simon Sapin (:SimonSapin) from comment #0) > > Using the number of CPUs may not even be the best choice on all systems, > > because codegen-units has decreasing returns > > [http://www.ncameron.org/blog/how-fast-can-i-build-rust/], > > and because on an n-core machine parts of the build would have up to > > n rustc processes running, for a total of up to n² codegen threads. > > (Ideally, Cargo would have a feature like Make's “job server” > > to coordinate parallel builds.) Mike, ISTR you were working on this...did you get anywhere with that?
Making debug builds faster certainly sounds like a good idea! I think the only concern would be that Cargo's processes are going to compete for CPU time with other processes started up by `make`, resulting in more processes than the user might have expected running at once, and *maybe* resulting in a slower build? But there's not much we can do about that unless we get jobserver support in Cargo.
We're already in that situation since we're invoking cargo and it's spawning however many rustc processes it wants in parallel with whatever else make is doing. Making rustc use more cores will certainly make cpu contention worse from that aspect, but I've heard complaints about Rust build times in Gecko already so it might be a net win for people actively working on Rust code.
This is probably a niche use case, but I have icecream configured to compile C/C++ code on many CPU cores across multiple machines in the office. So on a full build, C/C++ files are done compiling relatively quickly and the rest of the time is spent compiling Rust code on one core (because the end of the crate dependency graph tends to be narrow). (I might help to tweak the build’s schedule to start building Rust things as early as possible, but that’s another story.)
Yeah, there are basically two use-cases: (1) Building the whole browser. In this case, we want cargo to play well with other processes. (2) Explicit ./mach build toolkit/library/rust (or the equivalent when building something else in the tree that subsequently causes toolkit/library to be rebuilt). In this case, we currently just use one core to rebuild most of the complicated rust stuff, which sucks. I wonder if there's a way to separate (1) and (2).
Comment on attachment 8853750 [details] Bug 1352815 - "codegen-units = 4" for Rust code in debug mode https://reviewboard.mozilla.org/r/125816/#review130418 I think we do want this, so let's land this to make life easier for some people.
Attachment #8853750 - Flags: review+
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/autoland/rev/760d66e6c82e "codegen-units = 4" for Rust code in debug mode r=froydnj
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox55: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
You need to log in before you can comment on or make changes to this bug.