(In reply to Mike Hommey [:glandium] from comment #3)
Maybe we should change that too, and only do codegen-units=1 on non-DEVELOPER_OPTIONS builds.
I've been looking into this some more and I think the situation is even a bit more complicated than I thought initially:
rustc defaults to 16 codegen units per crate. We definitely want to override this with
codegen-units=1 in non-DEVELOPER_OPTIONS builds. We might also want to override it for non-incremental DEVELOPER_OPTIONS builds (i.e. builds using sccache?), but see 2.i below.
cargo will invoke
rustc will use one of two settings:
- If the crate comes from an external source (e.g. crates.io) then
rustc is invoked non-incrementally, i.e. defaulting to 16 codegen units (if
codegen-units isn't specified explicitly somewhere). Given the high amount of parallel work we already have, it does not really make sense to also have per-crate parallelism.
- If the crate comes from a local source then
cargo will invoke
rustc incrementally, using many many codegen units.
So the current settings are actually pretty good:
- non-DEVELOPER_OPTIONS builds compile everything with
codegen-units=1 which is the only option that makes sense there.
- DEVELOPER_OPTIONS builds compile external crates with
codegen-units=1. And local crates are compiled with
-Cincremental which makes sense for those since we expect them actually change.
However, once codegen-units will start to have an effect in incremental mode, we'll get a bit of a problem with incremental builds:
- If we keep the
codegen-units=1 setting then the effectiveness of incremental compilation will be reduced by 70-90% because any tiny change will force
rustc to send the entire crate through LLVM again, which makes up the bulk of compilation time.
- If we just remove the
codegen-units=1 setting then all external crates will use rustc's default 16 codegen units setting, which introduces unnecessary overhead for these.
I think the second option is preferable but far from ideal. I'll try it out locally; let's see how it affects compile times.
It would be great if Cargo had a way of using specific settings for external crates in incremental mode (then we could just set
codegen-units=1 for just those) but I don't think that is possible at the moment.