1427313 - Windows builds somehow get lots of rust sccache misses

Status: NEW

(Firefox Build System :: General, enhancement)

Description

[Mike Hommey [:glandium]]

I've been doing multiple successive opt builds on all platforms while preparing landable patches for bug 1427312, and the Windows builds were all taking an awful long time, although they were all based on the exact same code (all based on the same central revision, with only changes under taskcluster/)

Looking at the cache misses in the sccache log shows something rather bad:

$ curl -sL https://queue.taskcluster.net/v1/task/JKINgSiITMGWNLawVluQ1g/runs/0/artifacts/public/build/sccache.log.gz | zgrep "Cache miss"
DEBUG:sccache::compiler::compiler: [darling]: Cache miss in 0.015 s
DEBUG:sccache::compiler::compiler: [cssparser]: Cache miss in 0.013 s
DEBUG:sccache::compiler::compiler: [serde]: Cache miss in 0.009 s
DEBUG:sccache::compiler::compiler: [dwrote]: Cache miss in 0.012 s
DEBUG:sccache::compiler::compiler: [selectors]: Cache miss in 0.015 s
DEBUG:sccache::compiler::compiler: [webrender_api]: Cache miss in 0.010 s
DEBUG:sccache::compiler::compiler: [style_traits]: Cache miss in 0.009 s
DEBUG:sccache::compiler::compiler: [style]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [webrender]: Cache miss in 0.009 s
DEBUG:sccache::compiler::compiler: [webrender_bindings]: Cache miss in 0.012 s
DEBUG:sccache::compiler::compiler: [city.obj]: Cache miss in 0.045 s
DEBUG:sccache::compiler::compiler: [ProfileReset.obj]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [Unified_cpp_toolkit_xre0.obj]: Cache miss in 0.012 s
DEBUG:sccache::compiler::compiler: [nsEmbedFunctions.obj]: Cache miss in 0.013 s
DEBUG:sccache::compiler::compiler: [nsAppRunner.obj]: Cache miss in 0.019 s
DEBUG:sccache::compiler::compiler: [nsBrowserApp.obj]: Cache miss in 0.012 s
DEBUG:sccache::compiler::compiler: [miniz_sys]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [bzip2_sys]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [flate2]: Cache miss in 0.010 s
DEBUG:sccache::compiler::compiler: [bzip2]: Cache miss in 0.010 s
DEBUG:sccache::compiler::compiler: [geckoservo]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [gkrust_shared]: Cache miss in 0.037 s
DEBUG:sccache::compiler::compiler: [gkrust_gtest]: Cache miss in 0.011 s
DEBUG:sccache::compiler::compiler: [gkrust_shared]: Cache miss in 0.023 s
DEBUG:sccache::compiler::compiler: [gkrust]: Cache miss in 0.010 s

That's most of the time-taking rust crates with cache misses...

Comment 1

[ISHIKAWA, Chiaki]

Can it be happening again now (but I am using linux for local testing.)
I am not sure if the bug is related to the recent posting to mozilla dev-platform mailing list.

https://groups.google.com/forum/#!topic/mozilla.dev.platform/EnZbltbwUeE
Must we rebuild all our rust code constantly

Comment 2

[ISHIKAWA, Chiaki]

Under linux, after setting

#
# Disable incremental compilation Aug 2019 when sccache is used.
#
mk_add_options "export CARGO_INCREMENTAL=0"

in my mozconfig, I don't think I see unnecessary compilations, but to be honest, these days, I update the M-C, C-C a few times a week and then full compilation occurs anyway and not paid much attention.
I can certainly that the rust compilation steps print out informative diagnostics regarding what are skipped and what are compiled and WHY these days and so I think the situation is under control now on my PC that runs linux for development.
I can't vouch for Windows, though.