2059623 - Sccache base dirs are not always set

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Mike Hommey [:glandium]]

There are two very common scenarios where sccache base dirs are not set properly, other than the one mentioned in the documentation about a sccache server already being running:

• A full ./mach build when a server is not already running (essentially, the most expected case). The problem here is that sccache starts during configure, but is not fed the SCCACHE_BASEDIRS variable. A work around here is to run ./mach configure, stop sccache, then run ./mach build.
• When the sccache server is client.mk-managed (e.g. automation), it is started without SCCACHE_BASEDIRS being set.

Arguably, sccache should handle SCCACHE_BASEDIRS on a per-invocation basis, and whatever was used when starting the server shouldn't be relevant (or be considered a default). But until that part is sorted out, we should at least work around the problem in the build system.

The same applies to other SCCACHE_* variables we set during configure that sccache relies upon (i.e. SCCACHE_VERBOSE_STATS doesn't count).

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 2059623 - Move Windows SDK detection to windows-toolchain.configure.

windows.configure is included after the compiler checks, but the Windows
SDK directory doesn't depend on the compiler in any way. Moving its
detection next to the other toolchain path lookups makes it available to
things that need to run before the first compiler invocation.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 2059623 - Set SCCACHE_BASEDIRS in the environment before invoking sccache.

sccache only reads SCCACHE_BASEDIRS when its server starts, and the
server is started by the first sccache invocation, which, during
configure, is one of the compiler checks. Because the variable was only
exported from config.mk at build time, the daemon left running after
configure had no base directories at all.

Compute the value before the compiler checks and set it in the
environment there, so that the daemon configure starts is configured
correctly, and so that configure's own compilations get the same cache
keys as the build's.

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 2059623 - Pass SCCACHE_BASEDIRS to the sccache daemon managed by client.mk.

In automation, the daemon is started from client.mk so that it inherits
make's jobserver, but SCCACHE_BASEDIRS is a configure output and client.mk
only knows about mozconfig options, so the daemon was started without it.
Add it to the environment mach runs client.mk with.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/85f1dec32951
https://hg.mozilla.org/integration/autoland/rev/f6cde698868b
Move Windows SDK detection to windows-toolchain.configure. r=firefox-build-system-reviewers,ahochheiden
https://github.com/mozilla-firefox/firefox/commit/6dc4b2df63f0
https://hg.mozilla.org/integration/autoland/rev/6a5437427b8b
Set SCCACHE_BASEDIRS in the environment before invoking sccache. r=firefox-build-system-reviewers,ahochheiden
https://github.com/mozilla-firefox/firefox/commit/5aef16c9b58b
https://hg.mozilla.org/integration/autoland/rev/883a33ae6c6a
Pass SCCACHE_BASEDIRS to the sccache daemon managed by client.mk. r=firefox-build-system-reviewers,ahochheiden

Comment 5

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/f6cde698868b
https://hg.mozilla.org/mozilla-central/rev/6a5437427b8b
https://hg.mozilla.org/mozilla-central/rev/883a33ae6c6a