755145 - Define CCACHE_CPP2 when using ccache and Clang

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

We should probably define CCACHE_CPP2 when using ccache and Clang. See bug 754988.

Comment 1

[Gregory Szorc [:gps]]

As part of this, it would also be nice to tackle the general problem of not being able to pass the various CCACHE_* environment variables to ccache. Currently, if you add an export to your .mozconfig (e.g. "export CCACHE_BASEDIR=/Users/gps"), this has no effect (at least if you execute through client.mk - maybe if you launch the Makefile in the objdir directly it works). Furthermore, you can't ac_add_options --with-ccache="CCACHE_BASEDIR=/Users/gps /usr/local/bin/ccache" because configure is testing -e and -z on the string value.

Perhaps we could solve this by adding CC_PRE and CXX_PRE or something similar? That would also give us the backdoor to easily proxy all compile commands through another script (e.g. something that intercepted compiler warning output and aggregated information or Clang's scan-build tools). In the current world, getting complex control requires disabling ccache. Just a thought.

Comment 2

[Gregory Szorc [:gps]]

I don't have any specifically quantitative evidence to back this up, but it seems that setting CCACHE_CPP2 results in my ccache filling up quicker. We should probably measure this and run it past RelEng before turning this on by default.

Comment 3

[Gregory Szorc [:gps]]

(In reply to Gregory Szorc [:gps] from comment #2)
> I don't have any specifically quantitative evidence to back this up, but it
> seems that setting CCACHE_CPP2 results in my ccache filling up quicker. We
> should probably measure this and run it past RelEng before turning this on
> by default.

Upon further analysis, the results appear to be similar to non-CPP2 builds. Maybe it was CCACHE_BASEDIR (which I set at the same time) affecting things.

Comment 4

[Joshua Cranmer [:jcranmer]]

Here's a trick I chanced upon:
export CC="env ENV_VAR=foo clang"

Comment 5

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: Define CCACHE_CPP2 when using ccache and Clang

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 643582 [details] [diff] [review]
Define CCACHE_CPP2 when using ccache and Clang

Review of attachment 643582 [details] [diff] [review]:
-----------------------------------------------------------------

This is beyond my comfort to give feedback on.

Mike Hommey has been doing a lot of work on autoconf foo lately and might have some things to say though.

Comment 7

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

I did some simple benchmarking of my laptop:

ccache build 1
real	36m40.916s
user	99m18.844s
sys	12m22.869s

ccache build 2
real	11m53.582s
user	9m9.587s
sys	3m22.222s

no ccache
real	32m11.518s
user	90m58.849s
sys	8m43.691s

Which suggests it is worth enabling, but really depends on the cache hit rate we have on try. Do we have any numbers on that?

Comment 8

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

p.s.: I used a 10GB cache on that benchmark.

Comment 9

[Mike Hommey [:glandium]]

Comment on attachment 643582 [details] [diff] [review]
Define CCACHE_CPP2 when using ccache and Clang

Review of attachment 643582 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/autoconf/compiler-opts.m4
@@ +206,5 @@
>          _WARNINGS_CXXFLAGS="${_WARNINGS_CXXFLAGS} $1$2"
>      fi
>  ])
> +
> +AC_DEFUN([MOZ_TOOL_VARIABLES],

I'd move that in a separate toolchain.m4 file.

Comment 10

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/3e4e9e518bef