1054892 - CCache stats reporting might not show when cache size is nearly full

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

When cache size is nearly full, the files in cache might be decreased after a successful build so that the stats reporting might not show. For example:

Before build:
cache directory                   /home/tlin/.ccache
cache hit (direct)                     598
cache hit (preprocessed)               168
cache miss                           22613
called for link                       1918
called for preprocessing               655
multiple source files                  316
compiler produced no output             10
compiler produced empty output           5
compile failed                         240
preprocessor error                      41
can't use precompiled header            10
bad compiler arguments                  54
unsupported source language            897
compiler check failed                    3
autoconf compile/link                  722
unsupported compiler option             98
no input file                          248
error hashing extra file                 7
files in cache                       12490
cache size                             2.8 Gbytes
max cache size                         3.0 Gbytes


After build:
cache directory                   /home/tlin/.ccache
cache hit (direct)                    1066
cache hit (preprocessed)               256
cache miss                           25242
called for link                       2134
called for preprocessing              1105
multiple source files                  316
compiler produced no output             10
compiler produced empty output           5
compile failed                         272
preprocessor error                      50
can't use precompiled header            10
bad compiler arguments                  63
unsupported source language            908
compiler check failed                    3
autoconf compile/link                  887
unsupported compiler option            101
no input file                          287
error hashing extra file                 7
files in cache                       10997
cache size                             2.6 Gbytes
max cache size                         3.0 Gbytes


Diff (before - after)
cache directory                   /home/tlin/.ccache
cache hit (direct)                     468
cache hit (preprocessed)                88
cache miss                            2628
called for link                        216
called for preprocessing               449
multiple source files                    0
compiler produced no output              0
compiler produced empty output           0
compile failed                          32
preprocessor error                       9
can't use precompiled header             0
bad compiler arguments                   9
unsupported source language             11
compiler check failed                    0
autoconf compile/link                  165
unsupported compiler option              3
no input file                           39
error hashing extra file                 0
files in cache                       -1246
cache size                        -209715.2 Kbytes
max cache size                         3.0 Gbytes

Comment 1

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

In the description, The diff should be "after - before", not "before - after."

Comment 2

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Attachment: Fix ccache status not shown after shrinking cache size. (v1)

The value of cache files or cache size might decrease under these
circumstances.

* The user manually changes the max cache size to a value smaller than
  the current cache size.
* The cache size is reaching the cache max size.

We should not assume both values will be increased after the build is
finished.

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8567024 [details] [diff] [review]
Fix ccache status not shown after shrinking cache size. (v1)

Review of attachment 8567024 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM.

Comment 4

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7a4c34ca4904

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/271b01ac0c8d

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/271b01ac0c8d