Closed Bug 947256 Opened 6 years ago Closed 5 years ago

Report ccache statistics at the end of the build

Categories

(Firefox Build System :: General, enhancement, P3)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED
mozilla34

People

(Reporter: gps, Assigned: TYLin, Mentored)

References

Details

(Whiteboard: [lang=python])

Attachments

(1 file, 1 obsolete file)

I think it would be useful if |mach build| printed ccache hit/miss rates at the end of the build.

Implementation:

1) Run and parse `ccache -s` at the beginning of the build
2) Run and parse `ccache -s` at the end of the build
3) Compute some simple hit/miss rates and print them.

e.g.

ccache hit rate: 32%; miss rate: 68%; files in cache: 159693 (+145)

This would probably live in the |mach build| command, which is implemented in python/mozbuild/mozbuild/mach_commands.py.

FWIW, I have a theory hit rates in the wild aren't as high as people suspect they are and this print out should shed some light on the data (whatever it is).
I banged this one out real quick. Not the most robust code in the world.
Needs refinement before landing. Perhaps someone can take it the last
mile.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Severity: normal → enhancement
Priority: -- → P3
> Needs refinement before landing.
Gregory, it is not very clear what you expect from a potential contributor here. Could you explain a bit more?
Flags: needinfo?(gps)
The patch needs a few tests and some documentation.
Assignee: gps → nobody
Flags: needinfo?(gps)
Mentor: gps
Whiteboard: [mentor=gps][lang=python] → [lang=python]
Assignee: nobody → tlin
This patch add a line at the end of the build like follows:
ccache (direct) hit rate: 98.7%; (preprocessed) hit rate: 0.5%; miss rate: 0.8%

To run the test_ccachestats.py:
./mach python-test python/mozbuild/mozbuild/test/controller/

Gregory, if the documentation or tests are still incomplete, please give some
suggestions. Thanks.
Attachment #8464573 - Flags: review?(gps)
Comment on attachment 8464573 [details] [diff] [review]
Report ccache hit rate at the end of the build

Review of attachment 8464573 [details] [diff] [review]:
-----------------------------------------------------------------

This looks sane. I'm going to send this to Try then land it for you if it looks good.

::: python/mozbuild/mozbuild/test/controller/test_ccachestats.py
@@ +1,3 @@
> +# This Source Code Form is subject to the terms of the Mozilla Public
> +# License, v. 2.0. If a copy of the MPL was not distributed with this
> +# file, You can obtain one at http://mozilla.org/MPL/2.0/.

This new file will need to be added to python/moz.build.
Attachment #8464573 - Flags: review?(gps) → review+
Attachment #8345202 - Attachment is obsolete: true
Gregory, thank you for helping landing this patch. Excuse me for forgetting to mention your credit in the commit message since this patch is derived from your wip.
https://hg.mozilla.org/mozilla-central/rev/56b1d9be395b
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla34
Blocks: 1049263
QA Whiteboard: [qa-]
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.