1613355 - 60 - 60% sccache requests_not_cacheable (windows2012-aarch64) regression on push bc4e1d55f5a42ea61a0e1198b678da2a02989f5c (Tue February 4 2020)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Florin Strugariu [:Bebe]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=bc4e1d55f5a42ea61a0e1198b678da2a02989f5c

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

60% sccache requests_not_cacheable windows2012-aarch64 debug aarch64 70.00 -> 112.00
60% sccache requests_not_cacheable windows2012-aarch64 opt aarch64 70.00 -> 112.00

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24840

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Comment 1

[(Away)]

Before:

[task 2020-02-04T22:54:14.290Z] 22:54:14     INFO -  Non-cacheable reasons:
[task 2020-02-04T22:54:14.290Z] 22:54:14     INFO -  crate-type                            62
[task 2020-02-04T22:54:14.290Z] 22:54:14     INFO -  -                                      6
[task 2020-02-04T22:54:14.290Z] 22:54:14     INFO -  -Fi                                    2

After:

[task 2020-02-04T23:16:03.399Z] 23:16:03     INFO -  Non-cacheable reasons:
[task 2020-02-04T23:16:03.399Z] 23:16:03     INFO -  crate-type                           62
[task 2020-02-04T23:16:03.399Z] 23:16:03     INFO -  -fsyntax-only                        42
[task 2020-02-04T23:16:03.399Z] 23:16:03     INFO -  -                                     6
[task 2020-02-04T23:16:03.399Z] 23:16:03     INFO -  -Fi                                   2

Those -fsyntax-only are coming from the moz.build for clang-plugin/tests -- these are files that we did not compile before the patch.

In other words, we did not lose sccache on any existing files. We merely gained some non-sccache compilations in the build. (And I'll note that we are still well below the win64 value of ~156 non-cacheable requests.) So this is not really a regression and could easily be a WONTFIX if necessary.

However. Perhaps this points out an opportunity to improve things all around, not just for aarch64-windows. I see that sccache marks -fsyntax-only with TooHardFlag, although maybe that comes from a long time ago. Chris: do you think it would be worthwhile to revisit supporting that flag? Alternatively, what do you think about removing -fsyntax-only from clang-plugin tests? I suspect that the flag was just trying to save some time, but paradoxically, it's making things slower.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

As far as I can tell -fsyntax-only means the compiler wont generate any output, so sccache wouldn't be helpful in this case. It might be interesting to test if removing that flag and therefore turning on caching for these files speeds things up significantly, but I sort of doubt it would be significant with respect to the entire build.

Comment 3

[(Away)]

You're right, we're talking about very short times here. On my machine, it takes about 1.5s to build all the clang-plugin tests with -fsyntax-only, and 1.6s if we allow codegen. It's not even clear that the sccache overhead would be less than the 0.1s of potential savings.

There might be an argument that it would still be worthwhile to get rid of these 42 requests_not_cacheable entries that are constantly present, in order to make actual regressions stand out better, but maybe that's a stretch.

Comment 4

[(Away)]

As discussed above there is nothing to worry about from the perf side. I wouldn't mind reducing the levels for the sake of cleanup, but it's not really important.