1487322 - 53.98 - 74% sccache requests_not_cacheable (linux32, linux64, linux64-noopt) regression on push 2b045052d4aa (Wed Aug 29 2018)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=2b045052d4aad4e77f3b8fce35a40efeed4e19f1

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 74%  sccache requests_not_cacheable linux32 opt taskcluster-m4.4xlarge     50.00 -> 87.00
 74%  sccache requests_not_cacheable linux32 debug rusttests taskcluster-m5.4xlarge50.00 -> 87.00
 74%  sccache requests_not_cacheable linux32 debug taskcluster-c5.4xlarge   50.00 -> 87.00
 69%  sccache requests_not_cacheable linux64 debug rusttests taskcluster-c4.4xlarge54.00 -> 91.00
 69%  sccache requests_not_cacheable linux64-noopt debug taskcluster-c4.4xlarge54.00 -> 91.00
 69%  sccache requests_not_cacheable linux64 opt rusttests taskcluster-m4.4xlarge54.00 -> 91.00
 69%  sccache requests_not_cacheable linux64 debug taskcluster-c5.4xlarge   54.00 -> 91.00
 66%  sccache requests_not_cacheable linux32 opt rusttests taskcluster-m4.4xlarge50.00 -> 83.00
 66%  sccache requests_not_cacheable linux32 opt rusttests taskcluster-m5.4xlarge50.00 -> 83.00
 66%  sccache requests_not_cacheable linux32 debug rusttests taskcluster-m4.4xlarge50.00 -> 83.00
 65%  sccache requests_not_cacheable linux64 opt taskcluster-c5.4xlarge valgrind52.00 -> 85.92
 64%  sccache requests_not_cacheable linux32 opt rusttests taskcluster-c5.4xlarge50.00 -> 82.08
 64%  sccache requests_not_cacheable linux32 debug taskcluster-m4.4xlarge   50.00 -> 82.08
 64%  sccache requests_not_cacheable linux32 opt taskcluster-m5.4xlarge     50.00 -> 82.08
 64%  sccache requests_not_cacheable linux32 opt rusttests taskcluster-m5d.4xlarge50.17 -> 82.08
 63%  sccache requests_not_cacheable linux64 opt rusttests taskcluster-m5.4xlarge54.00 -> 87.92
 63%  sccache requests_not_cacheable linux64-noopt debug taskcluster-c5.4xlarge54.00 -> 87.92
 63%  sccache requests_not_cacheable linux64 opt rusttests taskcluster-c5.4xlarge54.00 -> 87.92
 63%  sccache requests_not_cacheable linux64 debug taskcluster-m4.4xlarge   54.00 -> 87.92
 63%  sccache requests_not_cacheable linux64 opt taskcluster-m4.4xlarge     54.00 -> 87.92
 62%  sccache requests_not_cacheable linux64 opt taskcluster-m4.4xlarge valgrind55.08 -> 89.00
 61%  sccache requests_not_cacheable linux32 opt taskcluster-c4.4xlarge     54.00 -> 87.00
 61%  sccache requests_not_cacheable linux32 debug taskcluster-c4.4xlarge   50.00 -> 80.25
 61%  sccache requests_not_cacheable linux32 opt taskcluster-c5.4xlarge     50.00 -> 80.25
 59%  sccache requests_not_cacheable linux64 debug rusttests taskcluster-c5.4xlarge57.08 -> 91.00
 59%  sccache requests_not_cacheable linux64 debug rusttests taskcluster-m4.4xlarge57.08 -> 91.00
 59%  sccache requests_not_cacheable linux64 opt taskcluster-c4.4xlarge     57.08 -> 91.00
 57%  sccache requests_not_cacheable linux64 opt taskcluster-c4.4xlarge valgrind56.62 -> 89.00
 55%  sccache requests_not_cacheable linux64 debug taskcluster-c4.4xlarge   58.62 -> 91.00
 55%  sccache requests_not_cacheable linux64-noopt debug taskcluster-m4.4xlarge58.62 -> 91.00
 54%  sccache requests_not_cacheable linux32 debug rusttests taskcluster-c5.4xlarge56.50 -> 87.00


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=15399

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Ionuț Goldan [:igoldan]]

The precise range of the patch that caused this regression initially is: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=3c3eda8b2ee5d4349d36fdb698b0367a5c6e1741&tochange=2b045052d4aad4e77f3b8fce35a40efeed4e19f1

Comment 2

[Mike Hommey [:glandium]]

That's from bug 1486654, which enabled the clang-plugin. Doing that enables the clang-plugin tests, which there are 36 of, and they are not cacheable. There might be a few more configure-time uncacheable tests too. All in all, it seems to account for the regression.