1411884 - 6.23% build times (windows2012-32) regression on push 64a852e835b7469715fe13dc76b12458e12bb5cb (Wed Oct 25 2017)

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=64a852e835b7469715fe13dc76b12458e12bb5cb

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  6%  build times summary windows2012-32 debug static-analysis taskcluster-c4.4xlarge     1,853.70 -> 1,969.28

Improvements:

 66%  sccache hit rate summary windows2012-32 debug static-analysis taskcluster-c4.4xlarge     0.78 -> 0.26
 66%  sccache hit rate summary windows2012-32 opt static-analysis taskcluster-c4.4xlarge       0.78 -> 0.27
 65%  sccache hit rate summary windows2012-64 opt static-analysis taskcluster-c4.4xlarge       0.86 -> 0.30


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=10168

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Ionuț Goldan [:igoldan]]

The improvements mentioned above are actually regressions. They actually show that the sccache hit rate dropped from 100% to 66%. Fortunately, it was a just temporary issue.

:gps The increase in build time still remains. Is this something we should accept from now on, for debug builds?

Comment 2

[Gregory Szorc [:gps]]

The intent of the patches wasn't to change any build configurations for release builds.

However, this measured a difference for static analysis builds, which aren't release builds.

I need to look at the logs, but we /may/ have accidentally changed build flags for static analysis builds when we didn't intend to. I'm glad this monitoring is here and Perfherder alerted us to this possibility.

I'll look at this in more detail later. I don't think it is terribly important to do right this minute.

Comment 3

[Mike Hommey [:glandium]]

If you look at the graph, the build times are back to what they were.

Comment 4

[Ionuț Goldan [:igoldan]]

(In reply to Mike Hommey [:glandium] from comment #3)
> If you look at the graph, the build times are back to what they were.
Pretty much yes, but not entirely.

The new min-values are just slighly higher (<1%) than the ones before bug 1411081 landed (old mins 1845 vs new mins 1855).
I believe we have to thank build caching for that.

So the 6% regression we noticed initially wasn't that serious really.

Comment 5

[Ionuț Goldan [:igoldan]]

I see bug 1412952 managed to regain the original baseline. Did it resolve our specific regression or bring a separate improvement by its own?

Comment 6

[Gregory Szorc [:gps]]

I think we can call this resolved.