1428018 - 129.87% build times (osx-cross) regression on push 62b9b376360f19408917ee1fa3a3b1e1877b4183 (Sat Dec 30 2017)

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=62b9b376360f19408917ee1fa3a3b1e1877b4183

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

130%  build times osx-cross opt taskcluster-m4.4xlarge     912.60 -> 2,097.80


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=11056

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Ionuț Goldan [:igoldan]]

:glandium If you look over the osx-cross build times for the last 90 days[1], you can see that we gained some improvements since 14 to 15th of November.
Unfortunately, some of the changes made in these patches[2] seem to have canceled that.
Can you review the changes from bug 1427155, bug 1427145, bug 1427232 and bug 1410148 to see if we can reduce the build times again?

[1]:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=7776000&series=autoland,1528358,1,2
[2]:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=a3ed5bb21ec17ea841999c341c4ebf0e6a51efcc&tochange=62b9b376360f19408917ee1fa3a3b1e1877b4183
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=62b9b376360f19408917ee1fa3a3b1e1877b4183&tochange=5fa379ef34e98a308d09c00a36a00ef50db108bc

Comment 2

[Ionuț Goldan [:igoldan]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> :glandium If you look over the osx-cross build times for the last 90
> days[1], you can see that we gained some improvements since 14 to 15th of
> November.
To avoid any confusion: the reduced times happened between November 14th and December 30th.

Comment 3

[Mike Hommey [:glandium]]

Bug 1410148 seems plausible, with https://hg.mozilla.org/integration/mozilla-inbound/rev/e963baf002da landing on nov 14. Specifically, that lowered debug info level, and the most recent landing in that bug re-elevated it. IOW, debug info was degraded between nov 14 and dec 30. The build times improvements were a consequence of that degradation, but that was not a good thing.