1428311 - 14.75% build times (windows2012-32) regression on push e2d86922b3ef78df0a0774ef1326f4fd979de371 (Thu Jan 4 2018)

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=e2d86922b3ef78df0a0774ef1326f4fd979de371

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 15%  build times windows2012-32 opt rusttests taskcluster-c4.4xlarge     2,369.56 -> 2,719.12


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=11081

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Ionuț Goldan [:igoldan]]

:ted Can you please take a look over this Windows build time regression? It happens on both 32 and 64 bit architectures.

Since January 2nd, build times on this platform got obviously up. I noticed multiple Build Config changes which are related to Rust toolchain updates.

Comment 2

[Ionuț Goldan [:igoldan]]

I need a second pair of eyes over this alerts also [1], as I'm not sure if they are related to this bug or not.

[1] https://treeherder.mozilla.org/perf.html#/alerts?id=11068

Comment 3

[Ionuț Goldan [:igoldan]]

Is it possible we're having infra changes, that aren't visible from in-tree updates?

Comment 4

[(not currently active) Ted Mielczarek]

I thought this was just rusttest builds, but the regression shows up in regular win32 builds as well:
https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1625065,1,2&series=autoland,1460334,1,2

Comment 5

[Ionuț Goldan [:igoldan]]

FYI Since January 12, this regression lowered down half-way. I'm pretty sure this is from infrastructure changes like those we talked about in bug 1429319 and bug 1429311.

Still, it's clear that some configs from our side caused some regressions.

Comment 6

[Ionuț Goldan [:igoldan]]

Since March ~10, build times have improved by *a lot*. Closing this as WORKSFORME.