1497133 - 3.21 - 10.2% build times (windows2012-64) regression on push 556b2f4cd653 (Thu Oct 4 2018)

Status: VERIFIED FIXED

(Firefox Build System :: Toolchains, defect, P3)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=4bada21c38bf68253216b06b7f21e2beaf047f6e&tochange=556b2f4cd653aa7642956ba66e60839b4ea9f474

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 10%  build times windows2012-64 opt plain taskcluster-c5.4xlarge     1,591.35 -> 1,753.66
  7%  build times windows2012-64 opt plain taskcluster-c4.4xlarge     1,897.62 -> 2,036.99
  3%  build times windows2012-64 pgo msvc taskcluster-c4.4xlarge      4,496.93 -> 4,641.27


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=16456

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Chris Manchester (limited bugmail, email directly)]

We're planning to back this out before the next merge, so I'm not too worried, but in the meantime we should notify the rust team about this.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Alex, do we have benchmarks that run against rustc to attempt to detect regressions like this? Between this and https://bugzilla.mozilla.org/show_bug.cgi?id=1490145#c7 this seems like a potentially bad trend for windows build times and rustc upgrades.

Comment 3

[Nathan Froyd [:froydnj]]

njn might also be interested in knowing about slowdowns.

Comment 4

[Alex Crichton [:acrichto]]

We have an extensive perf regression suite suited for compile time at https://perf.rust-lang.org/ but it's unfortunately only measured on Linux so it's possible that Windows-specific regression could sneak in without us knowing it. 

If there's something locally reproducible we can also try to narrow down a build time regression on our end!

Comment 5

[Mike Hommey [:glandium]]

Note that this is only a regression on Windows because the rust version was only bumped to beta on Windows.

Comment 6

[Mike Hommey [:glandium]]

Interestingly, it does seem like it's Windows-specific: upgrading to the same beta for linux builds doesn't cause a regression (in fact, the beta might even be faster)
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=1587dffb77e8&newProject=try&newRevision=1f32a302eef4&framework=2

Comment 7

[Ionuț Goldan [:igoldan]]

Any updates here?

Comment 8

[Chris Manchester (limited bugmail, email directly)]

This is getting backed out tomorrow.