1449108 - 5.23 - 5.54% build times (windows2012-32, windows2012-64) regression on push 3619a41def167002da5ed93d69833193c0f2edb3 (Tue Mar 27 2018)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=3619a41def167002da5ed93d69833193c0f2edb3

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  6%  build times windows2012-32 debug static-analysis taskcluster-c4.4xlarge     1,995.66 -> 2,106.21
  5%  build times windows2012-64 debug static-analysis taskcluster-c4.4xlarge     2,260.76 -> 2,379.10


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=12378

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Ionuț Goldan [:igoldan]]

:froydnj Looks like bug 1448978 increased our build times on Windows, at least for the debug ones.
Is this something we should accept? Or can we do something to reduce these durations?

Comment 2

[Nathan Froyd [:froydnj]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> :froydnj Looks like bug 1448978 increased our build times on Windows, at
> least for the debug ones.
> Is this something we should accept? Or can we do something to reduce these
> durations?

We changed optimization settings, so it makes sense that we saw changes in build times.  I think we get to accept this.  (I'm not completely sure we need to care about build times for debug static analysis anyway?)

Comment 3

[Ionuț Goldan [:igoldan]]

(In reply to Nathan Froyd [:froydnj] from comment #2)
> (I'm not completely sure we
> need to care about build times for debug static analysis anyway?)

Should we consider disabling these perf alerts for debug static analysis or maybe set a new threshold?

Comment 4

[Nathan Froyd [:froydnj]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #3)
> (In reply to Nathan Froyd [:froydnj] from comment #2)
> > (I'm not completely sure we
> > need to care about build times for debug static analysis anyway?)
> 
> Should we consider disabling these perf alerts for debug static analysis or
> maybe set a new threshold?

Yes.  Didn't we recently disable perf alerts for other static analysis and/or debug builds?  Maybe we missed some?

Comment 5

[Ionuț Goldan [:igoldan]]

I think we resolved a bug of this kind, though I have to check the tracking bugs to be sure.
:jmaher Do you remember something about this?

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

I don't remember disabling alerts specifically- we make a lot of changes and I could be forgetting about it.  

looking in the source, it looks like we avoid debug builds:
https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/building/buildbase.py#1690

possibly consider tweaking those options.