Closed Bug 1503179 Opened 6 years ago Closed 6 years ago

0.2 - 12.16% build times / installer size (osx-cross, windows2012-32, windows2012-64) regression on push 5c420d0182a3913e5ed8ab6c7cc55ee73850d14a (Mon Oct 29 2018)

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox-esr60 unaffected, firefox63 unaffected, firefox64 unaffected, firefox65- wontfix)

RESOLVED WONTFIX
Tracking Status
firefox-esr60 --- unaffected
firefox63 --- unaffected
firefox64 --- unaffected
firefox65 - wontfix

People

(Reporter: igoldan, Unassigned)

References

Details

(Keywords: regression)

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=5c420d0182a3913e5ed8ab6c7cc55ee73850d14a

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  9%  build times windows2012-32 opt rusttests taskcluster-c5.4xlarge     1,737.82 -> 1,894.50
  0%  installer size osx-cross opt                                        73,568,143.83 -> 73,712,719.25


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=17204

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics
Product: Testing → Firefox Build System
Flags: needinfo?(cmanchester)
I will look into this. I'm not too worried about the rusttests builds, they're only run in our automation.
Flags: needinfo?(cmanchester)
Assignee: nobody → cmanchester
Bug 1500263 also brought some big AWSY improvements:

== Change summary for alert #17218 (as of Mon, 29 Oct 2018 15:28:46 GMT) ==

Improvements:

 10%  Base Content Explicit windows10-64 opt stylo        12,478,936.62 -> 11,288,576.00
  9%  Base Content Explicit windows10-64-qr opt stylo     12,658,176.00 -> 11,458,048.00
  9%  Base Content Explicit windows10-64 pgo stylo        12,522,837.33 -> 11,393,877.33
  6%  Explicit Memory windows10-64-qr opt stylo           368,955,751.58 -> 347,521,352.31

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=17218
I'm not too worried about the rusttests builds, there have been some recent improvements there as well so I think we'll end up around where we started.

The binary size regression is real and persistent based on graphs, but I'm not really sure how to judge its severity. Of course we should investigate this, but I don't know if we should consider backing out.
If I add the libxul .text section size to the graph you can see that we added ~100k of code to libxul with this change:
https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1514740,1,2&series=autoland,1700442,1,2

That still doesn't account for the full installer size regression which seems more like 150k. It's possible that this is a Nightly-only issue though--we don't fully strip binaries on Nightly (because of --enable-profiling) so the Rust update may simply have caused us to add more symbol data to our binaries. It might be informative to push the before/after changesets to try with a patch to --disable-profiling for both of them to get data to compare without debug symbols conflating the issue.
OK. If someone wants to dig into the ~100k codesize increase that'd be reasonable, but I think that's out of scope for the build team.
I've done as much investigation here as I can right now.
Assignee: cmanchester → nobody
I guess we should close this as WONTFIX.
Per discussion.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.