1369727 - 3.08 - 11.34% build times / installer size (linux32, linux64, linux64-stylo) regression on push 8fdb9e30b6a77b46cf81232bcff642d939f7aa51 (Fri Jun 2 2017)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=8fdb9e30b6a77b46cf81232bcff642d939f7aa51

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 11%  installer size summary linux64 opt valgrind     453,250,141.00 -> 504,668,535.08
 10%  build times summary linux64 opt taskcluster-c4.4xlarge valgrind1,410.04 -> 1,547.85
  8%  installer size summary linux32 opt              55,282,258.00 -> 59,550,068.83
  7%  installer size summary linux64 opt              54,517,313.17 -> 58,558,963.00
  7%  installer size summary linux64-stylo opt        56,320,973.00 -> 60,381,583.75
  3%  installer size summary linux64 opt static-analysis53,898,879.92 -> 55,557,246.58

Improvements:

 56%  compiler warnings summary linux32 opt      1,081.83 -> 475.00
 55%  compiler warnings summary linux64 opt valgrind1,094.17 -> 491.00
 55%  compiler warnings summary linux64-stylo opt 1,094.00 -> 491.00
 55%  compiler warnings summary linux64 opt      1,094.08 -> 491.00


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=7001

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Nathan Froyd [:froydnj]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #0)
> We have detected a build metrics regression from push:
> 
> https://hg.mozilla.org/integration/autoland/
> pushloghtml?changeset=8fdb9e30b6a77b46cf81232bcff642d939f7aa51
> 
> As author of one of the patches included in that push, we need your help to
> address this regression.

There's only one patch in that push, and none of the people CC'd are the author of said patch?  CC'ing glandium as the author, though.

Regardless, this is an expected regression; we changed the optimization settings for Linux to improve performance, and doing this comes at some cost to the installer size.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

adding :sylvestre as he is interested in installer size increases.

Comment 3

[Mike Hommey [:glandium]]

Yes, this was expected from bug 1368649. Note those installer sizes are of opt builds, which is not what we ship. We ship the pgo builds, which are unchanged.

Comment 4

[Ionuț Goldan [:igoldan]]

Thanks for your feedback. Sorry for not properly filling the CC list.
I'm marking this as WONTFIX.

Comment 5

[Sylvestre Ledru [:Sylvestre]]

guys, maybe we should just ignore opt builds from this monitoring to only care about what we ship. don't you think?

Comment 6

[Ionuț Goldan [:igoldan]]

I agree with this for the opt installer_size alerts. For the compiler warnings, if you don't see any kind of help in them, then it's your call to deactivate those too on opt builds.

But shouldn't we keep the build_times, as they are important for overwatching the infrastructure?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

I am fine ignoring opt for installer size and only tracking pgo.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

I filed bug 1370599 to figure this out.