1369034 - 2.32 - 9.77% build times / installer size (linux32, linux64, linux64-stylo, windows2012-32, windows2012-64, windows8-64, windowsxp) regression on push b7594e98945cb54f34d1afd0f5eac784a06ed8ea (Tue May 9 2017)

Status: RESOLVED FIXED

(Firefox :: Untriaged, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

We have detected a build metrics regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=b7594e98945cb54f34d1afd0f5eac784a06ed8ea

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 10%  installer size summary windows2012-32 pgo      52,036,370.58 -> 57,121,429.92
 10%  installer size summary windows2012-32 opt      50,682,572.17 -> 55,591,109.42
  9%  installer size summary windowsxp opt           50,744,827.17 -> 55,241,819.67
  9%  installer size summary windows2012-64 pgo      56,119,506.83 -> 61,063,949.33
  9%  installer size summary windows2012-64 opt      55,785,571.25 -> 60,676,352.25
  8%  installer size summary linux32 opt             58,902,534.75 -> 63,727,885.17
  8%  installer size summary linux64-stylo opt       59,739,827.50 -> 64,572,459.75
  8%  installer size summary windows8-64 opt         55,842,945.00 -> 60,327,427.67
  8%  installer size summary linux32 pgo             64,077,236.92 -> 68,895,432.50
  7%  installer size summary linux64 pgo             63,745,143.83 -> 68,506,836.75
  7%  installer size summary linux64 debug           70,150,552.17 -> 75,005,042.25
  3%  build times summary windows2012-64 pgo taskcluster-c4.4xlarge4,345.84 -> 4,483.24
  2%  installer size summary linux64 asan            211,222,228.75 -> 216,120,902.17


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=6485

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Automated_Performance_Testing_and_Sheriffing/Build_Metrics

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

this is somewhere in:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=b7594e98945cb54f34d1afd0f5eac784a06ed8ea&group_state=expanded

I am not able to get the list of revisions due to an hg error, but I thought I would file this bug as there is interest in tracking the installer size.

:sylvestre, is there anything different we should do for installer size regressions?

Comment 2

[Sylvestre Ledru [:Sylvestre]]

This is great output, thanks!

Pascal was been following the recent patches in hg, maybe he knows what increased the size of the binary.

Comment 3

[Pascal Chevrel:pascalc]

This regression dates back to almost a month ago (May 9) if I read the report correctly and since then the installer size decreased significantly and is even lower than on May 8:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=%5Bautoland,e8cc44207bb0bad4643184ce5d031b6f48a4c23f,1%5D&selected=%5Bautoland,e8cc44207bb0bad4643184ce5d031b6f48a4c23f%5D

Am I missing something?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

yes, this is from a few weeks old and there are reductions- I am filing bugs so we have historical reference for some of the larger changes.

Comment 5

[Gregory Szorc [:gps]]

Recent large installer size changes are almost certainly related to catlee's changes for omni.ja packaging. I don't have the bug or changeset available off-hand.

Comment 6

[Chris AtLee [:catlee]]

I don't know where the b7594e98945cb54f34d1afd0f5eac784a06ed8ea revision came from.

Looks like this regression was part of this push: https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=e31b45db8b71

That push contained https://hg.mozilla.org/integration/autoland/rev/ec15663c6331, which was a backout of bug 1351071 to see if it was responsible for input event handling regression in bug 1362094.

The patch was relanded as https://hg.mozilla.org/integration/autoland/rev/68cdf62dd865 on May 19th after telemetry showed no improvement.