1415125 - 2.8 - 16.75% Explicit Memory / Heap Unclassified / Images / JS / Resident Memory (all desktop platforms) 59856e397ead regression on push (Mon Nov 6 2017)

Status: RESOLVED WONTFIX

(Remote Protocol :: Marionette, defect)

Description

[Ionuț Goldan [:igoldan]]

We have detected an awsy regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=80a2fea82e8f627f3ed7d5823e6dd6c91c6ac647&tochange=59856e397ead753be46684b9a0498d3214bcdee4

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 17%  Images summary linux32-stylo-disabled opt stylo-disabled     3,933,570.86 -> 4,592,507.05
 17%  Images summary windows7-32-stylo-disabled opt stylo-disabled 4,078,463.98 -> 4,761,480.06
 16%  Images summary osx-10-10 opt                                 5,225,864.96 -> 6,047,665.85
 14%  Images summary linux64 opt                                   4,841,062.27 -> 5,517,025.59
 14%  Images summary macosx64-stylo-disabled opt stylo-disabled    5,147,053.98 -> 5,848,915.85
 13%  Images summary linux64-stylo-sequential opt stylo-sequential 4,927,492.13 -> 5,572,157.14
 12%  Explicit Memory summary linux64-stylo-sequential opt stylo-sequential280,639,606.55 -> 315,091,293.78
 12%  Explicit Memory summary windows10-64 opt                     286,164,411.66 -> 321,053,883.79
 12%  Explicit Memory summary linux64 opt                          283,527,139.71 -> 317,803,268.82
 12%  Explicit Memory summary windows10-64 pgo                     285,858,315.01 -> 320,019,176.94
 12%  Explicit Memory summary windows10-64-stylo-disabled opt stylo-disabled279,757,481.35 -> 313,156,505.56
 12%  Explicit Memory summary linux64-stylo-disabled opt stylo-disabled278,005,645.24 -> 310,250,694.48
 12%  Explicit Memory summary windows7-32 opt                      223,113,714.98 -> 248,883,085.62
 11%  Explicit Memory summary osx-10-10 opt                        295,530,545.11 -> 329,167,909.71
 11%  Explicit Memory summary windows7-32 pgo                      223,834,880.75 -> 248,670,762.70
 11%  Explicit Memory summary macosx64-stylo-disabled opt stylo-disabled289,065,164.08 -> 320,615,166.27
 11%  Explicit Memory summary linux32-stylo-disabled opt stylo-disabled213,250,525.57 -> 235,986,994.68
 10%  Explicit Memory summary windows7-32-stylo-disabled opt stylo-disabled219,067,219.36 -> 241,393,677.18
 10%  JS summary macosx64-stylo-disabled opt stylo-disabled        94,880,759.79 -> 104,236,730.62
  9%  JS summary linux64-stylo-disabled opt stylo-disabled         95,958,662.27 -> 104,484,859.37
  9%  JS summary windows10-64-stylo-disabled opt stylo-disabled    106,689,206.76 -> 116,101,636.25
  9%  JS summary linux64 opt                                       96,160,371.20 -> 104,508,843.06
  9%  Resident Memory summary windows10-64-stylo-disabled opt stylo-disabled449,525,989.12 -> 488,273,649.53
  9%  Resident Memory summary windows7-32 pgo                      289,596,231.99 -> 314,517,839.06
  9%  Resident Memory summary windows10-64 pgo                     450,101,035.76 -> 488,543,988.19
  9%  JS summary osx-10-10 opt                                     95,339,919.08 -> 103,439,584.55
  8%  Resident Memory summary windows10-64 opt                     458,363,855.93 -> 497,054,855.41
  8%  Resident Memory summary linux64 opt                          438,533,883.29 -> 474,718,638.00
  8%  JS summary linux64-stylo-sequential opt stylo-sequential     96,107,794.65 -> 103,959,479.57
  8%  JS summary windows10-64 pgo                                  107,260,738.02 -> 115,835,272.32
  8%  JS summary windows10-64 opt                                  107,455,723.80 -> 115,928,261.11
  8%  Resident Memory summary linux64-stylo-disabled opt stylo-disabled430,084,658.44 -> 463,687,516.40
  7%  Resident Memory summary linux64-stylo-sequential opt stylo-sequential436,657,170.74 -> 468,787,390.44
  7%  JS summary linux32-stylo-disabled opt stylo-disabled         70,683,340.63 -> 75,595,943.76
  7%  Resident Memory summary windows7-32 opt                      297,591,253.23 -> 317,471,960.14
  7%  Resident Memory summary macosx64-stylo-disabled opt stylo-disabled581,011,905.22 -> 619,374,760.16
  7%  JS summary windows7-32 pgo                                   80,981,083.40 -> 86,261,555.05
  6%  Resident Memory summary windows7-32-stylo-disabled opt stylo-disabled291,639,245.27 -> 310,181,089.52
  6%  JS summary windows7-32-stylo-disabled opt stylo-disabled     81,374,882.32 -> 86,486,666.23
  6%  JS summary windows7-32 opt                                   81,487,963.00 -> 86,558,285.74
  6%  Resident Memory summary osx-10-10 opt                        582,792,214.90 -> 618,287,090.42
  6%  Heap Unclassified summary linux64-stylo-sequential opt stylo-sequential53,335,044.79 -> 56,585,725.29
  6%  Heap Unclassified summary linux64 opt                        53,496,245.76 -> 56,707,254.69
  6%  Resident Memory summary linux32-stylo-disabled opt stylo-disabled367,855,120.98 -> 389,334,352.10
  6%  Heap Unclassified summary linux64-stylo-disabled opt stylo-disabled54,100,719.37 -> 57,179,378.97
  5%  Heap Unclassified summary linux32-stylo-disabled opt stylo-disabled41,212,093.03 -> 43,309,104.06
  5%  Heap Unclassified summary windows10-64-stylo-disabled opt stylo-disabled46,492,135.55 -> 48,711,615.01
  5%  Heap Unclassified summary windows10-64 opt                   46,191,336.20 -> 48,335,913.13
  5%  Heap Unclassified summary windows10-64 pgo                   46,270,663.41 -> 48,389,411.93
  4%  Heap Unclassified summary windows7-32 opt                    38,595,841.88 -> 40,064,296.58
  4%  Heap Unclassified summary windows7-32 pgo                    38,762,380.30 -> 40,165,983.85
  3%  Heap Unclassified summary osx-10-10 opt                      67,855,332.99 -> 69,770,716.51
  3%  Heap Unclassified summary macosx64-stylo-disabled opt stylo-disabled68,500,875.56 -> 70,416,012.21


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=10375

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/AWSY

Comment 1

[Ionuț Goldan [:igoldan]]

:whimboo Where these memory regressions expected? Should we look for a fix or just accept these test changes?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Please see bug 1411393 comment 18 and following comments. Given that Marionette forced Firefox to not use bfcache before my patch landed, we expected a memory increase now. Eric, do we have to spend any time in making any more thoughts into that increase?

Comment 3

[Eric Rahm [:erahm]]

Sadly the regression is expected, I think we just need to accept it given this represents how we behave outside of testing automation. I *do* think we need to follow up on working to reduce the regression, but that shouldn't block landing. Olli filed bug 1414505 to look at this further.