1339138 - 2.02 - 3.68% tabpaint / tps (windows7-32, windows8-64) regression on push 910ceca7e8637c4c3ea963fd054e6c449eeff6a7 (Sat Feb 11 2017)

Status: RESOLVED WONTFIX

(Firefox :: Untriaged, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push 910ceca7e8637c4c3ea963fd054e6c449eeff6a7. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  4%  tabpaint summary windows7-32 opt      91.04 -> 94.39
  3%  tabpaint summary windows8-64 opt      86.46 -> 88.79
  2%  tps summary windows7-32 opt           66.89 -> 68.24

Improvements:

 19%  tresize windows8-64 pgo                    12.39 -> 10.02
 12%  tresize linux64 pgo                        20 -> 17.57
 12%  tresize windows7-32 pgo                    10.52 -> 9.28
 10%  tabpaint summary windows8-64 opt e10s      62.76 -> 56.49
  8%  tresize linux64 opt                        23.07 -> 21.23
  5%  tsvg_static summary windows8-64 opt        73.09 -> 69.3
  5%  tsvgr_opacity summary osx-10-10 opt        426.72 -> 405.16
  4%  tsvgr_opacity summary windows8-64 opt      378.77 -> 362.55
  3%  sessionrestore linux64 pgo e10s            642.33 -> 626.25


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=5104

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

good news, we have a lot of wins, bad news a couple regressions.  What is interesting is the tabpaint win8 regression is non-e10s, but the win8 e10s tabpaint had a nice win.

:smaug, is there anything concerning here- reading the original bug, it looks like there were a few concerns about perf.  If we like the wins we get vs the small regressions, that seems reasonable- just making sure there is nothing simple to do in order to reduce or remove the regressions would be nice to know.

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

It is very much expected that timings change, when we start to run refresh driver at different time
(usually earlier than before). There is no way to keep numbers unchanged.

And if non-e10s regress and e10s improves, that is the way to go.