1315715 - 2.48 - 7.4% cart / sessionrestore / sessionrestore_no_auto_restore / tresize (windows8-64) regression on push b4ade2b0841c (Fri Nov 4 2016)

Status: RESOLVED WONTFIX

(Core :: Graphics, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push b4ade2b0841c . As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  7%  tresize windows8-64 opt e10s                            10.64 -> 11.43
  7%  cart summary windows8-64 opt e10s                       31.34 -> 33.47
  3%  sessionrestore windows8-64 opt e10s                     753.83 -> 773.08
  2%  sessionrestore_no_auto_restore windows8-64 opt e10s     790 -> 809.58

Improvements:

  5%  tsvgx summary windows8-64 opt e10s     124.33 -> 117.58
  2%  damp summary windows8-64 opt e10s      274.98 -> 268.66


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=4005

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I did a lot of retriggers here and have data for this specific push vs the previous one:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=c04f84afb1bd&newProject=mozilla-inbound&newRevision=b4ade2b0841c&framework=1

:dvander, I see you are the patch author.  Can you confirm this is expected on windows 8 vs windows 7 and if there is anything we can do to address these regressions?

Comment 2

[David Anderson [:dvander] - inactive, e-mail if emergency]

Yes, some kind of startup regression was potentially anticipated for Windows 8 versus Windows 7, since this feature only affects Windows 8+ on TreeHerder. I'll look to see if there is anything we can do.

Comment 3

[Alison Shiue]

As a note, this regression patch has been merged into Aurora, so please make sure to uplift any fixes to Aurora, thank you.

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=4129

Comment 4

[David Anderson [:dvander] - inactive, e-mail if emergency]

This patch is Nightly-only, even on Aurora. There should be no regression there.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

the list of alerts on the original regression is fairly long here:
https://treeherder.mozilla.org/perf.html#/alerts?id=4005

looking at aurora, I do not see the cart/damp/tresize regressions, but the sessionrestore regressions are either related or we missed a bunch of micro regressions, here are two examples:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=7776000&series=%5Bmozilla-inbound,5f783f35de9134018990e631fb6ff725b2f3a9da,1,1%5D&series=%5Bmozilla-aurora,555ac79a588637a3ec5752d5b9b3ee769a55d7f6,1,1%5D

https://treeherder.mozilla.org/perf.html#/graphs?timerange=7776000&series=%5Bmozilla-inbound,cc4be2ad4c55588a87acfe014c9c55cdde0528f6,1,1%5D&series=%5Bmozilla-aurora,c3f0064e247fc3825e3a4b5367a4d898f86cfc1f,1,1%5D

just looking at the data from the last merge (Sept 19th), it seems that this specific patch is to be the cause.


looking at the improvements, I focus on tsvgx and see:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=7776000&series=%5Bmozilla-inbound,2a94379c1bce14b854c6fddec17d4e319b2f2199,0,1%5D&series=%5Bmozilla-aurora,801468cb00bf0ca29ad9135a05a3bcfcdba8d480,1,1%5D&series=%5Bmozilla-inbound,801468cb00bf0ca29ad9135a05a3bcfcdba8d480,1,1%5D

this indicates that we are not seeing the improvement- this leans a bit more towards the micro regressions adding up, although I would wonder if somehow this feature as nightly only has an impact on sessionrestore accidentally.

Comment 6

[David Anderson [:dvander] - inactive, e-mail if emergency]

It's unlikely. Any improvements are certainly noise since the pref affects nothing performance-related. We were ready to accept small regressions in startup times though, since with the pref on we're launching an additional process and waiting for it.

I'm still investigating, but that seems like the most likely cause so far.

Comment 7

[David Anderson [:dvander] - inactive, e-mail if emergency]

Okay, I have some results on tart/cart. I can reproduce a 3-4% regression (with a standard deviation of <1%, so that seems real). It goes away if I disable the GPU process, or *force Direct2D on in the UI process*. Given that this is measuring complex stuff in the UI, Skia is most likely the culprit.

Milan - how much of a priority is it to investigate that further?

Note: I'm still looking at sessionrestore/tresize to see if there is an obvious cause. It doesn't seem to be Skia/D2D (yet).

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

We will take the regression that comes from turning acceleration off in the UI (tart/cart), as that gives us "graphics drivers don't crash the browser" benefit.  With Quantum, we may get that acceleration back, but that's longer term - for now, we should take this slowdown.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

it sounds like we will be marking this as resolved/wontfix?  if so, please go ahead and do it!