1489501 - 11.44 - 73.63% sessionrestore / tsvg_static / tsvgr_opacity (linux64, linux64-qr, osx-10-10, windows10-64, windows10-64-qr) regression on push 6fefaf842b05bea156591d27cbd64095617dbf27 (Wed Sep 5 2018)

Status: RESOLVED WONTFIX

(Core :: Layout, defect, P3)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=6fefaf842b05bea156591d27cbd64095617dbf27

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 74%  sessionrestore linux64-qr opt e10s stylo          449.08 -> 779.75
 49%  tsvgr_opacity linux64-qr opt e10s stylo           108.17 -> 161.65
 48%  tsvgr_opacity linux64 pgo e10s stylo              106.63 -> 157.33
 46%  sessionrestore windows10-64-qr opt e10s stylo     279.58 -> 407.00
 44%  tsvgr_opacity windows10-64-qr opt e10s stylo      115.56 -> 165.92
 41%  tsvgr_opacity linux64 opt e10s stylo              115.02 -> 161.91
 40%  tsvgr_opacity osx-10-10 opt e10s stylo            162.84 -> 228.48
 19%  tsvg_static windows10-64-qr opt e10s stylo        77.29 -> 92.05
 11%  tsvgr_opacity windows10-64 opt e10s stylo         104.93 -> 116.94

Improvements:

  6%  tresize linux64-qr opt e10s stylo     13.27 -> 12.46
  6%  tresize linux64-qr opt e10s stylo     13.25 -> 12.51
  4%  tscrollx linux64 pgo e10s stylo       0.70 -> 0.67
  3%  tscrollx linux64 opt e10s stylo       0.68 -> 0.66


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=15696

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:mattwoodrow, there are a few improvements and many large regressions here- can you take a look at fixing these regressions?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

here are some geckoprofiles to look at-

before:
* sessionrestore linux64-qr: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FGYbZGtleS9aU2pvVVUEA6A%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip
* svgr_opacity win10-qr: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLHLS9MElSz-wWq59T1fJVQ%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tsvgr_opacity.zip

after:
* sessionrestore linux64-qr: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2Fc7Gqd1zFROGvIDoQMfCPOQ%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip
* svgr_opacity win10-qr: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FGBAfBCbTRTiNp0Cl21HbYw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tsvgr_opacity.zip

Comment 3

[Matt Woodrow (:mattwoodrow)]

Unfortunately these regressions are because we're changing what's captured within the test, and we're not actually slower.

Previously we were firing MozAfterPaint (and ending the test) on a fixed fallback timeout, and not waiting for the paint to actually finish. The regressing change here got rid of the fallback, so the test now more accurately reflects how long painting takes, but the results are obviously worse.

Comment 4

[Sean Voisen (:svoisen)]

(In reply to Matt Woodrow (:mattwoodrow) from comment #3)

> Unfortunately these regressions are because we're changing what's captured
> within the test, and we're not actually slower.

Any reason to keep this open then?

Comment 5

[Matt Woodrow (:mattwoodrow)]

None at all!