1476865 - 4.38 - 70.47% sessionrestore / tpaint / tresize / ts_paint / ts_paint_heavy / ts_paint_webext (linux64-qr, windows10-64-qr) regression on push 83724fe36744 (Tue Jul 17 2018)

Status: RESOLVED WONTFIX

(Core :: Graphics: WebRender, defect, P3)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=82da16c50e25f9f7e63d235e2827cf9671a8e7f4&tochange=83724fe36744d82ffcfe7e501183bc1101b00630

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 70%  ts_paint windows10-64-qr opt e10s stylo     340.58 -> 580.58
 70%  ts_paint_webext windows10-64-qr opt e10s stylo345.17 -> 588.25
 70%  ts_paint_heavy windows10-64-qr opt e10s stylo342.83 -> 583.50
 62%  tpaint windows10-64-qr opt e10s stylo       229.45 -> 371.76
 23%  ts_paint linux64-qr opt e10s stylo          223.33 -> 275.42
 23%  ts_paint_webext linux64-qr opt e10s stylo   225.75 -> 278.00
 23%  ts_paint_heavy linux64-qr opt e10s stylo    223.83 -> 274.75
 12%  sessionrestore linux64-qr opt e10s stylo    343.75 -> 385.83
  4%  tresize windows10-64-qr opt e10s stylo      8.64 -> 9.02

Improvements:

 23%  tresize linux64-qr opt e10s stylo     17.48 -> 13.41


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=14405

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

And here are the Gecko profiles on Windows 10 Quantum Render:

for tpaint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FTkSzOtG-QKq1kqEgBPLyJg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tpaint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FW28EawitROihl3rkuXSw6A%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tpaint.zip


for tabpaint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FTkSzOtG-QKq1kqEgBPLyJg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tabpaint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FW28EawitROihl3rkuXSw6A%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tabpaint.zip


for ts_paint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FTkSzOtG-QKq1kqEgBPLyJg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_ts_paint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FW28EawitROihl3rkuXSw6A%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_ts_paint.zip


for sessionrestore:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FTkSzOtG-QKq1kqEgBPLyJg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FW28EawitROihl3rkuXSw6A%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip

Comment 2

[Ionuț Goldan [:igoldan]]

Also, here are the Gecko profiles from Linux 64 Quantum Render:


for tpaint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FPsSGXPIKRJeyCvHMwpkOog%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tpaint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLAC7AW0JTKOtruzq0BT7Mw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tpaint.zip


for tabpaint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FPsSGXPIKRJeyCvHMwpkOog%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tabpaint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLAC7AW0JTKOtruzq0BT7Mw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tabpaint.zip


for ts_paint:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FPsSGXPIKRJeyCvHMwpkOog%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_ts_paint.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLAC7AW0JTKOtruzq0BT7Mw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_ts_paint.zip


for sessionrestore:

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FPsSGXPIKRJeyCvHMwpkOog%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLAC7AW0JTKOtruzq0BT7Mw%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sessionrestore.zip

Comment 3

[Kartikaya Gupta (email:[e-mail])]

Yup, this regression was expected, but the change was needed for correctness. We'll find ways to speed up these tests. I suspect bug 1476876 will help a lot.

Comment 4

[Ionuț Goldan [:igoldan]]

Bug 1476876 fixed the regressions from Windows QR and even brought some perf wins there.

== Change summary for alert #14473 (as of Fri, 20 Jul 2018 13:36:33 GMT) ==

Improvements:

 41%  ts_paint_webext windows10-64-qr opt e10s stylo     582.25 -> 342.00
 41%  ts_paint_heavy windows10-64-qr opt e10s stylo      577.92 -> 340.17
 41%  ts_paint windows10-64-qr opt e10s stylo            572.83 -> 340.83
 40%  sessionrestore windows10-64-qr opt e10s stylo      491.25 -> 292.33
 39%  tpaint windows10-64-qr opt e10s stylo              362.94 -> 222.80
 21%  sessionrestore_many_windows windows10-64-qr opt e10s stylo2,918.17 -> 2,319.42

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=14473

Comment 5

[Ionuț Goldan [:igoldan]]

However, the regressions from Linux 64 QR remained.
Also, there's one last regressions on Windows 10 QR, namely 4% tresize windows10-64-qr opt e10s stylo: it seems that bug 1476876 fixed it for a very short while, but then the improvement disappeared; probably a separate bug canceled it, but I have to check this to properly confirm.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

It looks like tresize did in fact get better from bug 1476876, and then was regressed by the changes in https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=80efe717b563. We can file a separate bug to investigate that, and leave this bug open for the remaining Linux regressions from comment 0.

Comment 7

[Ionuț Goldan [:igoldan]]

Any updates regarding the Linux regressions?

Comment 8

[Kartikaya Gupta (email:[e-mail])]

Not at this point. The Linux regressions are lower priority since we're focusing on Windows for the first release.

Comment 9

[Ionuț Goldan [:igoldan]]

(In reply to Kartikaya Gupta (email:[e-mail]) (parental leave) from comment #6)
> It looks like tresize did in fact get better from bug 1476876, and then was
> regressed by the changes in
> https://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=80efe717b563. We can file a separate bug to
> investigate that, and leave this bug open for the remaining Linux
> regressions from comment 0.

:nical, :jrmuizel was the regression :kats mentioned fixed? I'm asking this because I noticed bug 1477638 brought some big perf improvements and I'm not sure whether they did it on themselves or fixed that regression.

Comment 10

[Nicolas Silva [:nical]]

The regression mentioned is covered by bug 1477638