1524482 - 4.58 - 5.68% raptor-assorted-dom-firefox (linux64, linux64-qr) regression on push74d7d41da9e43c2fb0524c3af354978ff322b417 (Tue Jan 29 2019)

Status: RESOLVED INVALID

(Core :: JavaScript Engine: JIT, defect)

Description

[Ionuț Goldan [:igoldan]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c9a9ccd1c619068b426546f5abd95ca4b39a9567&tochange=74d7d41da9e43c2fb0524c3af354978ff322b417

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

6% raptor-assorted-dom-firefox linux64 opt 24.67 -> 26.08
5% raptor-assorted-dom-firefox linux64 pgo 24.96 -> 26.20
5% raptor-assorted-dom-firefox linux64-qr opt 24.71 -> 25.84

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=19027

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

These are the Gecko profiles for raptor-assorted-dom-firefox on Linux 64bit (PGO builds):

before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FPgjhpEEUReex95QxPv9RNQ%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_raptor-assorted-dom-firefox.zip

after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FLhEi_TqpTumZA9NXOJ9rbA%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_raptor-assorted-dom-firefox.zip

Comment 2

[Jan de Mooij [:jandem]]

Note that we've seen a lot of assorted-dom "regressions" and "improvements" lately so there might be something bimodal going on.

Comment 3

[Ionuț Goldan [:igoldan]]

(In reply to Jan de Mooij [:jandem] from comment #2)

Note that we've seen a lot of assorted-dom "regressions" and "improvements" lately so there might be something bimodal going on.

Looking again over the graphs, it seems the baseline is returning to normal by itself. I'll leave this bug open until the end of day. Likely I'll close this as INVALID. Thanks for pointing this out, Jan!

Comment 4

[Jan de Mooij [:jandem]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #3)

Thanks for pointing this out, Jan!

No problem! Thank you for filing these bugs :)

Comment 5

[André Bargull [:anba]]

Even if the performance regression manifests, we either need to accept it or alternatively disable Spectre mitigations for C++ calls as originally proposed in bug 1438886 comment #15. (The bug which triggered this performance regression, bug 1523558, only fixed a wrong check for the Spectre mitigation added in bug 1438886.)

So it's more or less a case where, even though I may have caused a performance regression, I can't be blamed this time! :-D

Comment 6

[Ionuț Goldan [:igoldan]]

I confirm Jan's comment 2. All noted regressions here have returned to previous baselines.
Perfherder automatically detected one of the improvements.

== Change summary for alert #19104 (as of Thu, 31 Jan 2019 11:59:53 GMT) ==

Improvements:

5% raptor-assorted-dom-firefox linux64 opt 26.06 -> 24.73

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=19104