1602443 - perf_reftest_singletons (macosx1014-64-shippable) regression on push 724d7b9360785fbb558b67cacf88b71e35ce9e37 (Mon December 2 2019)

Status: RESOLVED INVALID

(Core :: DOM: Core & HTML, defect)

Description

[Florin Strugariu [:Bebe]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=724d7b9360785fbb558b67cacf88b71e35ce9e37

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

46% perf_reftest_singletons parent-basic-singleton.html macosx1014-64-shippable opt e10s stylo 198.65 -> 289.36
30% perf_reftest_singletons bloom-basic-2.html macosx1014-64-shippable opt e10s stylo 77.93 -> 101.48
29% perf_reftest_singletons bloom-basic.html macosx1014-64-shippable opt e10s stylo 80.54 -> 104.12
28% tp5o_webext macosx1014-64-shippable opt e10s stylo 545.60 -> 699.50
28% tsvg_static macosx1014-64-shippable opt e10s stylo 45.71 -> 58.37
27% tart macosx1014-64-shippable opt e10s stylo 6.11 -> 7.76
27% perf_reftest_singletons style-sharing-style-attr.html macosx1014-64-shippable opt e10s stylo 6.33 -> 8.04
26% a11yr macosx1014-64-shippable opt e10s stylo 211.01 -> 264.86
25% perf_reftest_singletons style-sharing.html macosx1014-64-shippable opt e10s stylo 6.39 -> 8.00
25% tp5o macosx1014-64-shippable opt e10s stylo 360.88 -> 450.27
24% tsvgx macosx1014-64-shippable opt e10s stylo 185.06 -> 229.39
21% tp5o_scroll macosx1014-64-shippable opt e10s stylo 2.98 -> 3.61
21% perf_reftest_singletons coalesce-2.html macosx1014-64-shippable opt e10s stylo 222.97 -> 269.36
20% perf_reftest_singletons coalesce-1.html macosx1014-64-shippable opt e10s stylo 272.92 -> 327.58
20% perf_reftest_singletons tiny-traversal-singleton.html macosx1014-64-shippable opt e10s stylo 1,206.70 -> 1,443.38
19% displaylist_mutate macosx1014-64-shippable opt e10s stylo 2,665.73 -> 3,179.54
19% kraken macosx1014-64-shippable opt e10s stylo 1,107.40 -> 1,319.32
19% tsvgr_opacity macosx1014-64-shippable opt e10s stylo 156.16 -> 185.93
19% perf_reftest_singletons inline-style-cache-1.html macosx1014-64-shippable opt e10s stylo 2,256.37 -> 2,684.71
18% perf_reftest_singletons link-style-cache-1.html macosx1014-64-shippable opt e10s stylo 1,482.94 -> 1,755.54
18% perf_reftest_singletons window-named-property-get.html macosx1014-64-shippable opt e10s stylo 737.42 -> 868.74
18% perf_reftest_singletons scrollbar-styles-1.html macosx1014-64-shippable opt e10s stylo 634.61 -> 745.88
17% perf_reftest_singletons getElementById-1.html macosx1014-64-shippable opt e10s stylo 57.27 -> 67.28
17% perf_reftest_singletons id-getter-5.html macosx1014-64-shippable opt e10s stylo 540.86 -> 634.73
17% tabpaint macosx1014-64-shippable opt e10s stylo 62.27 -> 73.06
17% perf_reftest_singletons id-getter-4.html macosx1014-64-shippable opt e10s stylo 541.56 -> 634.03
17% perf_reftest_singletons id-getter-6.html macosx1014-64-shippable opt e10s stylo 541.94 -> 632.86
17% perf_reftest_singletons id-getter-7.html macosx1014-64-shippable opt e10s stylo 539.60 -> 630.02
16% perf_reftest_singletons style-attr-1.html macosx1014-64-shippable opt e10s stylo 4.97 -> 5.78
16% perf_reftest_singletons slow-selector-2.html macosx1014-64-shippable opt e10s stylo 0.12 -> 0.14
16% perf_reftest_singletons id-getter-3.html macosx1014-64-shippable opt e10s stylo 543.70 -> 632.23
16% perf_reftest_singletons abspos-reflow-1.html macosx1014-64-shippable opt e10s stylo 83.31 -> 96.69
16% about_preferences_basic macosx1014-64-shippable opt e10s stylo 150.63 -> 174.68
16% perf_reftest_singletons external-string-pass.html macosx1014-64-shippable opt e10s stylo 1,010.76 -> 1,171.48
16% perf_reftest_singletons id-getter-1.html macosx1014-64-shippable opt e10s stylo 547.01 -> 633.37
16% perf_reftest_singletons id-getter-2.html macosx1014-64-shippable opt e10s stylo 982.43 -> 1,135.66
16% perf_reftest_singletons bidi-resolution-1.html macosx1014-64-shippable opt e10s stylo 188.10 -> 217.26
15% perf_reftest_singletons only-children-1.html macosx1014-64-shippable opt e10s stylo 0.12 -> 0.14
15% perf_reftest style-attr-1.html macosx1014-64-shippable opt e10s stylo 4.93 -> 5.66
15% tscrollx macosx1014-64-shippable opt e10s stylo 1.35 -> 1.55
15% perf_reftest_singletons slow-selector-1.html macosx1014-64-shippable opt e10s stylo 0.12 -> 0.14
12% dromaeo_css macosx1014-64-shippable opt e10s stylo 14,354.75 -> 12,692.66

Improvements:

13% ts_paint macosx1014-64-shippable opt e10s stylo 1,308.79 -> 1,136.00
12% ts_paint_webext macosx1014-64-shippable opt e10s stylo 1,304.04 -> 1,152.08

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24183

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/TestEngineering/Performance/Talos

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Andrew McCreight [:mccr8]]

That's pretty strange given that literally all of the code changes in this patch are behind #ifdef DEBUG as far as I can tell.

Comment 2

[Andrew McCreight [:mccr8]]

Presumably detectShutdownLeaks is only set in debug builds, but that seems more likely to be a cause than the C++ changes.

Comment 3

[Andrew McCreight [:mccr8]]

Though I don't know why perf_reftest_singletons would use mochitest/runtests.py

Comment 4

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Correct, it only runs on debug builds.....

Comment 5

[Marian Raiciof [:marauder]]

We had some issues with retriggers and backfills because of the coalescing settings.
Now that it was fixed/disabled, i was able to do some retriggers and now this alert seems invalid.
Old data points are at the top and all new data points are displayed at the bottom.
It looks like a spike / infra-change.

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&selected=2056468,603351,451.4424584031844,65.95041528854017,988365525&series=autoland,2056468,1,1&series=mozilla-inbound,2131876,1,1&timerange=1209600&zoom=1575260995976,1575363909022,67.6531748545376,109.87862137874554

Comment 6

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Yay!