1737604 - 11.01 - 9.29% perf_reftest_singletons parent-basic-singleton.html / perf_reftest_singletons parent-basic-singleton.html + 1 more (Linux) regression on Thu October 21 2021

Status: RESOLVED WONTFIX

(Firefox :: Installer, defect)

Description

[Andra Esanu (needinfo me)]

Perfherder has detected a talos performance regression from push 2946940c35df53633bf58b87e5bb2786b6fe5450. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
11%	perf_reftest_singletons parent-basic-singleton.html	linux1804-64-shippable-qr	e10s stylo webrender	80.52 -> 89.39
10%	perf_reftest_singletons parent-basic-singleton.html	linux1804-64-shippable-qr	e10s fission stylo webrender	79.99 -> 88.08
9%	perf_reftest_singletons parent-basic-singleton.html	linux1804-64-shippable-qr	e10s stylo webrender	81.94 -> 89.55

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=2946940c35df53633bf58b87e5bb2786b6fe5450 is a straight reversion of part of Bug 1709969. I don't see any performance improvements noted for Bug 1709969, which is what I expected.

This particular patch has to do with how profiles are chosen at startup time. There's essentially no interaction with the details of this test, which exercises page loading. It seems like this is:

1. not this push
2. a faulty test that is somehow including startup time for the harness in its measurement
3. noise

I don't really have a next step here. We're not likely to revert this patch, because that would reinstate a product experience that our product manager (Romain Testard) doesn't want. But I don't have any insight into how this is interacting.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1736876

Comment 3

[Marco Castelluccio [:marco]]

Andra, could you retriage the regression?

Comment 4

[Andra Esanu (needinfo me)]

I retriaged and the retriggers confirm this culprit. You can look again on the graph.