1683893 - 21.99 - 52.38% startup_about_home_paint_realworld_webextensions (linux64-shippable, windows10-64-shippable, windows10-64-shippable-qr) regression on push ecf2ab52eb03e986eacbb347ff2b67393eaab7e4 (Fri December 18 2020)

Status: RESOLVED WONTFIX

(Testing :: Talos, defect, P5)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a talos performance regression from push ecf2ab52eb03e986eacbb347ff2b67393eaab7e4. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
52%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	windows10-64-shippable	e10s stylo	486.50 -> 741.33
43%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	linux64-shippable	e10s stylo	511.58 -> 733.75
23%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	windows10-64-shippable-qr	e10s stylo webrender	610.83 -> 751.67
22%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	windows10-64-shippable-qr	e10s stylo webrender	609.46 -> 743.50

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1579683

Comment 2

[Florin Strugariu [:Bebe]]

@alex can you move this to the correct component?

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

It is in the correct component. See the culprit one's bug 1579683.

Comment 4

[Mike Conley (:mconley) (:⚙️)]

This is expected from bug 1579683 - that patch disabled a performance optimization in order to allow the test to run more reliably.

The patch to fix the flaky nature of the startup tests with the about:home cache is in bug 1674827. I'll aim to get this re-enabled for startup_about_home_paint_realworld_webextensions there as well.

Comment 5

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Any news here?

Comment 6

[Alex Finder]

Hi Alex! Looks like the alert for this defect is also not accessible anymore.