1678608 - 2.06 - 11.42% tp5o / tp5o_webext (linux64-shippable, linux64-shippable-qr, windows10-64-shippable-qr) regression on push 87f72a8ec898093c54d7920a2dfa5b85540b7a1d (Tue November 17 2020)

Status: RESOLVED WONTFIX

(Core :: Security: PSM, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a talos performance regression from push 87f72a8ec898093c54d7920a2dfa5b85540b7a1d. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
11%	tp5o		linux64-shippable-qr	e10s stylo webrender	196.90 -> 219.38
10%	tp5o		linux64-shippable-qr	e10s stylo webrender-sw	193.84 -> 213.55
7%	tp5o		linux64-shippable	e10s stylo	203.13 -> 218.09
7%	tp5o		windows10-64-shippable-qr	e10s stylo webrender-sw	260.16 -> 277.61
5%	tp5o_webext		linux64-shippable-qr	e10s stylo webrender-sw	273.64 -> 286.24
4%	tp5o_webext		linux64-shippable	e10s stylo	277.29 -> 287.98
2%	tp5o		windows10-64-shippable-qr	e10s stylo webrender-sw	265.71 -> 271.19

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Dana Keeler (she/her) [:keeler]]

https://hg.mozilla.org/integration/autoland/rev/87f72a8ec898093c54d7920a2dfa5b85540b7a1d removed a nested event loop. For stability and safety reasons, I think it's an important change, even if it caused a performance regression. It may be that the other changes (particularly https://hg.mozilla.org/integration/autoland/rev/dfc54b523a0b7fe7221c1b78abd4ca949cb086a9) can be modified to not cause such a performance drop - I'll investigate that when I can (although, as a reminder, everyone in the US is off all of next week, and this includes me). Overall, though, my sense is we may just have to accept this.

Comment 2

[Florin Strugariu [:Bebe]]

== Change summary for alert #27818 (as of Mon, 23 Nov 2020 09:34:52 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
11%	raptor-tp6-wikipedia-firefox-cold	fcp	windows10-64-shippable		1,594.50 -> 1,417.92
9%	raptor-tp6-wikipedia-firefox-cold		windows10-64-shippable-qr	webrender	1,825.28 -> 1,658.24
9%	raptor-tp6-wikipedia-firefox-cold	fcp	windows10-64-shippable-qr	webrender	1,788.67 -> 1,625.42
9%	raptor-tp6-wikipedia-firefox-cold	fcp	windows10-64-shippable-qr	webrender	1,789.08 -> 1,628.42
8%	raptor-tp6-wikipedia-firefox-cold		windows10-64-shippable		1,771.97 -> 1,624.06

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27818

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1634065

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:keeler is there anything else to do here or should we close this as WONTFIX and accept the new baseline? You mention potential modifications in comment 1. If you think there's something we could followup with perhaps you could file a new bug to track?

Comment 5

[Dana Keeler (she/her) [:keeler]]

If I'm reading comment 2 correctly, there were also some improvements on Windows, so I'm inclined to wontfix this. The improvements I had in mind were basically work we're planning on doing anyway.