1776398 - 76.92 - 2.47% twitch FirstVisualChange / reddit LastVisualChange + 5 more (OSX) regression on Mon June 20 2022

Status: RESOLVED WONTFIX

(Core :: XPCOM, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a browsertime performance regression from push cc80efc5b21956dd23d6e58f91d74bdbb392ca5f. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
77%	twitch FirstVisualChange	macosx1015-64-shippable-qr	fission warm webrender	43.33 -> 76.67
67%	twitch FirstVisualChange	macosx1015-64-shippable-qr	fission warm webrender	40.00 -> 66.67
16%	twitch PerceptualSpeedIndex	macosx1015-64-shippable-qr	fission warm webrender	154.62 -> 178.75
10%	amazon LastVisualChange	macosx1015-64-shippable-qr	fission warm webrender	1,263.33 -> 1,383.33
8%	google-slides LastVisualChange	macosx1015-64-shippable-qr	cold fission webrender	3,640.00 -> 3,936.67
6%	twitch SpeedIndex	macosx1015-64-shippable-qr	fission warm webrender	313.00 -> 332.83
2%	reddit LastVisualChange	macosx1015-64-shippable-qr	cold fission webrender	7,351.62 -> 7,533.33

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
2%	ebay loadtime	macosx1015-64-shippable-qr	fission warm webrender	186.40 -> 182.42

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1767396

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The twitch regressions are clear but very small, so I would say it's worth the potential power saving. The amazon and google-slides regressions are less clear that this is the culprit. I've triggered some backfills and retriggers. :alexandrui can you revisit on Monday once we have more data?

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

the retriggers confirmed the culprit.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1767396

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:alexandrui could you generate a side-by-side for amazon and google-sides?

Comment 6

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: warm-side-by-side.mp4

amazon

Comment 7

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: cold-side-by-side.mp4

google-slides

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Thanks Alex! I see from the alert that there are some more improvements, which balance things out. For example amazon cold load improved more than the warm load regressed. I say we accept these as a new baseline.

Comment 9

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you Dave.
There are also some perf improvements listed in alert 34598 (found in bug 1767396 comment 14).

I'll keep the NI:myself, to try and understand why the timer change caused these effects. It could lead to some interesting discoveries and further improvements.

Comment 10

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

I won't have time to look deeper into this.