1720267 - 17.93 - 13.36% youtube SpeedIndex / about:welcome ContentfulSpeedIndex + 1 more (Windows) regression on Fri July 9 2021

Status: RESOLVED WONTFIX

(Firefox :: Security, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a browsertime performance regression from push a51628e3d9559952ddd68dbb6034b8ce40c80fb0. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
18%	youtube	SpeedIndex	windows10-64-shippable-qr	warm webrender	793.33 -> 935.58
14%	welcome	ContentfulSpeedIndex	windows10-64-shippable-qr	cold webrender	379.21 -> 432.42
13%	welcome	ContentfulSpeedIndex	windows10-64-shippable-qr	cold webrender	378.08 -> 428.58

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
47%	welcome	FirstVisualChange	windows10-64-shippable-qr	cold webrender	75.29 -> 40.00
43%	welcome	FirstVisualChange	windows10-64-shippable-qr	cold webrender	70.00 -> 40.00
40%	welcome	FirstVisualChange	windows10-64-shippable-qr	cold webrender	67.20 -> 40.00
35%	welcome	loadtime	windows10-64-shippable-qr	cold webrender	100.25 -> 64.79
32%	welcome	loadtime	linux1804-64-shippable-qr	cold webrender	117.83 -> 80.12
...	...	...	...	...	...
4%	welcome	PerceptualSpeedIndex	windows10-64-shippable-qr	cold webrender	435.25 -> 416.75

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Chris Peterson [:cpeterson]]

Tentatively tracking this bug for Fission since this was a regression from a Fission code change (bug 1706302).

Comment 2

[Nika Layzell [:nika] (ni? for response)]

It's somewhat surprising that this is causing such large changes. Given that the tests which are changes are the 'welcome' tests, it seems likely that this is related to the changes to AboutWelcomeChild to stop installing a WebProgressListener. This means that we do less work when loading about:welcome, but also has the impact that we'll not initialize the actor in the parent process until slightly later, which could potentially slow down fetching theme information from the parent process slightly. I might give a shot at doing some pre-loading of the parent process actor to see if it fixes the regression.

From the graph looking at the youtube regression, it seems like this change just regressed the improvement from https://bugzilla.mozilla.org/show_bug.cgi?id=1720435, which only lasted for ~1 day. I don't know why this would be happening exactly, but it seems like it might be an intermittent issue?

Comment 3

[Chris Peterson [:cpeterson]]

WONTFIX

We don't care about this apparent regression in about:welcome's ContentfulSpeedIndex. While ContentfulSpeedIndex regressed, some other about:welcome page load metrics improved.

Also, the about:welcome page is only shown once to users install or upgrading Firefox and the page is not performance sensitive. The animations are slow and probably impacting the ContentfulSpeedIndex measurements.