1779705 - 1108.33 - 32.5% google-docs FirstVisualChange / wikia fcp + 7 more (Linux) regression on Tue July 12 2022

Status: RESOLVED FIXED

(Core :: Graphics: Canvas2D, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a browsertime performance regression from push b8040f4090250260fab3e5dd6cb60d7f48a994a6. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
1108%	google-docs FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	40.00 -> 483.33
83%	google-docs SpeedIndex	linux1804-64-shippable-qr	fission warm webrender	470.08 -> 858.08
67%	google-slides FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	406.67 -> 680.00
62%	google-docs PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	675.75 -> 1,094.67
56%	google-docs ContentfulSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	777.33 -> 1,215.58
48%	google-slides SpeedIndex	linux1804-64-shippable-qr	fission warm webrender	553.58 -> 818.08
47%	google-slides PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	571.67 -> 837.58
42%	google-slides ContentfulSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	630.50 -> 894.42
33%	wikia fcp	linux1804-64-shippable-qr	cold fission webrender	228.96 -> 303.38

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
14%	google-docs-canvas FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	420.00 -> 360.00
12%	google-docs-canvas FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	416.67 -> 366.67
10%	google-docs-canvas PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	484.50 -> 437.50
10%	google-docs-canvas ContentfulSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,214.04 -> 1,097.58

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1777426

Comment 2

[Lee Salzman [:lsalzman]]

This appears to be a bug in our glue code for the browsertime harness itself and does not actually represent a performance regression. Details are posted in bug 1779953. Essentially, our browsertime results for FirstVisualChange and similar metrics are not entirely reliable until bug 1779953 is fixed, and so this perf alert is essentially erroneous.

Comment 3

[Lee Salzman [:lsalzman]]

Going to mark this as "fixed" for now, since the underlying bug that caused it was addressed in bug 1779953.