1789828 - 11.41 - 2.89% youtube ContentfulSpeedIndex / youtube PerceptualSpeedIndex + 3 more (Android) regression on Fri September 2 2022

Status: RESOLVED WONTFIX

(Core :: DOM: CSS Object Model, defect)

Description

[Alex Finder]

Perfherder has detected a browsertime performance regression from push 3f26214c64a46e20926bdb4ed277ab57b0d2bd61. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
11%	youtube ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable-qr	warm webrender	481.33 -> 536.25
11%	youtube ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable-qr	warm webrender	484.96 -> 539.33
4%	youtube SpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	887.67 -> 922.75
4%	youtube SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable-qr	warm webrender	556.67 -> 578.33
3%	youtube PerceptualSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	740.25 -> 761.67

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
5%	ebay-kleinanzeigen FirstVisualChange	android-hw-p2-8-0-android-aarch64-shippable-qr	warm webrender	582.17 -> 552.08
5%	ebay-kleinanzeigen loadtime	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	1,058.15 -> 1,009.62
4%	ebay-kleinanzeigen loadtime	android-hw-p2-8-0-android-aarch64-shippable-qr	warm webrender	727.42 -> 695.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1787072

Comment 2

[Emilio Cobos Álvarez [:emilio]]

Is there any chance to get the side-by-side recordings? My guess is that my patch should very much be a win, but I suppose having the img::before rule unconditionally may cause more work in some cases where images are never broken?

Comment 3

[Timothy Nikkel (:tnikkel)]

If you click on a perf job on tree herder in the artifacts you can download an archive, inside the archive you'll find videos and screenshots. You can compare from a job with/without the patch in question.

Another way that is probably overall easier to analyze is to again find the job in treeherder and then click "generate profile", this will trigger another job, and then you can open the profiler and look at the screenshots and more. I'd recommend you push with the patch in bug 1780315 to fix a bug where the last screenshot doesn't show up in the profiler.

In my experience a lot of these come from more or less invalidates/paints that change the computed metrics but due to some weird artifact and not a real perf change.

Comment 4

[Pascal Chevrel (relman team) -> :pascalc]

Emilio, is that something we should track for a potential uplift in beta 106?

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1787072

Comment 6

[Emilio Cobos Álvarez [:emilio]]

I don't think this is worth fixing. It was a win on metrics we cared about more, and lots of work has gone into perf since then.