1929606 - 30.58 - 3.74% imgur FirstVisualChange / paypal FirstVisualChange + 6 more (Linux) regression on Thu October 24 2024

Status: RESOLVED WONTFIX

(Core :: Widget: Gtk, defect)

Description

[Treeherder Bug Filer]

Perfherder has detected a browsertime performance regression from push 703fb617c8b70339e2c95cc43de4b8116d867f3e. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
31%	imgur FirstVisualChange	linux1804-64-shippable-qr	cold fission webrender	524.07 -> 684.34	Before/After
11%	imgur PerceptualSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	874.18 -> 970.54	Before/After
8%	paypal fcp	linux1804-64-shippable-qr	fission warm webrender	131.53 -> 141.79	Before/After
7%	paypal largestContentfulPaint	linux1804-64-shippable-qr	fission warm webrender	137.04 -> 146.22	Before/After
5%	paypal loadtime	linux1804-64-shippable-qr	fission warm webrender	227.31 -> 238.85	Before/After
4%	paypal ContentfulSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	179.62 -> 186.98	Before/After
4%	imgur ContentfulSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,125.60 -> 1,170.02	Before/After
4%	paypal FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	158.80 -> 164.74	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
9%	wikipedia largestContentfulPaint	linux1804-64-shippable-qr	cold fission webrender	1,286.06 -> 1,176.32	Before/After
8%	wikipedia fcp	linux1804-64-shippable-qr	cold fission webrender	1,127.96 -> 1,037.01	Before/After
7%	wikipedia ContentfulSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,224.01 -> 1,132.58	Before/After
7%	wikipedia FirstVisualChange	linux1804-64-shippable-qr	cold fission webrender	1,223.41 -> 1,132.10	Before/After
7%	wikipedia PerceptualSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,228.16 -> 1,136.98	Before/After
7%	wikipedia SpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,235.56 -> 1,144.56	Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 42385

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to [e-mail].

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1924100

Comment 2

[Emilio Cobos Álvarez (:emilio)]

I'm a bit confused about these regressions, how sure are we of the regressing bug? In particular, the regressing patch was also an improvement in talos (see bug 1924100 comment 15), plus it should generally always be doing less work...

Comment 3

[Alex Finder]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #2)

I'm a bit confused about these regressions, how sure are we of the regressing bug? In particular, the regressing patch was also an improvement in talos (see bug 1924100 comment 15), plus it should generally always be doing less work...

Hi Emilio, thanks for reaching out via the need-info! It is not uncommon for the same revision to produce improvements on some tests, but cause regressions in others, even when it's across test harnesses (such as the talos improvement you mentioned and the regressions caused on browsertime). Looking at the imgur graph linked in comment 0, zoomed-in at around the specified culprit revision, there's a noticeable trend change introduced with this changeset, which also shows up as a "Regression" in the following Perfcompare link. I also triggered a side-by-side job for that revision, to see if the regression is reflected in the video comparison. Please let me know if there are other questions or concerns that need to be addressed.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

It has been over 7 days with no activity on this performance regression.

:emilio, since you are the author of the regressor, bug 1924100, which triggered this performance alert, could you please provide a progress update?

If you need additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Comment 5

[Emilio Cobos Álvarez (:emilio)]

So looking at the graphs, bug 1922872 improved these, but regressed wikipedia, and bug 1924100 restored the the previous baseline.

Bug 1924100 is strictly less work. I think there's something a bit fishy about linux startup bleeding into other tests that we haven't figured out yet, but this is not a real regression afaict.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

It has been over 7 days with no activity on this performance regression.

:emilio, since you are the author of the regressor, bug 1924100, which triggered this performance alert, could you please provide a progress update?

If this regression is something that fixes a bug, changes the baseline of the regression metrics, or otherwise will not be fixed, please consider closing it as WONTFIX. See this documentation for more information on how to handle regressions.

For additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Comment 7

[Emilio Cobos Álvarez (:emilio)]

See comment 5