Closed Bug 1929606 Opened 29 days ago Closed 2 hours ago

30.58 - 3.74% imgur FirstVisualChange / paypal FirstVisualChange + 6 more (Linux) regression on Thu October 24 2024

Categories

(Core :: Widget: Gtk, defect)

defect

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox-esr128 --- unaffected
firefox132 --- unaffected
firefox133 --- wontfix
firefox134 --- fix-optional

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a browsertime performance regression from push 703fb617c8b70339e2c95cc43de4b8116d867f3e. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
31% imgur FirstVisualChange linux1804-64-shippable-qr cold fission webrender 524.07 -> 684.34 Before/After
11% imgur PerceptualSpeedIndex linux1804-64-shippable-qr cold fission webrender 874.18 -> 970.54 Before/After
8% paypal fcp linux1804-64-shippable-qr fission warm webrender 131.53 -> 141.79 Before/After
7% paypal largestContentfulPaint linux1804-64-shippable-qr fission warm webrender 137.04 -> 146.22 Before/After
5% paypal loadtime linux1804-64-shippable-qr fission warm webrender 227.31 -> 238.85 Before/After
4% paypal ContentfulSpeedIndex linux1804-64-shippable-qr fission warm webrender 179.62 -> 186.98 Before/After
4% imgur ContentfulSpeedIndex linux1804-64-shippable-qr cold fission webrender 1,125.60 -> 1,170.02 Before/After
4% paypal FirstVisualChange linux1804-64-shippable-qr fission warm webrender 158.80 -> 164.74 Before/After

Improvements:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
9% wikipedia largestContentfulPaint linux1804-64-shippable-qr cold fission webrender 1,286.06 -> 1,176.32 Before/After
8% wikipedia fcp linux1804-64-shippable-qr cold fission webrender 1,127.96 -> 1,037.01 Before/After
7% wikipedia ContentfulSpeedIndex linux1804-64-shippable-qr cold fission webrender 1,224.01 -> 1,132.58 Before/After
7% wikipedia FirstVisualChange linux1804-64-shippable-qr cold fission webrender 1,223.41 -> 1,132.10 Before/After
7% wikipedia PerceptualSpeedIndex linux1804-64-shippable-qr cold fission webrender 1,228.16 -> 1,136.98 Before/After
7% wikipedia SpeedIndex linux1804-64-shippable-qr cold fission webrender 1,235.56 -> 1,144.56 Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 42385

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to afinder@mozilla.com.

Flags: needinfo?(emilio)

Set release status flags based on info from the regressing bug 1924100

I'm a bit confused about these regressions, how sure are we of the regressing bug? In particular, the regressing patch was also an improvement in talos (see bug 1924100 comment 15), plus it should generally always be doing less work...

Flags: needinfo?(emilio) → needinfo?(afinder)

(In reply to Emilio Cobos Álvarez (:emilio) from comment #2)

I'm a bit confused about these regressions, how sure are we of the regressing bug? In particular, the regressing patch was also an improvement in talos (see bug 1924100 comment 15), plus it should generally always be doing less work...

Hi Emilio, thanks for reaching out via the need-info! It is not uncommon for the same revision to produce improvements on some tests, but cause regressions in others, even when it's across test harnesses (such as the talos improvement you mentioned and the regressions caused on browsertime). Looking at the imgur graph linked in comment 0, zoomed-in at around the specified culprit revision, there's a noticeable trend change introduced with this changeset, which also shows up as a "Regression" in the following Perfcompare link. I also triggered a side-by-side job for that revision, to see if the regression is reflected in the video comparison. Please let me know if there are other questions or concerns that need to be addressed.

Flags: needinfo?(afinder)

It has been over 7 days with no activity on this performance regression.

:emilio, since you are the author of the regressor, bug 1924100, which triggered this performance alert, could you please provide a progress update?

If you need additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Flags: needinfo?(emilio)

So looking at the graphs, bug 1922872 improved these, but regressed wikipedia, and bug 1924100 restored the the previous baseline.

Bug 1924100 is strictly less work. I think there's something a bit fishy about linux startup bleeding into other tests that we haven't figured out yet, but this is not a real regression afaict.

Flags: needinfo?(emilio)

It has been over 7 days with no activity on this performance regression.

:emilio, since you are the author of the regressor, bug 1924100, which triggered this performance alert, could you please provide a progress update?

If this regression is something that fixes a bug, changes the baseline of the regression metrics, or otherwise will not be fixed, please consider closing it as WONTFIX. See this documentation for more information on how to handle regressions.

For additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Flags: needinfo?(emilio)
Status: NEW → RESOLVED
Closed: 2 hours ago
Flags: needinfo?(emilio)
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.