Open Bug 1952486 Opened 15 days ago Updated 20 hours ago

4.14% yahoo-mail fcp (Linux) regression on Sat February 22 2025

Categories

(Testing :: Performance, defect)

Product:
Testing
Component:
Performance
Type:
defect

Tracking

(Not tracked)

People

(Reporter: intermittent-bug-filer, Unassigned, NeedInfo)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a browsertime performance regression from push 9f38989c451ec2a42f8a824ad55eba3d0086ad58. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
11% ebay largestContentfulPaint macosx1470-64-shippable fission warm webrender 153.21 -> 169.48 Before/After

Improvements:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
4% yahoo-mail fcp linux1804-64-shippable-qr bytecode-cached fission warm webrender 162.17 -> 155.45 Before/After
4% twitter loadtime windows11-64-shippable-qr cold fission webrender 376.72 -> 361.39 Before/After
4% yahoo-mail loadtime windows11-64-shippable-qr fission warm webrender 236.72 -> 227.72 Before/After
4% yahoo-mail fcp windows11-64-shippable-qr bytecode-cached fission warm webrender 89.20 -> 85.94 Before/After
4% yahoo-mail loadtime windows11-64-24h2-shippable fission warm webrender 238.02 -> 229.33 Before/After
... ... ... ... ... ...
2% yahoo-mail loadtime linux1804-64-shippable-qr bytecode-cached fission warm webrender 299.53 -> 293.01 Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 44179

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to fbilt@mozilla.com.

Flags: needinfo?(mh+mozilla)

It's hard to believe that bug 1910796 is involved. zlib functions don't even seem to appear in the profiles.

Flags: needinfo?(mh+mozilla)

Moving to Testing::Performance as this is a perf regression. Please retarget to appropriate Product/Component if this is incorrect.
Someone needs to look deeper into the regression as Mike doesn't think the identified bug is the regressor here.

Component: General → Performance
Product: Core → Testing

For good measure I pushed a backout to try yesterday, but the mac worker pool is saturated and I'm still waiting for the results.

Try would seem to confirm there's something going on: https://perf.compare/compare-results?baseRev=39d890ae576d9ab97477600677f8b2898c63b338&baseRepo=try&newRev=88c2caf5397527a6a5d5aee08679cd8ae78d289d&newRepo=try&framework=13

But also, looking at the details, and seeing how the values span from ~80 to ~240... also, that when looking at profiles, largestContentfulPaint either doesn't appear at all or it has a different value than the one reported in the test result... it all seems like a huge red herring.

Flags: needinfo?(nkochar)

Frank, who from the perf team could look into this?

Flags: needinfo?(nkochar) → needinfo?(fdoty)

Reaching out to the perf test team for assistance

Flags: needinfo?(fdoty)

:fbilt, can you double-check to see if the regression culprit is correct?

Flags: needinfo?(fbilt)

I'm pretty sure this is the culprit behind the regression (9f38989c451ec2a42f8a824ad55eba3d0086ad58).
I also talked to other sheriffs who pointed to the same culprit as I did.

Flags: needinfo?(fbilt)
You need to log in before you can comment on or make changes to this bug.