1945079 - 7.72% perf_reftest_singletons tiny-traversal-singleton.html (Linux) regression on Wed January 22 2025

Status: RESOLVED WONTFIX

(Firefox :: Remote Settings Client, defect)

Description

[Treeherder Bug Filer]

Perfherder has detected a talos performance regression from push 176f6d1d5c2367b138a91e8315b72c5af0cb0cbf. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
8%	perf_reftest_singletons tiny-traversal-singleton.html	linux1804-64-shippable-qr	e10s fission stylo webrender	932.16 -> 1,004.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 43457

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to [e-mail].

Comment 1

[Mathieu Leplatre [:leplatrem]]

Valentin, could you please check whether it's not related to https://bugzilla.mozilla.org/show_bug.cgi?id=1943262 ?

I don't think I can personnally do much here unfortunately

Thank you

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1786310

Comment 3

[Valentin Gosu [:valentin] (he/him)]

I triggered a couple of perf runs for https://treeherder.mozilla.org/jobs?repo=autoland&revision=b233889facbe1460da732b79549b4a34991e9c04 to see if that improved things.

The alert pushlog does include a few other bugs, so I'm not 100% sure this is caused by bug 1786310
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=293b78da4ba7cf98218a99b75a960ebebc90b67e&tochange=176f6d1d5c2367b138a91e8315b72c5af0cb0cbf

At worst we could backout bug 1786310 and see if things go back to normal.

Comment 4

[Valentin Gosu [:valentin] (he/him)]

I triggered a perf run for a backout to confirm.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

The Try perf for the backout doesn't show any improvement.

Looking at the original perf alert closer, it doesn't seem to have been caused by 1786310, but possibly something else in that pushlog.
https://perf.compare/compare-results?baseRev=2e6723e81dd19a3172a49d6c6bb880ca4bcd8a0b&baseRepo=autoland&newRev=35eeb4c510c6599b186dacc51009414e1125f23e&newRepo=try&newRev=176f6d1d5c2367b138a91e8315b72c5af0cb0cbf&newRepo=autoland&framework=1

Comment 6

[Julien Wajsberg [:julienw]]

Hey Alex, would you be able to look at this alert again and double check this is the right regressor? Thanks!

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1786310

Comment 8

[Alex Finder]

(In reply to Julien Wajsberg [:julienw] from comment #6)

Hey Alex, would you be able to look at this alert again and double check this is the right regressor? Thanks!

Hey Julien! Looking at the try perf for the backout posted by Valentin in comment #5, it seems the comparison is looking at the suite level (perf_reftest_singletons). If we look at the subtest tiny-traversal-singleton.html reported in comment 0, both the graph and perfcompare seem to show an improvement on the backout.

Valentin, let me know if this helps with the bug investigation. I can also look if there is any other push that could have caused the regression, the area around that revision is a bit noisy on the graph.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

It has been over 7 days with no activity on this performance regression.

:leplatrem, since you are the author of the regressor, bug 1786310, which triggered this performance alert, could you please provide a progress update?

If this regression is something that fixes a bug, changes the baseline of the regression metrics, or otherwise will not be fixed, please consider closing it as WONTFIX. See this documentation for more information on how to handle regressions.

For additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Comment 10

[Valentin Gosu [:valentin] (he/him)]

I tried to figure it out last week, but I can't make out why there's a perf regression. I couldn't figure out anything obvious from the profiler.
If this is important I'm OK with backing out the regression.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

It has been over 7 days with no activity on this performance regression.

:leplatrem, since you are the author of the regressor, bug 1786310, which triggered this performance alert, could you please provide a progress update?

If this regression is something that fixes a bug, changes the baseline of the regression metrics, or otherwise will not be fixed, please consider closing it as WONTFIX. See this documentation for more information on how to handle regressions.

For additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Comment 12

[Valentin Gosu [:valentin] (he/him)]

I'm going to close this to silence the bugbot and open a different one to investigate the causes of the regression.