1935744 - 11.06 - 7.56% perf_reftest_singletons external-string-pass.html / perf_reftest_singletons external-string-pass.html (OSX) regression on Mon November 25 2024

Status: RESOLVED WORKSFORME

(Firefox :: Search, defect)

Description

[Treeherder Bug Filer]

Perfherder has detected a talos performance regression from push 2111ce34dd4b26fa94b0fb2fdc7896eedcca1750. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
11%	perf_reftest_singletons external-string-pass.html	macosx1015-64-shippable-qr	e10s fission stylo webrender	712.89 -> 791.73
8%	perf_reftest_singletons external-string-pass.html	macosx1015-64-shippable-qr	e10s fission stylo webrender	702.81 -> 755.91

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 42967

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to [e-mail].

Comment 1

[Andrej (:aglavic)]

Hey mbeier,
The commits that caused the regression are the following 4:
I'm not sure which of the 4 caused this can you help me narrow this down?
2111ce34dd4b 1930488 - Remove support for IconUpdateUrl definition in OpenSearch engines (no longer necessary). r=Standard8
241193b34db1 1928410 - Refactor search settings file migration into separate functions. r=Standard8
e4d5a9f29386 1655076 - Drop the separate _iconURL for search engines, store everything in the icon map. r=Standard8
3814d81f7265 1655070 - getIconURL should do a best-fit icon comparison and always return an icon. r=Standard8

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1655076

Comment 3

[Mark Banner (:standard8)]

I don't see how the changes referenced would have affected this. As far as I can tell, the test is only testing assigning a string to a drop property.

The changes have changed the toolkit search service and at worst, I think would only affect startup performance, however not in a way that would be found here - this looks like back-end code rather than front-end code that's causing the issue.

That said, I also can't see a revision nearby that might have caused this.

Given that this also went down a few days later, might there have been a back-out that caused this originally, was backed out and not shown here?

Also adding NI to :emelio as he appears to be owner of this test, so may have some ideas.

Comment 4

[Emilio Cobos Álvarez [:emilio]]

Agreed that shouldn't have caused it. Given the numbers went back to baseline after this I don't think it's worth digging too much into it...

Comment 5

[Andrej (:aglavic)]

If you are of the opinion that none of the revisions nearby could have likely cause this, and since the regression has reverted on its own I will close this as worksforme

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

A comment containing a reason for why the performance regression was resolved as WORKSFORME could not be found. It should provided when the status of the bug is changed.

:aglavic, since you resolved this bug, could you provide a comment explaining the resolution? If one has already been provided, this needinfo can be ignored/removed.

If you need additional information/help, reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.