1838934 - 13.84% perf_reftest_singletons id-getter-2.html (Windows) regression on Wed June 14 2023

Status: RESOLVED WORKSFORME

(Firefox :: Translations, defect)

Description

[Andrej (:aglavic)]

Hi :gregtatum,
This alert was generated from: https://treeherder.mozilla.org/jobs?repo=autoland&revision=379da4a3c107c5db724c73619cde6773ee13e14d
Which includes both bug 1837078, and 1836505
Based on the content of 1836505 I figured this bug was the more likely culprit, if I put down the wrong bug file a need info and I will re-generate the bug on the correct ticket

Perfherder has detected a talos performance regression from push 379da4a3c107c5db724c73619cde6773ee13e14d. As author of one of the patches included in that push, we need your help to address this regression.

== Change summary for alert #38727 (as of Fri, 16 Jun 2023 22:50:53 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
14%	perf_reftest_singletons id-getter-2.html	windows10-64-shippable-qr	e10s fission stylo webrender	499.59 -> 568.72

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
14%	tp5o_webext responsiveness	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	4.11 -> 3.54
13%	tp5o_webext responsiveness	linux1804-64-shippable-qr	e10s fission stylo webrender	4.28 -> 3.73
13%	tp5o responsiveness	linux1804-64-shippable-qr	e10s fission stylo webrender	3.45 -> 3.01
12%	tp5o_webext responsiveness	linux1804-64-shippable-qr	e10s fission stylo webrender	4.29 -> 3.77

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=38727

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1837078

Comment 2

[Greg Tatum [:gregtatum]]

This profile stacks are empty, but it looks like we're attempting to identify the text on the page: https://share.firefox.dev/3XiFpct

The test could be updated with a declared language so it's measuring what we care about, or maybe we could not run if the text on the page is empty.

Comment 3

[Greg Tatum [:gregtatum]]

For this alert, I would like to have:

A link to the profile with the regressing test, here I had to figure out how to generate it myself: https://share.firefox.dev/3XiFpct

Working sampling on the test.

A filtered link to the job that is regressing: https://treeherder.mozilla.org/jobs?repo=autoland&revision=379da4a3c107c5db724c73619cde6773ee13e14d&searchStr=talos&selectedTaskRun=Rp40XQmPSsepiMgIJF0J8w.0

A ./mach try ... snippet for re-triggering the test on my own.

A snippet to run the test locally (if possible): ./mach test ...

Comment 4

[Dianna Smith [:diannaS]]

:aglavic for https://bugzilla.mozilla.org/show_bug.cgi?id=1838934#c3 is this something you can help with?

Comment 5

[Andrej (:aglavic)]

Hi :diannaS,
Yes I can help with this and my apologies to you :gregtatum for missing your questions, with H1 coming to a close I must have missed your comment in my inbox, appologies for this

A ./mach try ... snippet for re-triggering the test on my own.

• You can run ../mach try fuzzy --full with the following query: 'windows10-64-shippable 'talos 'perf-reftest-singletons to get this running on CI

A snippet to run the test locally (if possible): ./mach test ...

• You can run the test locally with: ./mach talos-test --activeTests perf_reftest_singletons

Could you elaborate on what you mean by working sampling on the test? Is that within the profiler

Comment 6

[Andrej (:aglavic)]

Please disregard my question, I see now the test does no sampling on the profiler I will investigate why this is occurring and get back to you

Comment 7

[Andrej (:aglavic)]

Bug tracking the issue for the working profiling on the perf_reftest_singletons: Bug 1838824

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1837078

Comment 9

[Greg Tatum [:gregtatum]]

This is either solved, or the translations code is not to blame in the difference on the numbers.

I was able to load the test manually locally by serving up the files in the directory, and run the profiler on it, without invoking the talos machinery.

https://share.firefox.dev/3JLGxzZ

There are 2 translations samples in the parent process, and I can't see any reason why translations would slow this down. This is about element id access, with the translations code doesn't have anything to do with. I originally suspected some of the language ID code, but that is not anywhere to be seen, and I did a lot of perf work to remove anything translations related.

Comment 10

[Greg Tatum [:gregtatum]]

/cc :canova This bug was super difficult to work with since the gecko profiler isn't providing any samples for Talos tests, in automation and locally.

Comment 11

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Sorry to hear that it wasn't a good experience. Someone else actually had the same issue you had lately and I started looking into this. I have a patch waiting for review in Bug 1838824 which fixes no-samples issue you had.

Unfortunately talos is not properly resuming the profiler when it's starting this test. Usually talos does that automatically, but some tests are handling their own test start/finish functions and this was one of them. Profiler resume/pause needs to be handled in those functions explicitly.

Comment 12

[Greg Tatum [:gregtatum]]

Great, thanks for working on it and letting me know about it. I appreciate the help on it 👍

Comment 13

[:cmkm]

Hi Greg, is there anything we can do to move this along or close it out?

Comment 14

[Greg Tatum [:gregtatum]]

I don't see any evidence of the translations code regressing this test, as it's not present in the profile I gathered. I would suggest we close it out.

Comment 15

[Dianna Smith [:diannaS]]

closing out per https://bugzilla.mozilla.org/show_bug.cgi?id=1838934#c14