1860853 - 13.3 - 11.17% youtube ContentfulSpeedIndex / youtube ContentfulSpeedIndex (Android) regression on Wed October 11 2023

Status: RESOLVED FIXED

(Testing :: Performance, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a browsertime performance regression from push bbe69c1ce114d5a824dc1903e6421b9b720cc88b. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
13%	youtube ContentfulSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	526.95 -> 597.05
11%	youtube ContentfulSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	523.36 -> 581.81

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Chris Peterson [:cpeterson]]

Olli, could this YouTube regression on Android be caused by your fix for "normal task" bug 1857618? Alex says your fix is in the regression range.

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&series=firefox-android,4653708,1,13&timerange=2592000&zoom=1695588494129,1698186499123,401.85555555555555,737.9444444444445

Comment 2

[Andrew Creskey [:acreskey]]

From a quick scan of the commits, this one also seemed like it could have changed the performance results for this test:
https://hg.mozilla.org/mozilla-central/rev/4953e7db7d0848437a80a5b181a780dd6ab97eff

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

Yes, most likely. As long as there are some significant scheduling changes happening, various numbers will fluctuate.

Comment 4

[Andrew Creskey [:acreskey]]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #3)

Yes, most likely. As long as there are some significant scheduling changes happening, various numbers will fluctuate.

Thanks, Olli.
Given that Bug 1857618 is a calculated, broader, performance optimization, then I imagine we will want to keep it and do nothing with this particular regression?

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

I am investigating if bug 1857618 could be fixed in a bit different way, but I don't have good numbers yet.

Comment 6

[Dianna Smith [:diannaS]]

fixed by backout of bug 1857618 on 120.0b6

Comment 7

[Alex Finder]

Hi Alex! This came up in triage today. Could you please check priority and severity for this ? And if it's still reproducible after the backout ?