1802512 - 8.57% office ContentfulSpeedIndex (OSX) regression on Mon November 14 2022

Status: RESOLVED INVALID

(Firefox :: Firefox View, defect)

Description

[Andra Esanu (needinfo me)]

Perfherder has detected a browsertime performance regression from push 4791730df49f40f22e496937ab5fe5dd3c50b483. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
9%	office ContentfulSpeedIndex	macosx1015-64-shippable-qr	cold fission webrender	830.54 -> 901.75

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
6%	office ContentfulSpeedIndex	macosx1015-64-shippable-qr	cold fission webrender	852.83 -> 800.50

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

Andra, the link to the alert summary is broken in the initial comment is broken. It uses localhost

Comment 2

[Andra Esanu (needinfo me)]

Sorry for that, fixed it now.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1780441

Comment 4

[Jeff Muizelaar [:jrmuizel]]

It also looks like this metric returned to the baseline and wasn't a permanent regression.

Comment 5

[Hanna Jones [:hjones]]

I can offer a bit of context (though it might not be needed based on the latest comment) - the linked push changed how some Firefox View tests opened the Firefox View tab. Previously tests were using BrowserTestUtils.withNewTab to load about:firefoxview instead of clicking on the Fx View toolbar button. I switched the tests to use the toolbar button click since that's more reflective of how users actually navigate to the page. There did seem to be some timing differences between the two methods of opening the tab, but if there are any perf changes they would be to the tests only, and might actually be more accurate now that the tests are opening the tab the same way users do.

Comment 6

[:Gijs (he/him)]

Yeah, this isn't the right regressing bug - that change only touched browser mochitest files and those don't run as part of raptor perf tests.