1650003 - 14.23 - 18.87% raptor-tp6-slides-firefox-cold loadtime (windows10-64-shippable, windows10-64-shippable-qr, windows7-32-shippable) regression on push ee812b6359befd98993f39f8cffd6735ebcd5644 (Thu June 25 2020)

Status: RESOLVED FIXED

(Testing :: Performance, defect, P3)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a raptor performance regression from push ee812b6359befd98993f39f8cffd6735ebcd5644. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

19% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable opt 1,918.08 -> 2,280.08
17% raptor-tp6-slides-firefox-cold loadtime windows7-32-shippable opt 1,884.69 -> 2,214.25
14% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,962.87 -> 2,242.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Florin Strugariu [:Bebe]]

:rcaliman :prathiksha : jonco one of your bugs generated this regression.
i can't find which one is the cause because of the build bustages

maybe one of you knows what bug caused this...

Comment 2

[Florin Strugariu [:Bebe]]

== Change summary for alert #26366 (as of Sun, 28 Jun 2020 16:50:18 GMT) ==

Regressions:

25% espn fcp android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,647.67 -> 7,060.75

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26366

Comment 3

[Razvan Caliman [:rcaliman]]

Bug 1644789 is unlikely to have caused this regression. The patch for that bug fixes a test in the DevTools suite which is not part of either the Raptor or DevTools Talos suites.

Comment 4

[Jon Coppeard (:jonco)]

Bug 1647803 moves fields around in a class hierarchy but results in the same layout in memory. I don't think it's to blame.

Comment 5

[Matthew N. [:MattN]]

Bug 1597358 is preffed off so the only cost is one new pref check. I guess that would leave bug 1599567 which is just adding telemetry but it doesn't run early during startup so I'm not sure it's relevant (I'd have to look at the pref. test more).

When I look at the graphs for this test almost looks bimodal and it has now gone back to the lower variant after unrelated backouts (alert 26395) so maybe there isn't a regression to fix anymore? Florin, do you agree?

Comment 6

[Florin Strugariu [:Bebe]]

I agree that the regression was fixed. not sure if the fix is related with bug 1599567

Comment 7

[Matthew N. [:MattN]]

Should the bug be resolved then?