1638265 - 2.04 - 6.81% raptor-motionmark-htmlsuite-firefox / raptor-speedometer-firefox / raptor-tp6 raptor-webaudio-firefox (linux64-shippable, linux64-shippable-qr) regression on push 80ba3f3cfaf9b31341bca28e2e6aea383cec8acc (Thu May 14 2020)

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a raptor performance regression from push 80ba3f3cfaf9b31341bca28e2e6aea383cec8acc. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

7% raptor-webaudio-firefox linux64-shippable opt 139.42 -> 148.92
7% raptor-webaudio-firefox linux64-shippable-qr opt 144.00 -> 153.58
5% raptor-tp6-netflix-firefox-cold fcp linux64-shippable-qr opt 594.25 -> 624.50
5% raptor-tp6-netflix-firefox-cold fcp linux64-shippable opt 567.92 -> 596.50
4% raptor-motionmark-htmlsuite-firefox linux64-shippable opt 31.07 -> 29.70
4% raptor-tp6-twitch-firefox-cold loadtime linux64-shippable-qr opt 1,248.50 -> 1,292.92
3% raptor-tp6-twitch-firefox-cold loadtime linux64-shippable opt 1,208.00 -> 1,249.67
3% raptor-tp6-outlook-firefox-cold loadtime linux64-shippable opt 817.54 -> 843.75
3% raptor-tp6-google-mail-firefox-cold loadtime linux64-shippable-qr opt 602.21 -> 619.33
3% raptor-speedometer-firefox linux64-shippable opt 93.99 -> 91.43
3% raptor-speedometer-firefox linux64-shippable-qr opt 93.14 -> 90.60
3% raptor-tp6-youtube-firefox-cold fcp linux64-shippable-qr opt 883.54 -> 907.42
3% raptor-tp6-outlook-firefox-cold loadtime linux64-shippable-qr opt 815.79 -> 836.75
3% raptor-tp6-google-mail-firefox-cold loadtime linux64-shippable opt 593.42 -> 608.50
3% raptor-tp6-slides-firefox-cold fcp linux64-shippable opt 1,157.42 -> 1,186.75
3% raptor-tp6-slides-firefox-cold linux64-shippable-qr opt 1,258.74 -> 1,290.55
2% raptor-tp6-youtube-firefox-cold loadtime linux64-shippable-qr opt 1,445.33 -> 1,480.42
2% raptor-tp6-slides-firefox-cold linux64-shippable opt 1,182.02 -> 1,210.67
2% raptor-tp6-pinterest-firefox-cold loadtime linux64-shippable opt 1,957.17 -> 2,003.67
2% raptor-tp6-docs-firefox-cold fcp linux64-shippable-qr opt 1,049.96 -> 1,073.92
2% raptor-tp6-yahoo-mail-firefox-cold loadtime linux64-shippable opt 921.75 -> 942.58
2% raptor-tp6-slides-firefox-cold fcp linux64-shippable-qr opt 1,351.50 -> 1,381.50
2% raptor-tp6-docs-firefox-cold fcp linux64-shippable opt 1,021.46 -> 1,043.83
2% raptor-tp6-slides-firefox-cold loadtime linux64-shippable-qr opt 1,959.96 -> 2,002.33
2% raptor-tp6-sheets-firefox-cold linux64-shippable-qr opt 927.09 -> 946.04

Improvements:

5% raptor-tp6-yahoo-mail-firefox-cold replayed linux64-shippable opt 943.29 -> 993.42

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Florin Strugariu [:Bebe]]

== Change summary for alert #25960 (as of Fri, 15 May 2020 00:32:00 GMT) ==

Regressions:

3% speedometer linux64-shippable opt 93.30 -> 90.69
3% speedometer linux64-shippable-qr opt 93.28 -> 90.88
2% speedometer windows10-64-qr opt 74.44 -> 72.91

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25960

Comment 2

[(Away)]

It's my understanding that sometimes it takes a while for different sets of perf results to arrive and/or be assigned to the right alert. Can you please confirm that this is the full set of regressions from bug 1616692?

At the moment this looks strongly like a linux64-specific regression, but I don't want to waste time on that avenue of investigation if this is really a cross-platform problem that we're only seeing one face of.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1616692

Comment 4

[Florin Strugariu [:Bebe]]

== Change summary for alert #25979 (as of Sat, 16 May 2020 16:07:52 GMT) ==

Regressions:

3% speedometer windows7-32 opt 77.43 -> 75.45

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=25979

Comment 5

[Julien Cristau [:jcristau]]

The clang update was backed out.

Comment 6

[Florin Strugariu [:Bebe]]

== Change summary for alert #26015 (as of Wed, 20 May 2020 16:32:24 GMT) ==

Improvements:

3% speedometer linux64-shippable-qr opt 90.56 -> 92.98
2% speedometer linux64-shippable opt 90.75 -> 92.87

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26015

Comment 7

[Florin Strugariu [:Bebe]]

== Change summary for alert #26022 (as of Wed, 20 May 2020 21:10:08 GMT) ==

Regressions:

4% allrecipes-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 7,166.00 -> 7,432.21
3% allrecipes-cold LastVisualChange android-hw-g5-7-0-arm7-api-16 pgo 7,436.67 -> 7,646.67
3% allrecipes-cold SpeedIndex android-hw-g5-7-0-arm7-api-16 pgo 5,487.50 -> 5,630.58
2% allrecipes-cold PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16 pgo 3,881.21 -> 3,960.58

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26022

Comment 8

[Mike Hommey [:glandium]]

Presumably, this should have been closed around comment 5.