1605344 - 26.33 - 26.81% raptor-webaudio-firefox (macosx1014-64-shippable) regression on push 65cf656ecce94b8c8bc4933cf57eb760a3b8d10f (Wed December 18 2019)

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Florin Strugariu [:Bebe]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=65cf656ecce94b8c8bc4933cf57eb760a3b8d10f

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

27% raptor-webaudio-firefox macosx1014-64-shippable opt 155.08 -> 196.67
26% raptor-webaudio-firefox macosx1014-64-shippable opt 155.08 -> 195.92

Improvements:

11% raptor-speedometer-firefox macosx1014-64-shippable opt 58.37 -> 64.86
11% raptor-speedometer-firefox macosx1014-64-shippable opt 58.38 -> 64.70
8% raptor-tp6-twitch-firefox-cold macosx1014-64-shippable opt 345.80 -> 316.98
7% raptor-tp6-twitch-firefox-cold fcp macosx1014-64-shippable opt 112.88 -> 105.08
6% raptor-tp6-wikipedia-firefox-cold loadtime macosx1014-64-shippable opt 2,188.08 -> 2,052.00
6% raptor-tp6-twitch-firefox-cold loadtime macosx1014-64-shippable opt 1,709.92 -> 1,610.33
6% raptor-tp6-wikipedia-firefox-cold macosx1014-64-shippable opt 1,929.78 -> 1,821.35
4% raptor-tp6-google-mail-firefox-cold loadtime macosx1014-64-shippable opt 1,011.67 -> 970.42
4% raptor-tp6-slides-firefox-cold macosx1014-64-shippable opt 1,840.98 -> 1,776.43
3% raptor-tp6-sheets-firefox-cold macosx1014-64-shippable opt 1,369.07 -> 1,321.74
3% raptor-tp6-sheets-firefox-cold loadtime macosx1014-64-shippable opt 2,391.96 -> 2,310.58
3% raptor-tp6-google-firefox-cold fcp macosx1014-64-shippable opt 515.58 -> 499.08
3% raptor-tp6-reddit-firefox-cold macosx1014-64-shippable opt 1,103.15 -> 1,068.88

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24539

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Chris Manchester (limited bugmail, email directly)]

This should be taken care of by bug 1596182.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Per this graph: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,2051979,1,10&timerange=2592000&zoom=1576544815446,1578096250896,114.44780400177382,234.3339820340042

The regression is now fixed, and there is a modest improvement over the "before" state.