1585903 - 2.28 - 4.16% raptor-speedometer-fennec (android-hw-p2-8-0-android-aarch64-nightly, android-hw-p2-8-0-arm7-api-16-nightly) regression on push 8d4dfb36052db73c031be45d89405b560d382229 (Thu September 26 2019)

Status: RESOLVED WONTFIX

(Web Compatibility :: Interventions, defect)

Description

[Marian Raiciof [:marauder]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?changeset=8d4dfb36052db73c031be45d89405b560d382229

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

4% raptor-speedometer-fennec android-hw-p2-8-0-arm7-api-16-nightly opt 29.27 -> 28.05
2% raptor-speedometer-fennec android-hw-p2-8-0-android-aarch64-nightly opt 25.93 -> 25.33

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=23302

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Mike Taylor [:miketaylr]]

I believe we should accept this regression, as these patches address compatibility issues. Anything related to google.com will be short-lived, as it's for an experiment intended to run for a month.

Comment 2

[Marian Raiciof [:marauder]]

Thanks Mike, i'll close this bug as won't fix and mark the alert the same.