1463052 - 12.7 - 19.27% damp (linux64, linux64-qr, osx-10-10, windows10-64, windows10-64-qr, windows7-32) regression on push 6c13972f0b629a7634228d3867c411ac927223bb (Fri May 18 2018)

Status: RESOLVED WONTFIX

(DevTools :: Framework, defect, P3)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=6c13972f0b629a7634228d3867c411ac927223bb

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 19%  damp linux64 opt e10s stylo     66.59 -> 79.42
 19%  damp linux64 pgo e10s stylo     64.32 -> 76.42
 18%  damp windows10-64 pgo e10s stylo72.54 -> 85.29
 17%  damp windows7-32 pgo e10s stylo 75.07 -> 87.99
 17%  damp windows10-64 opt e10s stylo76.93 -> 89.93
 15%  damp linux64-qr opt e10s stylo  71.35 -> 81.80
 14%  damp osx-10-10 opt e10s stylo   115.81 -> 132.25
 14%  damp windows7-32 opt e10s stylo 83.83 -> 95.16
 13%  damp windows10-64-qr opt e10s stylo80.75 -> 91.01


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=13294

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:ochameau Given that this is just a damp test suite update, I would mark this as WONTFIX, unless you see something strange with these results.

Comment 2

[Alexandre Poirot [:ochameau]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> :ochameau Given that this is just a damp test suite update, I would mark
> this as WONTFIX, unless you see something strange with these results.

Yes. I only removed a couple of tests we no longer really track.
It is surprising to see the summary going up...
I imagine the removed tests were having lower baselines than the one that have been kept.