1380250 - 5.67 - 6.31% tps (linux64) regression on push 9c31c803b3b4982c1fe05782f7ea9524a5c3099b (Mon Jul 10 2017)

Status: RESOLVED WONTFIX

(Firefox :: Tabbed Browser, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=9c31c803b3b4982c1fe05782f7ea9524a5c3099b

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  6%  tps summary linux64 pgo e10s     25.23 -> 26.83
  6%  tps summary linux64 opt e10s     26.65 -> 28.16

Improvements:

 13%  tps summary osx-cross opt e10s     25.94 -> 22.60


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=7797

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

Mike, I know you worked on de-noising the TPS performance tests. Are the linux regressions expected so that we could accept them?

Comment 2

[Mike Conley (:mconley) (:⚙️)]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> Mike, I know you worked on de-noising the TPS performance tests. Are the
> linux regressions expected so that we could accept them?

Yeah, I think we can safely say that the test is strictly more accurate than the previous version. Since only the test changed, I think this is a WONTFIX.

Thanks!