1382638 - 3.87 - 8.89% a11yr / damp (macosx64-devedition, windows7-32) regression on push c9fc3cee2393 (Tue Jul 18 2017)

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=9d89b33a35a3ae9e252a43038fa649180bf07491

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  9%  a11yr summary macosx64-devedition opt e10s     484.39 -> 527.46
  6%  a11yr summary windows7-32 pgo e10s             442.43 -> 467.30
  4%  damp summary windows7-32 pgo e10s              200.26 -> 208.84
  4%  damp summary windows7-32 opt e10s              258.70 -> 268.71


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=8078

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

Mike, the Talos changes somehow affected at least these 2 tests. Were they expected? Or can we continue with them from now on?

Comment 2

[Ionuț Goldan [:igoldan]]

The description above doesn't hint to the right changeset. This is the correct one: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=7d43eed8af31c61d2c6e56b9b4882ef59631a91f&tochange=c9fc3cee23936687c84a9cc4120fe05c5fe972cf

Comment 3

[Mike Conley (:mconley) (:⚙️)]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> Mike, the Talos changes somehow affected at least these 2 tests. Were they
> expected? Or can we continue with them from now on?

Unsure why this affected a11yr like this, but since it was just a test change, I suggest we re-baseline and move on.