1419766 - 2.41 - 9.49% damp (linux64, osx-10-10, windows10-64, windows7-32) regression on push 3742c29706ebbdf72013b49f22ddc35d1e8c3c04 (Tue Nov 21 2017)

Status: RESOLVED WONTFIX

(DevTools :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=3742c29706ebbdf72013b49f22ddc35d1e8c3c04

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  9%  damp summary windows7-32 opt e10s     289.69 -> 317.19
  6%  damp summary linux64 pgo e10s         247.43 -> 263.15
  6%  damp summary linux64 opt e10s         274.25 -> 290.60
  4%  damp summary osx-10-10 opt e10s       312.72 -> 324.34
  2%  damp summary windows10-64 opt e10s    276.15 -> 282.79


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=10685

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:ochameau Most likely, these regressions are caused by the new panelsInBackground.reload DAMP test added by this changeset:
https://hg.mozilla.org/integration/autoland/rev/7eae6691b2f9
Before resolving this as WONTFIX, I just need you to confirm this.

Comment 2

[Alexandre Poirot [:ochameau]]

I think it is again because of a new subtest added:
  damp panelsInBackground.reload
Which increase the summary by its own value and bumps various subsequent tests as well.

I'll try to confirm by reverting only the subtest.

Comment 3

[Alexandre Poirot [:ochameau]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> :ochameau Most likely, these regressions are caused by the new
> panelsInBackground.reload DAMP test added by this changeset:
> https://hg.mozilla.org/integration/autoland/rev/7eae6691b2f9
> Before resolving this as WONTFIX, I just need you to confirm this.

If I revert the test, I got no significant difference on DAMP:
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-central&newProject=try&newRevision=cbf82e5d51b01f8b1b6c61583b09fc758022ed9f&originalSignature=edaec66500db21d37602c99daa61ac983f21a6ac&newSignature=edaec66500db21d37602c99daa61ac983f21a6ac&showOnlyImportant=1&framework=1&selectedTimeRange=172800
There is something reported on netmonitor, but this is because of recent landing which improved that and that I don't have in this push.

So, yes, this is only because of the test.