1372096 - 2.11% tresize (windows7-32) regression on push c76e85ca482fc4320b104a098e0c5fdf7da3e631 (Fri Jun 9 2017)

Status: RESOLVED WONTFIX

Description

[Ionuț Goldan [:igoldan], Performance Sheriff]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=c76e85ca482fc4320b104a098e0c5fdf7da3e631

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  2%  tresize windows7-32 opt e10s     12.44 -> 12.70

Improvements:

 34%  tp5n nonmain_startup_fileio windows7-32 opt e10s     42,326.54 -> 27,747.83


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=7201

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan], Performance Sheriff]

:florian Thank you for the big improvement on tp5n! Looks like the multi-bug patch brought a very noticeable boost.

Regarding the small 2% regression: is this something we can fix or should we accept it?

Comment 2

[Marco Bonardo [::mak]]

It's likely that some work that was happening before the tresize measurement bracket, now happens inside it

Comment 3

[Ionuț Goldan [:igoldan], Performance Sheriff]

The previous 944c5cc643a6 changeset from:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,4ac681a39a4caefb56468c5bc86fa23b8cee4c4f,1,1%5D&zoom=1496963283752.1367,1497051335033.4758,11.894052044609666,13&selected=%5Bmozilla-inbound,4ac681a39a4caefb56468c5bc86fa23b8cee4c4f,214425,284592041,1%5D
doesn't hint any regression. I did some extra retriggers for the preceding changesets, to make sure.
Also, this patch landed immediatelly after 944c5cc643a6.

Comment 4

[Florian Quèze [:florian]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1)
> :florian Thank you for the big improvement on tp5n! Looks like the multi-bug
> patch brought a very noticeable boost.

:-)

> Regarding the small 2% regression: is this something we can fix or should we
> accept it?

I discussed this with mconley yesterday, and we agreed that investigating this is too much effort given how small the regression is, and how likely it is to be a false positive (as the patches blamed here are not expected to have anything to do with window resizing).

Talos regression bugs would be significantly more actionable if we could effortlessly have before/after profiles to look into.