1304340 - 6.16% tart (windows7-32) regression on push ea326f6e7e6c (Sat Sep 17 2016)

Status: RESOLVED DUPLICATE of bug 1302727

(Firefox :: Untriaged, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push eaf5eb6f8fa. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  6%  tart summary windows7-32 opt e10s     7.49 -> 7.95


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=3295

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:billm, I see you are the patch author here can you take a look at this regression and help find the root cause.  This was landed right before we uplifted on Monday (and right after another regression, so we had missed it)

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

and I filed the bug on the downstream alert, here is the proper alert we should worry about for tracking in perfherder:
== Change summary for alert #3322 (as of September 21 2016 03:19 UTC) ==

Regressions:

 10%  tart summary windows7-32 opt e10s     7.11 -> 7.86

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=3322

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

:billm, do you have any questions about this regression?  Any luck coming up with a fix?

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Hi Joel,
I'm having some trouble understanding the magnitude of the regression here. Bug 1302727 landed shortly before my patch and it also regressed TART. I think the right comparison to make is this one:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=c6876f222943&newProject=mozilla-inbound&newRevision=ea326f6e7e6c485907fdb4e1739bcb2707f7c1bc&framework=1&showOnlyImportant=0

since revision c6876f222943 landed after bug 1302727. I do see a 3.90% regression, but the confidence is "low".

I also looked at the graph on inbound:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,5c04c558cb10f77bcf9a38249e74b2705d6b51b9,1,1%5D&zoom=1474000846534.81,1474155300541.1392,6.241635687732343,8.20446096654275

It seems like bug 1302727 took us from ~6.9 to ~7.5. Then my patch took us to ~7.9. Can you confirm this? I just want to make sure I understand.

Comment 5

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Hi Joel,
I did a try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1591bcba900b
It has my changes from bug 1279086. Then, on top of that, I backed out bug 1302727. I consistently see a tart score of ~7. That's back to the old value from before any of these patches landed. So I think the regression is entirely caused by bug 1302727. Please close this bug if you agree.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

:billm, thank you for checking on this.  I apologize that this was a randomization in your typical work flow.  It does seem as though bug 1302727 is the root cause.  As you look at the original graph on mozilla-inbound we had what looked like two regressions, even though there was a few revisions and retriggers.

I suspect bug 1302727 caused the regression, but bug 1279086 allowed it to fully be seen- fixing bug 1302727 would fix "both" changes in the graph.

Comment 7

[Gerry Chang [:gchang]]

Mark 51 fixed as bug 1302727 was fixed in 51.

Comment 8

[Andrew Overholt [:overholt]]

Setting status the same as the dupe for 52.