Talos has detected a Firefox performance regression from push eaf5eb6f8fa. As author of one of the patches included in that push, we need your help to address this regression. Regressions: 6% tart summary windows7-32 opt e10s 7.49 -> 7.95 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=3295 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
:billm, I see you are the patch author here can you take a look at this regression and help find the root cause. This was landed right before we uplifted on Monday (and right after another regression, so we had missed it)
Summary: 6.16% tart (windows7-32) regression on push eaf5eb6f8fa (Sat Sep 17 2016) → 6.16% tart (windows7-32) regression on push ea326f6e7e6c (Sat Sep 17 2016)
and I filed the bug on the downstream alert, here is the proper alert we should worry about for tracking in perfherder: == Change summary for alert #3322 (as of September 21 2016 03:19 UTC) == Regressions: 10% tart summary windows7-32 opt e10s 7.11 -> 7.86 For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=3322
a year ago
Assignee: nobody → wmccloskey
status-firefox49: --- → unaffected
status-firefox50: --- → unaffected
status-firefox51: --- → affected
status-firefox52: --- → affected
:billm, do you have any questions about this regression? Any luck coming up with a fix?
Hi Joel, I'm having some trouble understanding the magnitude of the regression here. Bug 1302727 landed shortly before my patch and it also regressed TART. I think the right comparison to make is this one: https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=c6876f222943&newProject=mozilla-inbound&newRevision=ea326f6e7e6c485907fdb4e1739bcb2707f7c1bc&framework=1&showOnlyImportant=0 since revision c6876f222943 landed after bug 1302727. I do see a 3.90% regression, but the confidence is "low". I also looked at the graph on inbound: https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,5c04c558cb10f77bcf9a38249e74b2705d6b51b9,1,1%5D&zoom=1474000846534.81,1474155300541.1392,6.241635687732343,8.20446096654275 It seems like bug 1302727 took us from ~6.9 to ~7.5. Then my patch took us to ~7.9. Can you confirm this? I just want to make sure I understand.
Flags: needinfo?(wmccloskey) → needinfo?(jmaher)
Hi Joel, I did a try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=1591bcba900b It has my changes from bug 1279086. Then, on top of that, I backed out bug 1302727. I consistently see a tart score of ~7. That's back to the old value from before any of these patches landed. So I think the regression is entirely caused by bug 1302727. Please close this bug if you agree.
:billm, thank you for checking on this. I apologize that this was a randomization in your typical work flow. It does seem as though bug 1302727 is the root cause. As you look at the original graph on mozilla-inbound we had what looked like two regressions, even though there was a few revisions and retriggers. I suspect bug 1302727 caused the regression, but bug 1279086 allowed it to fully be seen- fixing bug 1302727 would fix "both" changes in the graph.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1302727
Mark 51 fixed as bug 1302727 was fixed in 51.
status-firefox51: affected → fixed
Setting status the same as the dupe for 52.
status-firefox52: affected → fixed
You need to log in before you can comment on or make changes to this bug.