Closed Bug 1285532 Opened 3 years ago Closed 3 years ago

2.17% tart(windowsxp), tp5o (win8/linux64) regression on push 6041ca217693 (Thu Jul 7 2016)

Categories

(Core :: Widget: Win32, defect)

50 Branch
All
Windows
defect
Not set

Tracking

()

RESOLVED FIXED
Tracking Status
firefox50 --- fixed

People

(Reporter: jmaher, Unassigned)

References

Details

(Keywords: perf, regression, talos-regression, Whiteboard: [fixed by patch backout from bug #1012752])

Talos has detected a Firefox performance regression from push 6041ca217693:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=015a827edf567ce10c866c655aa6d7759d5871f7&tochange=6041ca2176939905e0d55e6dea4f98e2388bfdf3

As author of one of the patches included in that push, we need your help to address this regression.

Summary of tests that improved:

  tart summary windowsxp opt e10s - 2.17% regression


You can find links to graphs and comparsion views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=1765

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
I have done a series of retriggers here:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,3152f1677c8483994da80273e522cebbf01e6177,1,1%5D&series=%5Bfx-team,3152f1677c8483994da80273e522cebbf01e6177,1,1%5D&zoom=1467902456037.8499,1467922312248.2776,5.884471406438239,6.174847956602947&selected=%5Bmozilla-inbound,3152f1677c8483994da80273e522cebbf01e6177,34190,31426784,1%5D

I have pretty high confidence in this, I am not sure if there are other issues, I have done retriggers on other jobs just to be sure.

:mstange, can you take a look at this and determine what is the root cause?  if you would like, I can do a few pushes to try for you and bisect.
Component: Untriaged → Widget: Cocoa
Flags: needinfo?(mstange)
Product: Firefox → Core
Removing bug 1283637 and bug 1280546 from the blocking list, as those were isolated changes to Cocoa code only. Looks like bug 1012752 was the culprit here.
No longer blocks: 1280546, 1283637
Component: Widget: Cocoa → Widget: Win32
OS: Unspecified → Windows
Hardware: Unspecified → All
This is unexpected. The alert also shows a much more serious regression in tp5o on Windows 8 - are we tracking that separately? I don't see it mentioned in this bug.

Are the e10s tests also showing a regression?

I pushed to try with profiling enabled:
before: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b4b02dc1aa10
after: https://treeherder.mozilla.org/#/jobs?repo=try&revision=70b51b9a75e0
Flags: needinfo?(mstange) → needinfo?(jmaher)
Yes, tp5o on Windows 8 also have a regression, we will track all regression issues which were caused by the same push in this bug. 
 
And e10s test also shows a regression, the tart summary windowsxp opt e10s alert shows on https://treeherder.mozilla.org/perf.html#/alerts?id=1763

Besides, tp5o summary test in linux64 opt also has the regression issue:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,954c0eef0296505ed478961111939345b4e83960,0,1%5D&series=%5Bfx-team,954c0eef0296505ed478961111939345b4e83960,1,1%5D&zoom=1467841448748.8457,1467993359097.8762,330,367.0149253731343&selected=%5Bfx-team,954c0eef0296505ed478961111939345b4e83960,12647,10413289,1%5D
thanks Alison for updating this bug- I have changed the title.  :mstange, do you need anything from me to help narrow this down or collect me data?
Flags: needinfo?(jmaher) → needinfo?(mstange)
Summary: 2.17% tart (windowsxp) regression on push 6041ca217693 (Thu Jul 7 2016) → 2.17% tart(windowsxp), tp5o (win8/linux64) regression on push 6041ca217693 (Thu Jul 7 2016)
Thanks. I have backed out the offending patch for now.
The patch also caused bug 1286674, which I can reproduce locally, and it may well be responsible for the perf regression. So I don't need any more data at the moment.
Flags: needinfo?(mstange)
marking this as fixed as we backed this out- please re-open if there are needs to keep this around.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Well, the backout was backed out, so this isn't actually fixed anymore. But we can keep this fixed because I'm going to reland the backout tomorrow.
Should be fixed by backout of culprit patch

(In reply to Carsten Book [:Tomcat] from bug #1012752 comment #36)
> https://hg.mozilla.org/mozilla-central/rev/440f999f965f
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Whiteboard: [fixed by patch backout from bug #1012752]
You need to log in before you can comment on or make changes to this bug.