2.44 - 4.14% damp / tps (windows7-32) regression on push 0efb2a2aff2791f08e19eb5f13d6167a8fd44fe0 (Tue Aug 30 2016)

RESOLVED WORKSFORME

Status

()

RESOLVED WORKSFORME
2 years ago
2 years ago

People

(Reporter: ashiue, Unassigned)

Tracking

({perf, regression, talos-regression})

51 Branch
perf, regression, talos-regression
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox48 unaffected, firefox49 unaffected, firefox50 unaffected, firefox51 affected)

Details

(Reporter)

Description

2 years ago
Talos has detected a Firefox performance regression from push 0efb2a2aff2791f08e19eb5f13d6167a8fd44fe0. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  4%  tps summary windows7-32 pgo           54.38 -> 56.63
  2%  damp summary windows7-32 pgo e10s     199.02 -> 203.87


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=2936

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
(Reporter)

Comment 1

2 years ago
After doing some retriggers, this issue might be caused by the patch:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e11d9697af59672994c6cdc309c97564078b7dc1&tochange=0efb2a2aff2791f08e19eb5f13d6167a8fd44fe0

Hi JW, as you are the patch author, can you take a look at this and determine what is the root cause? Thanks!
Blocks: 1291351, 1299019
Flags: needinfo?(jwwang)
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=64666d844f3c&newProject=try&newRevision=e86d7507ca63&framework=1&showOnlyImportant=0

e86d7507ca63 includes the backout of bug 1299019 and shows no significant improvements.

tps opt
linux64      63.30   >   62.80   -0.79%  
osx-10-10    75.77   <   76.83   1.40%   
windows7-32  70.61   <   71.47   1.22%   

tps opt e10s
linux64      46.26   >   45.41   -1.84%  
osx-10-10    38.42   <   39.51   2.84%   
windows7-32  40.83   <   41.62   1.93%

In fact, it is even worse on osx-10-10 and windows7-32.

I think it is just a false alarm.
Flags: needinfo?(jwwang)
status-firefox48: --- → unaffected
status-firefox49: --- → unaffected
status-firefox50: --- → unaffected
I did some retriggers and it showed no difference.  This makes a lot of sense since non pgo builds (i.e. opt builds which we don't ship to users) showed no difference.  It would be better to push to try with a pgo build:
https://wiki.mozilla.org/ReleaseEngineering/TryChooser#What_if_I_want_PGO_for_my_build

when you do push (with the hack for pgo), you could do:
try -b o -p win32 -u none -t g2 --rebuild-talos 5

that will run just the test in question and give it enough data to be meaningful.
ok, with data on the backout, I am not seeing any real wins- most likely this is a red herring- we can close as worksforme.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.