Closed Bug 1340873 Opened 3 years ago Closed 3 years ago

2.01 - 3.03% sessionrestore / sessionrestore_no_auto_restore / ts_paint (linux64, osx-10-10, windows8-64) regression on push 29f4d23a9e97c324b40c63c71eff04add5fd2a09 (Thu Feb 16 2017)

Categories

(Firefox Build System :: General, defect)

53 Branch
defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jmaher, Assigned: catlee)

References

Details

(Keywords: perf, regression, talos-regression)

Talos has detected a Firefox performance regression from push 29f4d23a9e97c324b40c63c71eff04add5fd2a09. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  ts_paint windows8-64 pgo                            808.92 -> 833.42
  3%  ts_paint linux64 pgo                                1139.75 -> 1172.08
  3%  sessionrestore_no_auto_restore windows8-64 pgo      719.08 -> 739.08
  3%  ts_paint windows8-64 pgo e10s                       802.17 -> 824
  3%  ts_paint osx-10-10 opt                              1191.25 -> 1222.75
  3%  sessionrestore windows8-64 pgo                      696.04 -> 713.83
  2%  ts_paint windows8-64 opt e10s                       951.08 -> 974
  2%  ts_paint linux64 opt                                1333.33 -> 1364.42
  2%  ts_paint linux64 opt e10s                           1237.75 -> 1266.33
  2%  ts_paint windows8-64 opt                            960.5 -> 981.58
  2%  sessionrestore_no_auto_restore windows8-64 opt      862.42 -> 880.42
  2%  sessionrestore windows8-64 opt                      839.04 -> 856
  2%  ts_paint linux64 pgo e10s                           1074.88 -> 1096.5

Improvements:

 12%  tp5n main_startup_fileio windows7-32 pgo          70278447.25 -> 61793715.08
 12%  tp5n main_startup_fileio windows7-32 pgo e10s     70382523.58 -> 61987080.42
  5%  tp5o Main_RSS linux64 pgo e10s                    170999472.55 -> 163160602.48
  4%  tp5o Main_RSS linux64 opt e10s                    173635785.43 -> 166856555
  3%  tp5o Main_RSS linux64 pgo                         249584310.97 -> 241745580.81
  3%  tp5o Main_RSS linux64 opt                         245844351.87 -> 238333579.64
  2%  tp5o Main_RSS osx-10-10 opt                       397510251.14 -> 388817320.26


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=5164

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
here is a comparison view of this commit and the previous one:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=autoland&originalRevision=e76a72dceeea&newProject=autoland&newRevision=29f4d23a9e97c324b40c63c71eff04add5fd2a09&framework=1

it shows the same regressions as the alerts found.

:catlee, as the patch author here, can you figure out why this is happening and what we should do to resolve this?  Either fix it completely, fix it partially, back out the patch, or accept the regression.
Flags: needinfo?(catlee)
This is expected due to backing out the omni.ja decompression patch in bug 1340157.

We'd like to measure the effect again on Talos and Telemetry before making a final decision of whether to enable it on all branches, or keep it disabled.
Flags: needinfo?(catlee)
what is the timeline for measuring this and marking this as wontfix or reverting these changes?
Flags: needinfo?(catlee)
I'd like to make a decision by the end of this week. Telemetry data is still coming in.
Flags: needinfo?(catlee)
Component: Untriaged → Build Config
Assignee: nobody → catlee
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX
Component: Build Config → General
Product: Firefox → Firefox Build System
You need to log in before you can comment on or make changes to this bug.