Closed
Bug 1401127
Opened 7 years ago
Closed 7 years ago
2.23 - 61.65% tp5o_webext regressions on push aca18d5ec8efca82f3e94db8f31cff578e69fa10 (Fri Sep 15 2017)
Categories
(Testing :: Talos, defect)
Testing
Talos
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: igoldan, Unassigned)
References
Details
(Keywords: perf, regression, talos-regression)
Talos has detected a Firefox performance regression from push: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=aca18d5ec8efca82f3e94db8f31cff578e69fa10 As author of one of the patches included in that push, we need your help to address this regression. Regressions: 62% ts_paint_webext linux64 pgo e10s 854.23 -> 1,380.83 61% tp5o_webext responsiveness linux64 pgo e10s5.41 -> 8.70 61% tp5o_webext responsiveness linux64 opt e10s6.03 -> 9.69 58% tp5o_webext summary osx-10-10 opt e10s266.16 -> 421.01 52% ts_paint_webext linux64 opt e10s 944.29 -> 1,439.25 51% tp5o_webext responsiveness windows10-64 pgo e10s3.81 -> 5.75 49% tp5o_webext responsiveness windows10-64 opt e10s4.29 -> 6.39 43% tp5o_webext summary linux64 pgo e10s 251.19 -> 360.42 41% tp5o_webext summary linux64 opt e10s 297.85 -> 419.42 38% tp5o_webext summary windows10-64 opt e10s274.96 -> 380.54 38% tp5o_webext summary windows7-32 pgo e10s219.18 -> 301.83 37% tp5o_webext summary windows10-64 pgo e10s223.99 -> 307.72 37% tp5o_webext responsiveness windows7-32 opt e10s4.70 -> 6.45 36% tp5o_webext responsiveness windows7-32 pgo e10s3.77 -> 5.13 34% tp5o_webext summary windows7-32 opt e10s304.69 -> 407.77 20% tp5o_webext % Processor Time windows7-32 opt e10s56.41 -> 67.43 18% tp5o_webext Private Bytes windows7-32 pgo e10s152,425,778.63 -> 180,204,989.92 15% tp5o_webext Private Bytes windows7-32 opt e10s154,287,366.88 -> 177,443,498.39 14% tp5o_webext Main_RSS linux64 pgo e10s187,598,612.19 -> 213,456,847.05 13% tp5o_webext Main_RSS linux64 opt e10s194,286,897.76 -> 219,431,603.76 12% tp5o_webext Main_RSS osx-10-10 opt e10s352,973,998.14 -> 396,425,659.79 11% tp5o_webext Main_RSS windows7-32 pgo e10s128,619,324.46 -> 143,204,995.82 9% tp5o_webext Main_RSS windows7-32 opt e10s133,500,824.08 -> 145,165,425.15 4% tp5o_webext Private Bytes linux64 pgo e10s1,018,622,305.81 -> 1,056,033,445.52 2% tp5o_webext Private Bytes linux64 opt e10s1,019,742,992.52 -> 1,042,473,129.94 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=9471 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
Reporter | ||
Updated•7 years ago
|
Component: Untriaged → Talos
Product: Firefox → Testing
Reporter | ||
Comment 1•7 years ago
|
||
To my understading, these should represent the new baseline for our tp5o_webext tests. Am I right? As you noted here, Kris: https://bugzilla.mozilla.org/show_bug.cgi?id=1381961#c10 many tests here have come back to original baseline after the merge with bug 1381961, while the rest went down partially. Unless you find something unexpected in the regression list above, I would resolve this as WONTFIX.
Flags: needinfo?(kmaglione+bmo)
Comment 2•7 years ago
|
||
some of the values end up being reset fully or partially with bug 1381961 - maybe that is fixing some of the issues here?
Comment 3•7 years ago
|
||
(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1) > To my understading, these should represent the new baseline for our > tp5o_webext tests. Am I right? Yes, this is the new baseline. The changes represent intentional extra overhead added to the talos tests so that we can measure the actual performance impact of more of our code, rather than an actual performance regression. (In reply to Joel Maher ( :jmaher) (UTC-5) from comment #2) > some of the values end up being reset fully or partially with bug 1381961 - > maybe that is fixing some of the issues here? The interaction with bug 1381961 is complicated. Those changes improved baseline performance in a lot of areas considerably. So the new baseline with the more complicated talos webextension is similar to the old overhead with the trivial extension, but is still worse than the baseline without the extension. That said, bug 1381961 also disproportionately affected a lot of WebExtension framework code, so the gains in the webext tests are more than just the baseline gains. This is why I wanted to land these talos changes before bug 1381961 landed, so we get a better picture of the impact it had.
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(kmaglione+bmo)
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•