1616041 - 196.11 - 223% tp5n main_normal_fileio (windows10-64-shippable, windows10-64-shippable-qr) regression on push bd02e39a0b42472b4ebb3a84d220d0aecce1074c (Thu February 13 2020)

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Florin Strugariu [:Bebe]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=95e06979aeba8087551851bb0b25da6de2ec055f

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

223% tp5n main_normal_fileio windows10-64-shippable opt e10s stylo 909,433.08 -> 2,937,438.17
196% tp5n main_normal_fileio windows10-64-shippable-qr opt e10s stylo 936,182.42 -> 2,772,094.33
153% tp5n main_normal_fileio windows10-64-shippable-qr opt e10s stylo 1,109,984.50 -> 2,803,193.67

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24961

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/TestEngineering/Performance/Talos

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Alex Thayer [:alexical] (she/her)]

I believe this was fixed by bug 1615045

Comment 2

[Jens Stutte [:jstutte]]

Nathan, can you please triage this? Thank you!

Comment 3

[Nathan Froyd [:froydnj]]

Doug is correct, or at least has the most plausible hypothesis that I can see. The graph:

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&selected=1922690,1049569222&series=autoland,1922690,1,1&timerange=2592000&zoom=1581570924996,1581779081757,279542.24241719185,4460222.663004725

shows benchmark levels back to normal after a couple of days, and bug 1615045 is contained in the first push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=65204cac7dc47b303fe7e6234105b9609c12cef0&tochange=0ae34267ecfaec4a70c3430511e9dbe615cb402b

that has a data point at "normal" levels.