1503490 - 6.14 - 10.12% sessionrestore_no_auto_restore (linux64, linux64-qr, windows10-64, windows10-64-qr, windows7-32) regression on push 4eaa5f13d23184e6f8b2c22d4cbe31e98f5b630d (Mon Oct 29 2018)

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=4eaa5f13d23184e6f8b2c22d4cbe31e98f5b630d

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 10%  sessionrestore_no_auto_restore windows10-64 pgo e10s stylo        338.96 -> 373.25
  8%  sessionrestore_no_auto_restore windows7-32 pgo e10s stylo         340.92 -> 369.58
  8%  sessionrestore_no_auto_restore windows10-64 opt e10s stylo        367.75 -> 397.00
  8%  sessionrestore_no_auto_restore windows7-32 opt e10s stylo         366.29 -> 394.25
  7%  sessionrestore_no_auto_restore windows10-64-qr opt e10s stylo     380.46 -> 406.00
  6%  sessionrestore_no_auto_restore linux64 pgo e10s stylo             390.33 -> 415.25
  6%  sessionrestore_no_auto_restore linux64-qr opt e10s stylo          893.25 -> 948.08


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=17236

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:aswan Unless there's something fishy with this new baseline, I want to mark this as WONTFIX.

Comment 2

[Andrew Swan [:aswan]]

To be honest, I don't really understand this, the patch in bug 1495638 should not have affected how this test is measured.

Something is suspicious with the test generally though, baseline measurements for sessionrestore_no_auto_restore are higher than vanilla sessionrestore measurements, which makes no sense.

In any case, I agree with marking it WONTFIX.