1412777 - 0.18 - 3.39% tp5n main_startup_fileio / tsvgr_opacity (windows10-64, windows7-32) regression on push 6fe3d70877ed801274cc2f0da5490a8d6474d285 (Fri Oct 27 2017)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=6fe3d70877ed801274cc2f0da5490a8d6474d285

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  tsvgr_opacity summary windows10-64 pgo e10s     210.70 -> 217.85
  3%  tp5n main_startup_fileio windows7-32 pgo e10s   67,885,369.25 -> 70,021,694.42
  1%  tsvgr_opacity summary windows7-32 pgo e10s      239.30 -> 242.32
  0%  tsvgr_opacity summary windows10-64 opt e10s     228.97 -> 229.37

Improvements:

  3%  speedometer summary windows10-64 pgo e10s     40.45 -> 41.47
  2%  ts_paint_webext windows10-64 pgo e10s         566.67 -> 553.67
  2%  sessionrestore_no_auto_restore windows10-64 pgo e10s449.96 -> 439.92
  2%  tp6_google summary windows10-64 pgo e10s      405.04 -> 396.17
  2%  tp5o_webext summary windows10-64 pgo e10s     300.42 -> 294.39
  2%  sessionrestore windows10-64 pgo e10s          382.08 -> 375.33
  1%  speedometer summary windows7-32 pgo e10s      43.75 -> 44.40
  1%  sessionrestore_no_auto_restore windows10-64 opt e10s532.21 -> 526.33
  1%  tp5n main_startup_fileio windows7-32 opt e10s 67,877,638.21 -> 67,401,754.67


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=10212

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

The regressions happen only on Windows PGO builds. The equivalent OPT results don't show any real regressions, but rather improvements.

Comment 2

[(Away)]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #0)
> Talos has detected a Firefox performance regression from push:
> 
> https://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=6fe3d70877ed801274cc2f0da5490a8d6474d285
> 
> As author of one of the patches included in that push, we need your help to
> address this regression.
> 
> Regressions:
> 
>   3%  tsvgr_opacity summary windows10-64 pgo e10s     210.70 -> 217.85

I was unable to reproduce this difference locally using a VS2015 PGO build versus a VS2017 PGO, both of changeset 6fe3d70877ed. Without being able to profile this, I don't have any good ideas. :-(

>   3%  tp5n main_startup_fileio windows7-32 pgo e10s   67,885,369.25 ->
> 70,021,694.42

This is basically a restatement of the file size increase we saw earlier, so it should really be a part of bug 1412776.

Comment 3

[Ionuț Goldan [:igoldan]]

Marking this as WONTFIX, based on :dmajor's comment here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1412776#c2