25.73% tabpaint (windows8-64) regression on push 06d436f9330d2e8c7a7d1066f560c8a872c9eb57 (Wed May 3 2017)

RESOLVED FIXED in Firefox 55

Status

()

defect
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: jmaher, Assigned: mconley)

Tracking

({perf, regression, talos-regression})

53 Branch
mozilla55
Unspecified
Windows 8
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr52 unaffected, firefox54 unaffected, firefox55 fixed)

Details

Talos has detected a Firefox performance regression from push 06d436f9330d2e8c7a7d1066f560c8a872c9eb57. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 26%  tabpaint summary windows8-64 pgo e10s     43.63 -> 54.86


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=6430

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
please note that this is windows 8 PGO only, so we do not see any regression on opt.  We did verify that backing out this on try does improve the test:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=20a1593cf6cd571f417f214757bbf1e137746c13&filter-searchStr=talos

the problem is we don't have a lot of control over our PGO process, so I don't want to assume that this code changed things.

This is a large regression, on a platform we care very much about (64 bit windows e10s- main target for Firefox 57), so this will get attention.

:kmag, as the patch author, can you think of reasons that tabpaint would regression so much on win8 e10s only with your changes?
Flags: needinfo?(kmaglione+bmo)
Component: Untriaged → General
OS: Unspecified → Windows 8
this seems to be fixed as of:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=d420703275427bacc2585f3fd614d1453a5d444b&tochange=056dff7e2ec58d9b979da1c03ce8aeba75eee1bd

given that this is not the bug that caused the regression, I believe we should mark this as invalid.
Status: NEW → RESOLVED
Closed: 2 years ago
Flags: needinfo?(kmaglione+bmo)
Resolution: --- → INVALID
(In reply to Joel Maher ( :jmaher) from comment #2)
> this seems to be fixed as of:
> https://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?fromchange=d420703275427bacc2585f3fd614d1453a5d444b&tochange=056d
> ff7e2ec58d9b979da1c03ce8aeba75eee1bd
> 
> given that this is not the bug that caused the regression, I believe we
> should mark this as invalid.

The things in that pushlog seem unlikely to affect a tabpaint regression and that pushlog range is pretty close to the backout of bug 1353060. Are we sure on that? Bug 1353060 just got relanded, so let's be on the lookout for this happening again.
Flags: needinfo?(jmaher)
it does line up exactly with 1353060- should we reopen this and work on fixing the pgo regression?
Flags: needinfo?(jmaher)
Did the regression come back with the re-landing of bug 1353060? If so, yes, we should reopen this.
Yes, it did came back. I'm reopening this to monitor the fix for it.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Moving to the same component as the regressing bug.
Component: General → Graphics: Layers
Product: Firefox → Core
Ionuț, is this tabpaint regression from bug 1353060 still an issue for Firefox 55? Note that bug 1369662 recently landed some test changes with altered tabpaint's baseline.
Flags: needinfo?(ionut.goldan)
looking at the graph, the regression was the test going bi-modal:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bmozilla-inbound,26721ba0e181e2844da3ddc2284a331ba54eefe0,1%5D&series=%5Bautoland,26721ba0e181e2844da3ddc2284a331ba54eefe0,0%5D&selected=%5Bmozilla-inbound,26721ba0e181e2844da3ddc2284a331ba54eefe0%5D


Oddly around May 26th, we went back to a single mode and we should have seen the regression go away.

Now that we have tabpaint measuring the correct thing (bug 1369662), we expect to see the numbers change for tabpaint (we need to look at windows 10 for that):
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bautoland,26721ba0e181e2844da3ddc2284a331ba54eefe0,1,1%5D&series=%5Bautoland,a9cd333dff68ce0812dc85e0657af4edfc51ebe3,1,1%5D

I am not sure if there was a fix landed around may 26th to make tabpaint better on win8-pgo; I do not see a corresonding change in the non pgo values.

My recommendation is that this is fixed and bonus if we can figure out what landed to fix this around May 26th (unfortunately retriggers will not work, nor will try pushes- we just don't have easy ways to test talos on win8 as we determined win10 was more valuable)
Flags: needinfo?(ionut.goldan)
(In reply to Joel Maher ( :jmaher) from comment #9)
> My recommendation is that this is fixed and bonus if we can figure out what
> landed to fix this around May 26th (unfortunately retriggers will not work,
> nor will try pushes- we just don't have easy ways to test talos on win8 as
> we determined win10 was more valuable)

Sounds good to me. Thanks!
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Assignee: nobody → mconley
Target Milestone: --- → mozilla55
You need to log in before you can comment on or make changes to this bug.