1363124 - 25.73% tabpaint (windows8-64) regression on push 06d436f9330d2e8c7a7d1066f560c8a872c9eb57 (Wed May 3 2017)

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push 06d436f9330d2e8c7a7d1066f560c8a872c9eb57. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 26%  tabpaint summary windows8-64 pgo e10s     43.63 -> 54.86


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=6430

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

please note that this is windows 8 PGO only, so we do not see any regression on opt.  We did verify that backing out this on try does improve the test:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=20a1593cf6cd571f417f214757bbf1e137746c13&filter-searchStr=talos

the problem is we don't have a lot of control over our PGO process, so I don't want to assume that this code changed things.

This is a large regression, on a platform we care very much about (64 bit windows e10s- main target for Firefox 57), so this will get attention.

:kmag, as the patch author, can you think of reasons that tabpaint would regression so much on win8 e10s only with your changes?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

this seems to be fixed as of:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=d420703275427bacc2585f3fd614d1453a5d444b&tochange=056dff7e2ec58d9b979da1c03ce8aeba75eee1bd

given that this is not the bug that caused the regression, I believe we should mark this as invalid.

Comment 3

[Timothy Nikkel (:tnikkel)]

(In reply to Joel Maher ( :jmaher) from comment #2)
> this seems to be fixed as of:
> https://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?fromchange=d420703275427bacc2585f3fd614d1453a5d444b&tochange=056d
> ff7e2ec58d9b979da1c03ce8aeba75eee1bd
> 
> given that this is not the bug that caused the regression, I believe we
> should mark this as invalid.

The things in that pushlog seem unlikely to affect a tabpaint regression and that pushlog range is pretty close to the backout of bug 1353060. Are we sure on that? Bug 1353060 just got relanded, so let's be on the lookout for this happening again.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

it does line up exactly with 1353060- should we reopen this and work on fixing the pgo regression?

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Did the regression come back with the re-landing of bug 1353060? If so, yes, we should reopen this.

Comment 6

[Ionuț Goldan [:igoldan]]

Yes, it did came back. I'm reopening this to monitor the fix for it.

Comment 7

[:Gijs (he/him)]

Moving to the same component as the regressing bug.

Comment 8

[Chris Peterson [:cpeterson]]

Ionuț, is this tabpaint regression from bug 1353060 still an issue for Firefox 55? Note that bug 1369662 recently landed some test changes with altered tabpaint's baseline.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

looking at the graph, the regression was the test going bi-modal:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bmozilla-inbound,26721ba0e181e2844da3ddc2284a331ba54eefe0,1%5D&series=%5Bautoland,26721ba0e181e2844da3ddc2284a331ba54eefe0,0%5D&selected=%5Bmozilla-inbound,26721ba0e181e2844da3ddc2284a331ba54eefe0%5D


Oddly around May 26th, we went back to a single mode and we should have seen the regression go away.

Now that we have tabpaint measuring the correct thing (bug 1369662), we expect to see the numbers change for tabpaint (we need to look at windows 10 for that):
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bautoland,26721ba0e181e2844da3ddc2284a331ba54eefe0,1,1%5D&series=%5Bautoland,a9cd333dff68ce0812dc85e0657af4edfc51ebe3,1,1%5D

I am not sure if there was a fix landed around may 26th to make tabpaint better on win8-pgo; I do not see a corresonding change in the non pgo values.

My recommendation is that this is fixed and bonus if we can figure out what landed to fix this around May 26th (unfortunately retriggers will not work, nor will try pushes- we just don't have easy ways to test talos on win8 as we determined win10 was more valuable)

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to Joel Maher ( :jmaher) from comment #9)
> My recommendation is that this is fixed and bonus if we can figure out what
> landed to fix this around May 26th (unfortunately retriggers will not work,
> nor will try pushes- we just don't have easy ways to test talos on win8 as
> we determined win10 was more valuable)

Sounds good to me. Thanks!