1320875 - 1.91% tsvgr_opacity (osx-10-10) regression on push ee43b0af2a45d0bd9096f06e4aa5ea5c792c2ddc (Thu Nov 24 2016)

Status: RESOLVED WONTFIX

(Firefox :: Untriaged, defect)

Description

[Alison Shiue]

Talos has detected a Firefox performance regression from push ee43b0af2a45d0bd9096f06e4aa5ea5c792c2ddc. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  2%  tsvgr_opacity summary osx-10-10 opt e10s     354.52 -> 361.3


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=4378

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Alison Shiue]

I did bisect for this issue, here is the result:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&fromchange=7c1227a36dea

According to perf graph, we can find the regression shows on 63402e747d5e (update to 061d9d4d64ca)
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Btry,e36d4e6025ecbe2c591d4347294065bc90c453cf,1,1%5D&zoom=1480400291699.1106,1480401504000,347.58626884647805,371.9043407360318&selected=%5Btry,e36d4e6025ecbe2c591d4347294065bc90c453cf,148877,32014526,1%5D

Here is the result of comparing with previous patch: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=94acf4f7e372258eacba2598e77999db371dc8be&newProject=try&newRevision=63402e747d5e9485a876f6b79805861b9d026a62&framework=1&showOnlyImportant=0

Hi André, as you are the patch author, can you please take a look at this and determine what is the root cause? Thanks!

Comment 2

[André Bargull [:anba]]

I'm not sure how bug 1319465 could possibly regress tsvgr_opacity, because I don't actually find any callers to String.prototype.normalize in the tree (except for test files). I did find bug 1174792 and bug 1250350 which indicate that the SVG Talos tests are flaky on e10s, maybe this regression is related to those bugs?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I did some try pushes and see that the backout of the patch from bug 1319465 caused a regression (implying that it originally caused a perf win):
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&originalRevision=a7af8db58cd2&newProject=try&newRevision=b40781ab8a682028acb90eb413866beb01701ae0&originalSignature=e36d4e6025ecbe2c591d4347294065bc90c453cf&newSignature=e36d4e6025ecbe2c591d4347294065bc90c453cf&framework=1

this is the same data I see on the tree with the original landing as well as the link above in the original comment with the same try push that Alison did.

I am getting confused as we are seeing the same regression when looking at this patch, although my push showed it reversed.  Maybe Alison can take a second look at this so we can get more clarity?

Comment 4

[Alison Shiue]

I did the try push again, 061d9d4d64ca vs backout 061d9d4d64ca, and the backout push shows an improvement: 
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&originalRevision=fdf9137eb6e0e4db03b8efa39d2790a4dadb9535&newProject=try&newRevision=94bf53a93f445d2194dbb3119eaa0c9178d1f281&originalSignature=e36d4e6025ecbe2c591d4347294065bc90c453cf&newSignature=e36d4e6025ecbe2c591d4347294065bc90c453cf&framework=1

It looks like the only different between our pushes is the different code base (my code base is 061d9d4d64ca), do you think it might be the problem?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

this is really confusing.  Alison, I believe you did this:
* hg update 061d9d4d64ca; push to try
* hg update 061d9d4d64ca; hg backout 061d9d4d64ca; push to try

and you see an improvement- as your original try push showed and the original regression showed.  As you indicated could the difference be in the base revision we are starting from.  I think that is the key here- something else has landed since revision 061d9d4d64ca which plays with this test and causes this patch to show a regression.

Sadly, I think this ends up making this bug/regresssion not very actionable- possible we marks this as wontfix?

Comment 6

[Alison Shiue]

OK, I am okay to mark this bug as wontfix.