Closed Bug 1381533 Opened 7 years ago Closed 6 years ago

5.07 - 11.07% tcanvasmark / tresize / tsvgr_opacity / tsvgx (windows7-32) regression on push a16d92beb70daa33c4e29bf25300ca259ed61b2a (Tue Jul 11 2017)


(Release Engineering :: General, defect)

Windows 7
Not set


(Not tracked)



(Reporter: igoldan, Unassigned)



(Keywords: perf, regression, talos-regression)

Talos has detected a Firefox performance regression from push:

As author of one of the patches included in that push, we need your help to address this regression.


 11%  tcanvasmark summary windows7-32 pgo e10s     9,537.92 -> 8,481.67
  8%  tresize windows7-32 pgo e10s                 10.83 -> 11.68
  7%  tsvgx summary windows7-32 pgo e10s           436.51 -> 467.25
  5%  tsvgr_opacity summary windows7-32 pgo e10s   349.54 -> 367.25

You can find links to graphs and comparison views for each of the above tests at:

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see:

For information on reproducing and debugging the regression, either on try or locally, see:

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations:
these tests became very noisy, the regression is the median value of the increased noise.
Component: Untriaged → General
Product: Firefox → Release Engineering
looking in detail, I am not able to see any regression in opt, only pgo.  Likewise no difference in win10.

2 of the 4 regressions show a clear change on:

and the other 2 show it 2 pushes later:

Given the 20 retriggers we have for many of the pushes in the general range of this, I am going to suggest that we are dealing with odd pgo fallout, but that is a theory.

:ted, do you have thoughts here on how pgo or build state could be affecting the perf results to become so noisy on pgo only as of Monday July 10th (1 week ago) ?
Flags: needinfo?(ted)
the noise is getting "better" around July 19th- magic numbers
I see these improvements so far:
== Change summary for alert #8096 (as of July 19 2017 08:04 UTC) ==


 16%  tcanvasmark summary windows7-32 pgo e10s     8,338.29 -> 9,688.46
  8%  tsvgx summary windows7-32 pgo e10s           470.36 -> 430.86
  7%  tresize windows7-32 pgo e10s                 12.07 -> 11.18

For up to date results, see:
possibly we can resolve this as fixed?  :igoldan, any thoughts on that?
Flags: needinfo?(ionut.goldan)
Yes, I agree with closing this as RESOLVED.

But I would file another bug for tresize, because there's a visible regression between July 18 and 19th:,a14119c4d02daaf55113e3945c4385f4c927da27,1,1%5D&series=%5Bautoland,4ac681a39a4caefb56468c5bc86fa23b8cee4c4f,1,1%5D
It happens on both opt and pgo.

On the 1st look, it seems it's just a typical, separate regression than this one.
I'll do some retriggers and backfills, to properly identify it.
Flags: needinfo?(ionut.goldan)
Closed: 6 years ago
Resolution: --- → FIXED
Flags: needinfo?(ted)
You need to log in before you can comment on or make changes to this bug.