1473903 - 6.3 - 7.77% tp5o (windows10-64, windows7-32) regression (Thu Jul 5 2018)

Status: RESOLVED WORKSFORME

(Testing :: General, defect, P3)

Description

[Ionuț Goldan [:igoldan]]

Talos has automatically detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=4312952f4b2a9d7aaf1ce6cbd2c2fb2834711f7d

Regressions:

  8%  tp5o windows10-64 pgo e10s stylo     197.36 -> 212.69
  6%  tp5o windows7-32 pgo e10s stylo      202.89 -> 215.67


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=14218

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:fubar These regressions seem to have been caused by infra changes.

I retriggered older jobs which outputed new values similar to the regression. Could you look into this and confirm my assumption?

I zoomed the regression point here [1] so you can spot it better.

[1] https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1653280,1,1&zoom=1530670253707.3499,1530829381487.5254,163.37078651685394,232.13483146067415

Comment 2

[Kendall Libby [:fubar] (he/him)]

There were no changes made to OCC on the 4th, and commits made on July 2 and 3 are unlikely culprits (either AWS specific, or vendoring robustcheckout which is everywhere and updating the relengapi URLs)

Mark/Q, were any changes made to the moonshots?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

this would be moonshot specific and something that went live later in the day on the 4th of July.

Comment 4

[Kendall Libby [:fubar] (he/him)]

Q was doing some work in bug 1473728 that may have affected this. We've reverted that, and I'll have ciduty re-image any w10 moonshot that might have picked up those changes.

Comment 5

[Mark Cornmesser [:markco]]

From what it looks like all those changes have been reverted. Are we still seeing the regression?

Comment 6

[Ionuț Goldan [:igoldan]]

(In reply to Mark Cornmesser [:markco] from comment #5)
> From what it looks like all those changes have been reverted. Are we still
> seeing the regression?

You are right: looks like the regression went away.