1591010 - 36.53 - 38.91% ts_paint / ts_paint_webext (macosx1014-64-shippable) regression on push 31f16b9b58630abad5477ecf1c112af7218c324d (Mon October 21 2019)

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Posix OS, defect)

Description

[Marian Raiciof [:marauder]]

Talos has detected a Firefox performance regression around October 21st and it seems to be an infra change.

The regression also appears on a datapoint from the past (October 12th) on which i did several retriggers : 7073f1b5be058

Graph URL:
https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,2056951,1,1&series=mozilla-inbound,2132148,1,1&timerange=1209600&zoom=1570689228000,1571889862000,426.3333333333333,1971.6666666666667

Regressions:

39% ts_paint macosx1014-64-shippable opt e10s stylo 980.75 -> 1,362.33
37% ts_paint_webext macosx1014-64-shippable opt e10s stylo 973.50 -> 1,329.17

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=23521

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/TestEngineering/Performance/Talos

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[:dhouse]

Re https://bugzilla.mozilla.org/show_bug.cgi?id=1590783#c6
I changed log forwarding on the macos hardware on Oct 21st.

Comment 2

[Marian Raiciof [:marauder]]

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,2056951,1,1&timerange=31536000&zoom=1571203042670,1575534001771,635.4824011219873,1718.4773165014055