1616047 - 20.09% tscrollx (linux64-shippable) regression on push 503f96afd632f6e506a7bd7d8dbdae73b72a9147 (Fri February 14 2020)

Status: RESOLVED WONTFIX

(Core :: Layout, defect, P5)

Description

[Florin Strugariu [:Bebe]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=503f96afd632f6e506a7bd7d8dbdae73b72a9147

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

20% tscrollx linux64-shippable opt e10s stylo 0.54 -> 0.65

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24962

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/TestEngineering/Performance/Talos

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Florin Strugariu [:Bebe]]

Backout:

== Change summary for alert #24966 (as of Mon, 17 Feb 2020 12:47:30 GMT) ==

Improvements:

12% tscrollx linux64-shippable opt e10s stylo 0.62 -> 0.54

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24966

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

Investigating. Changes to perf numbers aren't surprised at all.
Bug 1506376 helps with responsiveness in certain cases, and thus may regress metrics which don't care about
responsiveness but only simple absolute numbers.

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

ahaa, based on https://wiki.mozilla.org/TestEngineering/Performance/Talos/Tests#tscrollx the test is totally artificial.
'layout.frame_rate': 0.
That is not a type of scrolling web pages could trigger, or anything, since it sets a pref to an unexpected value.
I'm Inclined to say WONTFIX for this.

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

...but I'm still considering if either the test should be changed, or if RefreshDriver should explicitly let one to bypass all the normal handling when
layout.frame_rate is 0.

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9310c21eee2eb15b632673525e09fefcc35ed6c3

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

Attachment: Bug 1616047, don't try to block refreshdriver if we're in ASAP mode

Comment 7

[Florin Strugariu [:Bebe]]

== Change summary for alert #24965 (as of Mon, 17 Feb 2020 12:46:02 GMT) ==

Regressions:

8% tp5o_scroll linux64-shippable opt e10s stylo 1.15 -> 1.24
5% tp5o_scroll linux64-shippable opt e10s stylo 1.15 -> 1.21

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24965

Comment 8

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #24973 (as of Tue, 18 Feb 2020 06:26:08 GMT) ==

Regressions:

4% raptor-tp6m-allrecipes-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 6,486.75 -> 6,750.83
1% raptor-tp6m-allrecipes-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 6,567.75 -> 6,646.38

Improvements:

2% raptor-motionmark-animometer-firefox linux64-shippable-qr opt 50.21 -> 51.24

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24973

Comment 9

[Ethan Tseng [:ethan]]

Olli, do you have any updates on this bug and your patch?

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

Not really, other than it being quite expected when the test doesn't test anything realistic.

Comment 11

[Sean Voisen (:svoisen)]

Deprioritizing based on comment 3 and comment 10.

Comment 12

[Olli Pettay [:smaug][[e-mail]]]

I don't think we're going to do anything here, given that the test isn't doing anything realistic.
(Doing some end-of-year needinfo clearance )

Comment 13

[Greg Mierzwinski [:sparky]]

Closing this as WONTFIX given comment #12.