1591921 - 2.45 - 11.56% raptor-tp6-instagram-firefox-cold fcp / raptor-tp6m-bing-restaurants-geckoview-cold fcp loadtime / raptor-tp6m-booking-geckoview-cold fcp / raptor-tp6m-google-maps-geckoview-cold / fcp / raptor-tp6m-google-restaurants-geckoview-cold / fcp /

Status: RESOLVED FIXED

(Testing :: Performance, defect)

Description

[Florin Strugariu [:Bebe]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=ce39fc80bb5b9060bab9a6ffe3774fbf7153f490

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

12% raptor-tp6m-wikipedia-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 1,121.29 -> 1,250.88
10% raptor-tp6m-wikipedia-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 906.83 -> 994.92
10% raptor-tp6m-wikipedia-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 1,119.96 -> 1,228.50
9% raptor-tp6m-instagram-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 529.75 -> 577.88
8% raptor-tp6m-google-restaurants-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 637.89 -> 689.04
7% raptor-tp6m-bing-restaurants-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 602.98 -> 647.94
7% raptor-tp6m-google-restaurants-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 710.04 -> 760.12
7% raptor-tp6m-wikipedia-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 935.00 -> 1,000.46
7% raptor-tp6m-booking-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 629.50 -> 673.50
7% raptor-tp6m-bing-restaurants-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 643.75 -> 688.00
7% raptor-tp6m-instagram-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 912.54 -> 972.18
6% raptor-tp6m-bing-restaurants-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 730.17 -> 773.42
4% raptor-tp6m-google-maps-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 1,238.83 -> 1,293.81
4% raptor-tp6m-google-maps-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 1,213.71 -> 1,267.25
2% raptor-tp6-instagram-firefox-cold fcp linux64-shippable opt 188.88 -> 193.50

Improvements:

36% raptor-tp6-twitch-firefox-cold fcp linux64-shippable-qr opt 117.50 -> 75.17
12% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable opt 2,325.42 -> 2,039.58
10% raptor-tp6-tumblr-firefox-cold loadtime windows10-64-shippable opt 1,843.75 -> 1,668.50
9% raptor-tp6-yahoo-mail-firefox-cold loadtime windows7-32-shippable opt 2,668.88 -> 2,431.25
7% raptor-tp6-twitch-firefox-cold fcp windows10-64-shippable-qr opt 136.25 -> 126.17
2% raptor-tp6-yahoo-mail-firefox-cold fcp windows7-32-shippable opt 289.92 -> 283.75
2% raptor-tp6-pinterest-firefox-cold loadtime windows10-64-shippable opt 1,948.08 -> 1,908.75

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=23576

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Florin Strugariu [:Bebe]]

tcampbell this regression is in one of the bugs:

Bug 1591209 - Add JSFunction::baseScript()
Bug 1589904 - Remove LazyScript/JSScript::functionDelazifying and friends
Bug 1590176 - Relazified scripts should not revive without updating canonical function

As all of the bugs are in the same commit assigned to you can you take a look and identify the correct culprit?

Comment 2

[Jan de Mooij [:jandem]]

There's also a pretty big AWSY regression with a similar regression range: https://treeherder.mozilla.org/perf.html#/alerts?id=23560&hideDwnToInv=0

Comment 3

[Ted Campbell [:tcampbell]]

I'll try to bisect on the AWSY regression as it is a more reliable test

Comment 4

[Ted Campbell [:tcampbell]]

The regression is in Bug 1591209 which is the top of the stack. I'll request a backout of just that patch. I'm not sure what is going on, but the results are clear in the AWSY jobs.

Comment 5

[Ted Campbell [:tcampbell]]

This is fixed by Backout: https://hg.mozilla.org/integration/autoland/rev/88d98643f874655565c32847817b487f0c6509ba

The numbers have popped back to what they were before. The AWSY regression was the most useful for tracking this. It would be nice to understand why things like twitch sped up so much, but I first need to understand how the relatively simple patch is causing increased memory usage.

Comment 6

[Ted Campbell [:tcampbell]]

I've opened Bug 1592356 to address the root cause.

Comment 7

[Florin Strugariu [:Bebe]]

Back out alerts:

== Change summary for alert #23638 (as of Thu, 31 Oct 2019 07:18:28 GMT) ==

Regressions:

55% raptor-tp6-twitch-firefox-cold fcp linux64-shippable-qr opt 74.88 -> 116.25
55% raptor-tp6-twitch-firefox-cold fcp linux64-shippable-qr opt 74.75 -> 115.58
17% raptor-tp6-yahoo-mail-firefox-cold loadtime windows7-32-shippable opt 2,304.33 -> 2,693.83
12% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable opt 2,057.54 -> 2,314.08
12% raptor-tp6-twitch-firefox-cold linux64-shippable-qr opt 226.17 -> 254.27
12% raptor-tp6-tumblr-firefox-cold loadtime windows10-64-shippable opt 1,667.25 -> 1,872.33
12% raptor-tp6-tumblr-firefox-cold loadtime windows10-64-shippable opt 1,679.96 -> 1,879.58
5% raptor-tp6-yahoo-mail-firefox-cold windows7-32-shippable opt 610.53 -> 640.42
3% raptor-tp6-slides-firefox-cold windows10-64-shippable opt 1,239.48 -> 1,277.31
2% raptor-tp6-netflix-firefox-cold fcp linux64-shippable-qr opt 607.88 -> 622.92
2% raptor-tp6-yahoo-mail-firefox-cold fcp linux64-shippable opt 280.88 -> 287.08
2% raptor-tp6-yahoo-mail-firefox-cold fcp linux64-shippable opt 281.33 -> 285.92

Improvements:

10% raptor-tp6m-wikipedia-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 1,232.29 -> 1,112.71
9% raptor-tp6m-google-restaurants-geckoview-cold android-hw-g5-7-0-arm7-api-16 pgo 695.41 -> 634.16
8% raptor-tp6m-instagram-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 577.08 -> 531.08
8% raptor-tp6m-google-restaurants-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 769.25 -> 708.42
8% raptor-tp6m-google-restaurants-geckoview-cold loadtime android-hw-g5-7-0-arm7-api-16 pgo 924.75 -> 853.83
7% raptor-tp6m-booking-geckoview-cold fcp android-hw-g5-7-0-arm7-api-16 pgo 670.08 -> 623.42
3% raptor-tp6-instagram-firefox-cold fcp linux64-shippable opt 193.83 -> 188.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=23638