1529290 - 16.82 - 82.58% raptor-tp6m-* (android-hw-*) regression on push 3924d1ec16af9553a1574c0cc87cbd34d695111b

Status: RESOLVED WONTFIX

(Testing :: Raptor, defect)

Description

[Ionuț Goldan [:igoldan]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=8a4ad06c05759fe087625f9fd8d047d848bee4c2&tochange=3924d1ec16af9553a1574c0cc87cbd34d695111b

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

83% raptor-tp6m-youtube-geckoview loadtime android-hw-p2-8-0-arm7-api-16 opt 228.67 -> 417.50
57% raptor-tp6m-youtube-geckoview loadtime android-hw-p2-8-0-android-aarch64 opt 234.29 -> 368.38
43% raptor-tp6m-youtube-geckoview fcp android-hw-p2-8-0-arm7-api-16 opt 264.88 -> 379.33
39% raptor-tp6m-youtube-geckoview android-hw-p2-8-0-arm7-api-16 opt 232.15 -> 322.23
33% raptor-tp6m-youtube-geckoview fcp android-hw-p2-8-0-android-aarch64 opt 281.29 -> 375.42
30% raptor-tp6m-facebook-geckoview android-hw-g5-7-0-arm7-api-16 opt 1,624.28 -> 2,109.45
28% raptor-tp6m-youtube-geckoview android-hw-p2-8-0-android-aarch64 opt 247.63 -> 317.74
22% raptor-tp6m-facebook-geckoview loadtime android-hw-g5-7-0-arm7-api-16 opt 1,670.04 -> 2,043.33
17% raptor-tp6m-youtube-geckoview loadtime android-hw-g5-7-0-arm7-api-16 opt 586.71 -> 685.38

Improvements:

71% raptor-tp6m-google-geckoview loadtime android-hw-g5-7-0-arm7-api-16 opt 453.92 -> 130.21
59% raptor-tp6m-amazon-geckoview loadtime android-hw-g5-7-0-arm7-api-16 opt 1,173.46 -> 483.75
58% raptor-tp6m-facebook-geckoview fcp android-hw-p2-8-0-arm7-api-16 opt 1,709.67 -> 724.67
58% raptor-tp6m-facebook-geckoview fcp android-hw-p2-8-0-android-aarch64 opt 1,732.96 -> 736.33
57% raptor-tp6m-amazon-geckoview loadtime android-hw-p2-8-0-arm7-api-16 opt 924.17 -> 394.04
56% raptor-tp6m-google-geckoview loadtime android-hw-p2-8-0-android-aarch64 opt 215.50 -> 94.96
54% raptor-tp6m-google-geckoview loadtime android-hw-p2-8-0-arm7-api-16 opt 204.58 -> 93.33
54% raptor-tp6m-amazon-geckoview loadtime android-hw-p2-8-0-android-aarch64 opt 851.42 -> 393.46
47% raptor-tp6m-facebook-geckoview loadtime android-hw-p2-8-0-arm7-api-16 opt 1,819.00 -> 958.00
47% raptor-tp6m-facebook-geckoview loadtime android-hw-p2-8-0-android-aarch64 opt 1,843.00 -> 981.33
46% raptor-tp6m-google-geckoview fcp android-hw-g5-7-0-arm7-api-16 opt 459.54 -> 249.88
44% raptor-tp6m-google-geckoview android-hw-g5-7-0-arm7-api-16 opt 342.20 -> 192.24
43% raptor-tp6m-amazon-geckoview fcp android-hw-p2-8-0-arm7-api-16 opt 836.46 -> 477.42
43% raptor-tp6m-google-geckoview fcp android-hw-p2-8-0-android-aarch64 opt 257.42 -> 147.12
41% raptor-tp6m-google-geckoview fcp android-hw-p2-8-0-arm7-api-16 opt 246.75 -> 145.33
40% raptor-tp6m-amazon-geckoview android-hw-p2-8-0-arm7-api-16 opt 1,293.62 -> 779.60
38% raptor-tp6m-amazon-geckoview fcp android-hw-p2-8-0-android-aarch64 opt 775.75 -> 480.54
37% raptor-tp6m-google-geckoview android-hw-p2-8-0-arm7-api-16 opt 190.44 -> 120.89
35% raptor-tp6m-google-geckoview android-hw-p2-8-0-android-aarch64 opt 198.96 -> 129.00
35% raptor-tp6m-amazon-geckoview android-hw-p2-8-0-android-aarch64 opt 1,271.12 -> 832.58
32% raptor-tp6m-amazon-geckoview fcp android-hw-g5-7-0-arm7-api-16 opt 972.12 -> 663.00
29% raptor-tp6m-facebook-geckoview android-hw-p2-8-0-arm7-api-16 opt 1,759.44 -> 1,257.18
26% raptor-tp6m-facebook-geckoview android-hw-p2-8-0-android-aarch64 opt 1,760.73 -> 1,296.90
25% raptor-tp6m-facebook-geckoview fcp android-hw-g5-7-0-arm7-api-16 opt 1,585.79 -> 1,192.58

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=19474

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

Florin, I believe these results are expected, given the fact we've updated some records.

Comment 2

[Florin Strugariu [:Bebe]]

Yep we updated the recordings