1671172 - 12.19% ebay-kleinanzeigen-search ContentfulSpeedIndex (android-hw-p2-8-0-android-aarch64-shippable) regression on push 5c2816e0436dcd0ce444ae32dee50c78aed4f63c (Fri October 9 2020)

Status: RESOLVED INVALID

(GeckoView :: General, defect)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a browsertime performance regression from push c3fe98d6045999fd751adcdece758838c49e25e3. As author of one of the patches included in that push, we need your help to address this regression.

== Change summary for alert #27209 (as of Tue, 13 Oct 2020 16:18:23 GMT) ==

Regressions:

12% ebay-kleinanzeigen-search ContentfulSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 538.50 -> 604.17
11% ebay-kleinanzeigen-search ContentfulSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 546.58 -> 606.33

Improvements:

58% ebay-kleinanzeigen loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 1,169.96 -> 493.96
54% booking loadtime android-hw-p2-8-0-android-aarch64-shippable opt cold 2,522.75 -> 1,169.42
52% ebay-kleinanzeigen loadtime android-hw-p2-8-0-android-aarch64-shippable opt cold 2,257.79 -> 1,088.79
48% ebay-kleinanzeigen loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,511.96 -> 1,812.83
44% ebay-kleinanzeigen loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,136.21 -> 1,193.79
43% ebay-kleinanzeigen-search loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 1,197.54 -> 683.88
41% ebay-kleinanzeigen-search loadtime android-hw-p2-8-0-android-aarch64-shippable opt cold 2,248.06 -> 1,318.71
41% ebay-kleinanzeigen-search loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,578.00 -> 1,516.12
36% espn LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 7,922.25 -> 5,069.83
36% bbc loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,675.83 -> 1,078.92
36% bbc loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 4,017.79 -> 2,589.29
34% bbc SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,984.50 -> 1,300.42
32% bbc SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 851.33 -> 576.33
32% bbc PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,044.92 -> 1,387.75
32% espn LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 4,777.92 -> 3,245.33
31% espn loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 1,345.50 -> 925.33
31% bbc loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 652.42 -> 449.54
31% ebay-kleinanzeigen-search loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,682.75 -> 2,544.21
29% bbc PerceptualSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 878.25 -> 620.50
29% espn PerceptualSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt cold 3,645.88 -> 2,583.75
28% allrecipes loadtime android-hw-p2-8-0-android-aarch64-shippable opt cold 2,689.83 -> 1,948.08
27% espn LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 9,334.38 -> 6,792.17
27% espn PerceptualSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 2,128.92 -> 1,560.00
26% bbc PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 4,681.92 -> 3,453.08
25% allrecipes LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt cold 2,875.17 -> 2,146.08
24% bbc SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,912.88 -> 2,972.33
24% espn LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt cold 5,792.33 -> 4,417.83
23% bbc LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 3,211.92 -> 2,463.17
23% booking loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 2,574.96 -> 1,994.58
22% bbc LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 1,361.92 -> 1,060.00
20% stackoverflow loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 2,415.19 -> 1,928.46
20% espn SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt cold 3,153.58 -> 2,530.33
20% bbc android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,057.17 -> 850.49
19% bbc ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,093.75 -> 881.17
19% bbc LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,332.50 -> 4,344.75
18% espn loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,664.50 -> 2,189.92
17% ebay-kleinanzeigen android-hw-g5-7-0-arm7-api-16-shippable opt warm 870.61 -> 719.00
17% allrecipes loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 1,666.50 -> 1,377.67
17% web-de loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,608.04 -> 2,156.08
17% bbc android-hw-p2-8-0-android-aarch64-shippable opt warm 436.41 -> 360.91
17% espn PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 4,321.00 -> 3,595.25
16% ebay-kleinanzeigen-search android-hw-g5-7-0-arm7-api-16-shippable opt warm 960.48 -> 809.55
16% ebay-kleinanzeigen LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,762.67 -> 2,331.00
15% allrecipes SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt cold 1,636.42 -> 1,383.75
15% ebay-kleinanzeigen android-hw-p2-8-0-android-aarch64-shippable opt warm 453.87 -> 384.43
15% allrecipes loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 3,998.00 -> 3,409.92
14% ebay-kleinanzeigen-search LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 1,132.58 -> 970.50
14% allrecipes LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 4,155.00 -> 3,562.58
13% espn SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 1,784.00 -> 1,544.17
13% allrecipes loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,243.69 -> 4,550.92
13% espn SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 4,362.92 -> 3,793.50
13% ebay-kleinanzeigen-search LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,609.67 -> 2,273.33
13% bbc ContentfulSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 396.58 -> 346.42
12% espn PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 6,117.58 -> 5,380.50
12% allrecipes LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,511.67 -> 4,864.00
11% bbc FirstVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 380.00 -> 336.58
11% web-de loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,445.60 -> 3,060.67
11% ebay-kleinanzeigen-search LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,827.50 -> 3,405.58
11% ebay-kleinanzeigen-search SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,891.08 -> 1,690.08
10% espn ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,354.54 -> 4,804.75
10% bbc fcp android-hw-p2-8-0-android-aarch64-shippable opt warm 363.83 -> 326.50
10% ebay-kleinanzeigen LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,405.33 -> 3,062.08
10% espn fcp android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,318.48 -> 4,785.71
10% espn FirstVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,331.50 -> 4,798.25
10% microsoft-support loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 2,906.90 -> 2,617.29
10% ebay-kleinanzeigen-search android-hw-p2-8-0-android-aarch64-shippable opt warm 510.23 -> 459.64
10% ebay-kleinanzeigen-search PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,764.50 -> 1,591.08
10% allrecipes SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,322.00 -> 2,095.83
10% espn ContentfulSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 1,539.29 -> 1,389.58
10% ebay-kleinanzeigen LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 1,442.58 -> 1,304.75
10% allrecipes SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 1,003.67 -> 908.33
9% espn android-hw-p2-8-0-android-aarch64-shippable opt warm 894.49 -> 810.49
9% espn FirstVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 1,504.79 -> 1,367.00
9% bbc fcp android-hw-g5-7-0-arm7-api-16-shippable opt warm 842.96 -> 766.38
9% bbc FirstVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 853.67 -> 777.00
9% espn fcp android-hw-p2-8-0-android-aarch64-shippable opt warm 1,492.02 -> 1,358.25
9% espn ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 3,161.42 -> 2,881.25
9% espn SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 6,065.17 -> 5,543.00
9% espn FirstVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 3,144.83 -> 2,877.25
8% espn fcp android-hw-g5-7-0-arm7-api-16-shippable opt warm 3,129.67 -> 2,866.79
7% ebay-kleinanzeigen-search SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 864.83 -> 805.83
7% espn android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,822.21 -> 1,699.50
7% ebay-kleinanzeigen PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,338.17 -> 1,249.67
6% allrecipes PerceptualSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt warm 874.08 -> 818.00
6% allrecipes PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,147.62 -> 2,022.75
6% booking loadtime android-hw-p2-8-0-android-aarch64-shippable opt warm 629.33 -> 592.96
6% booking android-hw-g5-7-0-arm7-api-16-shippable opt cold 1,176.32 -> 1,108.55
5% allrecipes SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,305.04 -> 3,148.33
4% microsoft-support loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,488.50 -> 2,378.33
4% booking loadtime android-hw-g5-7-0-arm7-api-16-shippable opt warm 1,636.79 -> 1,567.88
4% booking LastVisualChange android-hw-p2-8-0-android-aarch64-shippable opt warm 995.88 -> 955.92
4% booking LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt warm 2,577.33 -> 2,480.58
3% web-de android-hw-g5-7-0-arm7-api-16-shippable opt warm 676.74 -> 653.15

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=27209

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[[ex-Mozilla] Agi Sferro | :agi]

Reverted all the GVE changes to see if it makes a difference https://treeherder.mozilla.org/#/jobs?repo=try&revision=d96de7e5777336ae1fa5548797449213982c2024

Comment 2

[[ex-Mozilla] Agi Sferro | :agi]

So from what I can see in the try, these perf changes are all related to GVE (our prototyping app) and not to the actual code. I'll try to figure out why this change but in general it shouldn't affect Fenix or any GV consumer.

Comment 3

[[ex-Mozilla] Agi Sferro | :agi]

This is caused by us turning Tracking Protection On by default in GeckoViewExample, so nothing really changed in the product.