1674701 - 2.15 - 77.61% allrecipes SpeedIndex / bbc Similarity / bing-search-restaurants Similarity / bing-search-restaurants Similarity2D / ebay-kleinanzeigen Similarity2D / ebay-kleinanzeigen-search Similarity / ebay-kleinanzeigen-search Similarity2D / facebook C

Status: RESOLVED FIXED

(GeckoView :: General, defect)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a browsertime performance regression from push d03053f89c391892c76babb155365782648c72a0. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
78%	instagram	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	693.75 -> 1,232.17
72%	instagram	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	715.08 -> 1,230.75
23%	facebook	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	743.17 -> 916.92
12%	allrecipes	SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	974.67 -> 1,094.33
8%	microsoft-support	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.92 -> 0.84
7%	microsoft-support	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.92 -> 0.85
6%	bing-search-restaurants	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.96 -> 0.90
5%	stackoverflow	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.90 -> 0.85
5%	facebook	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.95 -> 0.91
4%	google-search-restaurants	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.95 -> 0.91
3%	ebay-kleinanzeigen-search	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.96 -> 0.92
3%	ebay-kleinanzeigen-search	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.95 -> 0.92
3%	stackoverflow	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.92 -> 0.89
3%	bbc	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.94 -> 0.91
3%	facebook	Similarity	android-hw-g5-7-0-arm7-api-16-shippable	cold	0.98 -> 0.95
3%	stackoverflow	Similarity	android-hw-g5-7-0-arm7-api-16-shippable	cold	0.97 -> 0.95
3%	facebook	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.92 -> 0.89
3%	facebook	Similarity2D	android-hw-g5-7-0-arm7-api-16-shippable	cold	0.99 -> 0.97
2%	bbc	Similarity	android-hw-g5-7-0-arm7-api-16-shippable	cold	0.97 -> 0.95
2%	instagram	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.97 -> 0.95
2%	bing-search-restaurants	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.97 -> 0.95
2%	ebay-kleinanzeigen	Similarity2D	android-hw-p2-8-0-android-aarch64-shippable	cold	0.92 -> 0.90
2%	google-search-restaurants	Similarity	android-hw-p2-8-0-android-aarch64-shippable	cold	0.97 -> 0.95

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
13%	bing	ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	650.96 -> 565.25
8%	bing		android-hw-p2-8-0-android-aarch64-shippable	warm	206.47 -> 190.37
7%	bing	loadtime	android-hw-p2-8-0-android-aarch64-shippable	warm	196.67 -> 182.25
6%	booking	SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	438.33 -> 411.50
5%	booking	PerceptualSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	390.96 -> 370.25
5%	bbc		android-hw-p2-8-0-android-aarch64-shippable	warm	381.52 -> 361.72
5%	booking		android-hw-p2-8-0-android-aarch64-shippable	warm	420.82 -> 400.45
4%	google-search-restaurants		android-hw-g5-7-0-arm7-api-16-shippable	warm	414.35 -> 395.90
4%	ebay-kleinanzeigen		android-hw-p2-8-0-android-aarch64-shippable	warm	403.67 -> 386.28
4%	bing	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	335.58 -> 321.50
4%	bbc	FirstVisualChange	android-hw-p2-8-0-android-aarch64-shippable	warm	352.75 -> 338.25
4%	allrecipes	PerceptualSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	885.96 -> 850.83
4%	wikipedia	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	367.83 -> 353.83
4%	booking	loadtime	android-hw-p2-8-0-android-aarch64-shippable	warm	648.83 -> 625.33
4%	google-search-restaurants	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	435.12 -> 419.75
4%	google-search-restaurants	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	435.12 -> 419.75
4%	google-search-restaurants	LastVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	435.12 -> 419.75
4%	google-search-restaurants	PerceptualSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	435.12 -> 419.75
4%	google-search-restaurants	SpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	435.12 -> 419.75
3%	bing		android-hw-g5-7-0-arm7-api-16-shippable	warm	305.85 -> 295.38
3%	bbc	loadtime	android-hw-p2-8-0-android-aarch64-shippable	warm	473.85 -> 460.21
2%	facebook-cristiano	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	719.25 -> 702.83
2%	facebook-cristiano	SpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	763.96 -> 746.92
2%	facebook	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	716.79 -> 701.58
2%	facebook	SpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	781.04 -> 765.17
2%	facebook-cristiano	PerceptualSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	950.08 -> 930.92

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[[ex-Mozilla] Agi Sferro | :agi]

The 70% change on Instagram is wild! Will look into it today.

Comment 2

[[ex-Mozilla] Agi Sferro | :agi]

This might be caused by a bug in the code that tries to determine what is a duplicate START event, sent out a try to verify: https://treeherder.mozilla.org/jobs?repo=try&revision=691dae9115bb22a862c6077ead813fa3dd47eae4

Comment 3

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1674701 - Don't ignore duplicate page loads.

This patch makes it so we only ignore duplicated start events if there is a
request in progress, so that navigating to the same URI twice in a row will
results in the right events being fired.

Comment 4

[[ex-Mozilla] Agi Sferro | :agi]

I'm not confident that the patch fixes this problem (although it fixes a problem, so I'm landing it anyway).

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/09cf5308e81c
Don't ignore duplicate page loads. r=esawin

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/09cf5308e81c

Comment 7

[[ex-Mozilla] Agi Sferro | :agi]

Looks like it worked! \o/ https://treeherder.mozilla.org/jobs?repo=autoland&revision=09cf5308e81cf666cd1aee22346a6efe5d3e6398&group_state=expanded&selectedTaskRun=YCcHywrXR2mnKkynB2u3UQ.0

Comment 8

[Florin Strugariu [:Bebe]]

== Change summary for alert #27501 (as of Wed, 04 Nov 2020 03:41:43 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
40%	instagram	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	1,298.50 -> 774.25

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27501

Comment 9

[Florin Strugariu [:Bebe]]

Update on the previous comment
== Change summary for alert #27501 (as of Wed, 04 Nov 2020 03:41:43 GMT) ==

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
18%	bing	ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	568.00 -> 670.67
14%	reddit	fcp	android-hw-p2-8-0-android-aarch64-shippable	warm	204.79 -> 232.96
11%	reddit	FirstVisualChange	android-hw-p2-8-0-android-aarch64-shippable	warm	260.83 -> 289.25
11%	imdb	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	2,067.58 -> 2,291.08
10%	bing	loadtime	android-hw-p2-8-0-android-aarch64-shippable	warm	179.88 -> 197.50
9%	google-search-restaurants	fcp	android-hw-p2-8-0-android-aarch64-shippable	warm	187.54 -> 203.67
8%	ebay-kleinanzeigen-search	ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	564.40 -> 607.42
7%	google-search-restaurants		android-hw-p2-8-0-android-aarch64-shippable	warm	219.01 -> 235.39
7%	reddit		android-hw-p2-8-0-android-aarch64-shippable	warm	425.06 -> 456.31
7%	google-search-restaurants	ContentfulSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	240.42 -> 257.33
7%	google-search-restaurants	FirstVisualChange	android-hw-p2-8-0-android-aarch64-shippable	warm	240.42 -> 257.33
7%	google-search-restaurants	LastVisualChange	android-hw-p2-8-0-android-aarch64-shippable	warm	240.42 -> 257.33
7%	google-search-restaurants	PerceptualSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	240.42 -> 257.33
7%	google-search-restaurants	SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	240.42 -> 257.33
6%	stackoverflow	fcp	android-hw-p2-8-0-android-aarch64-shippable	warm	270.00 -> 286.08
6%	facebook	PerceptualSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	405.38 -> 428.00
6%	facebook	FirstVisualChange	android-hw-p2-8-0-android-aarch64-shippable	warm	362.75 -> 382.92
5%	bing-search-restaurants	SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	380.58 -> 401.00
5%	facebook	SpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	389.79 -> 410.08
5%	stackoverflow		android-hw-p2-8-0-android-aarch64-shippable	warm	307.93 -> 323.91
5%	google-search-restaurants	fcp	android-hw-g5-7-0-arm7-api-16-shippable	warm	301.21 -> 316.04
4%	allrecipes	PerceptualSpeedIndex	android-hw-p2-8-0-android-aarch64-shippable	warm	854.33 -> 884.83
3%	google-search-restaurants		android-hw-g5-7-0-arm7-api-16-shippable	warm	369.24 -> 380.33
3%	booking	PerceptualSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	833.24 -> 857.67
3%	youtube	fcp	android-hw-g5-7-0-arm7-api-16-shippable	warm	718.69 -> 737.04

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
40%	instagram	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	1,298.50 -> 774.25
20%	facebook	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	933.17 -> 746.58
4%	youtube	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	648.79 -> 625.17

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27501

This looks like its reverting all other regressions

Comment 10

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #27525 (as of Thu, 05 Nov 2020 13:18:14 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
15%	tabswitch		windows10-64-shippable-qr	e10s stylo webrender	10.31 -> 8.77
15%	tabswitch		windows10-64-shippable-qr	e10s stylo webrender	10.34 -> 8.82

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27525

Comment 11

[[ex-Mozilla] Agi Sferro | :agi]

I'm not sure what the question is, the regressions are expected since the "improvments" from the original patch were bogus and caused by a bug.

Comment 12

[Alexandru Ionescu (needinfo me) [:alexandrui]]

I have no questions, the workflow is to comment with the improvements caused by a push associated to the bug, no matter they were because reverting another revision or any other reason.