1673316 - 56% raptor-speedometer-geckoview (android-hw-p2-8-0-android-aarch64-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020)

Status: RESOLVED FIXED

(GeckoView :: General, defect)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a browsertime performance regression from push f087af9e75e69519555b6b2e7a4601e31558f895. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
6%	amazon-search	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	1,021.25 -> 1,082.25
3%	amazon-search	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	1,034.50 -> 1,065.58
3%	amazon	ContentfulSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	695.00 -> 713.75
2%	amazon	PerceptualSpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	677.54 -> 694.17
2%	amazon-search	loadtime	android-hw-g5-7-0-arm7-api-16-shippable	warm	1,935.62 -> 1,979.54
2%	amazon-search	fcp	android-hw-g5-7-0-arm7-api-16-shippable	warm	876.29 -> 895.83
2%	amazon-search		android-hw-g5-7-0-arm7-api-16-shippable	warm	1,009.00 -> 1,031.02
2%	amazon	SpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	672.29 -> 686.58
2%	amazon-search	FirstVisualChange	android-hw-g5-7-0-arm7-api-16-shippable	warm	885.83 -> 904.08
2%	amazon-search	SpeedIndex	android-hw-g5-7-0-arm7-api-16-shippable	warm	937.42 -> 956.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Florin Strugariu [:Bebe]]

== Change summary for alert #27355 (as of Mon, 26 Oct 2020 08:00:02 GMT) ==

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
56%	raptor-speedometer-geckoview		android-hw-p2-8-0-android-aarch64-shippable	nocondprof	25.10 -> 11.07
56%	raptor-speedometer-geckoview		android-hw-p2-8-0-android-aarch64-shippable	nocondprof	25.28 -> 11.15

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=27355

Comment 2

[James Willcox (:snorp) ([e-mail]) (he/him)]

Ok, a 2-6% regression for page load seems like it could be noise or otherwise acceptable, but a 56% speedometer reduction is insane. I don't even think we expect to do process switching during Speedometer, right? Also, it's only on aarch64 but not 32-bit?

Comment 3

[James Willcox (:snorp) ([e-mail]) (he/him)]

Nika, maybe you have ideas on the perf regression above?

Comment 4

[[ex-Mozilla] Agi Sferro | :agi]

Sent a try here to confirm that the "meat" of Bug 1646810 is the problem (i.e. using document channel to switch process): https://treeherder.mozilla.org/#/jobs?repo=try&revision=33bf3a66b898283fc13ce6fc61a4b165a7079002

Comment 5

[Nika Layzell [:nika] (ni? for response)]

As I mentioned on Matrix, I'm really not sure what could be causing these issues. Theoretically there should be absolutely no difference in speedometer perf between before/after.

Comment 6

[[ex-Mozilla] Agi Sferro | :agi]

Update: after a little bit of work we're able to reproduce locally. Working on getting a profile of before/after to see what's going on.

Comment 7

[[ex-Mozilla] Agi Sferro | :agi]

Looks like this does not reproduce on browsertime, so it's likely a test-only issue.

Comment 8

[[ex-Mozilla] Agi Sferro | :agi]

This is because docShellIsActive is reset to false when switching process. Opened Bug 1673683 to address that.

Comment 9

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1673316 - Ensure docShellIsActive is preserved when switching process.

I'm not a fan of accessing private bits like docShellIsActive in tests like
that but it's both:

1. very important, if we don't correctly activate docShell performance will be
   very poor

2. stable, this API is not likely to change, and if it changes it
   should be easy to get whatever the replacement is

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6e193e6ff95d
Ensure docShellIsActive is preserved when switching process. r=snorp

Comment 11

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/6e193e6ff95d

Comment 12

[[ex-Mozilla] Agi Sferro | :agi]

Funnily enough I think this test has been removed shortly before this fix: https://hg.mozilla.org/mozilla-central/rev/815817bdeb1a62e45a19ad6305f1dc9f1ca074b2