Closed Bug 1673316 Opened 1 year ago Closed 1 year ago

56% raptor-speedometer-geckoview (android-hw-p2-8-0-android-aarch64-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020)

Categories

(GeckoView :: General, defect)

Firefox 84
defect

Tracking

(firefox82 unaffected, firefox83 unaffected, firefox84 fixed)

RESOLVED FIXED
84 Branch
Tracking Status
firefox82 --- unaffected
firefox83 --- unaffected
firefox84 --- fixed

People

(Reporter: Bebe, Assigned: agi)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Attachments

(1 file)

Perfherder has detected a browsertime performance regression from push f087af9e75e69519555b6b2e7a4601e31558f895. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
6% amazon-search ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 1,021.25 -> 1,082.25
3% amazon-search ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 1,034.50 -> 1,065.58
3% amazon ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 695.00 -> 713.75
2% amazon PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 677.54 -> 694.17
2% amazon-search loadtime android-hw-g5-7-0-arm7-api-16-shippable warm 1,935.62 -> 1,979.54
2% amazon-search fcp android-hw-g5-7-0-arm7-api-16-shippable warm 876.29 -> 895.83
2% amazon-search android-hw-g5-7-0-arm7-api-16-shippable warm 1,009.00 -> 1,031.02
2% amazon SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 672.29 -> 686.58
2% amazon-search FirstVisualChange android-hw-g5-7-0-arm7-api-16-shippable warm 885.83 -> 904.08
2% amazon-search SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable warm 937.42 -> 956.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Component: Performance → General
Flags: needinfo?(agi)
Product: Testing → GeckoView
Summary: 2 - 5.97% amazon visual metrics (android-hw-g5-7-0-arm7-api-16-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020) → 2 - 5.97% amazon browsertime visual metrics (android-hw-g5-7-0-arm7-api-16-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020)

== Change summary for alert #27355 (as of Mon, 26 Oct 2020 08:00:02 GMT) ==

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
56% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64-shippable nocondprof 25.10 -> 11.07
56% raptor-speedometer-geckoview android-hw-p2-8-0-android-aarch64-shippable nocondprof 25.28 -> 11.15

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=27355

Ok, a 2-6% regression for page load seems like it could be noise or otherwise acceptable, but a 56% speedometer reduction is insane. I don't even think we expect to do process switching during Speedometer, right? Also, it's only on aarch64 but not 32-bit?

Nika, maybe you have ideas on the perf regression above?

Flags: needinfo?(nika)

Sent a try here to confirm that the "meat" of Bug 1646810 is the problem (i.e. using document channel to switch process): https://treeherder.mozilla.org/#/jobs?repo=try&revision=33bf3a66b898283fc13ce6fc61a4b165a7079002

Flags: needinfo?(agi)

As I mentioned on Matrix, I'm really not sure what could be causing these issues. Theoretically there should be absolutely no difference in speedometer perf between before/after.

Flags: needinfo?(nika)
Summary: 2 - 5.97% amazon browsertime visual metrics (android-hw-g5-7-0-arm7-api-16-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020) → 56% raptor-speedometer-geckoview (android-hw-p2-8-0-android-aarch64-shippable) regression on push f087af9e75e69519555b6b2e7a4601e31558f895 (Tue October 20 2020)

Update: after a little bit of work we're able to reproduce locally. Working on getting a profile of before/after to see what's going on.

Looks like this does not reproduce on browsertime, so it's likely a test-only issue.

This is because docShellIsActive is reset to false when switching process. Opened Bug 1673683 to address that.

I'm not a fan of accessing private bits like docShellIsActive in tests like
that but it's both:

  1. very important, if we don't correctly activate docShell performance will be
    very poor

  2. stable, this API is not likely to change, and if it changes it
    should be easy to get whatever the replacement is

Assignee: nobody → agi
Status: NEW → ASSIGNED
Pushed by asferro@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6e193e6ff95d
Ensure docShellIsActive is preserved when switching process. r=snorp
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 84 Branch

Funnily enough I think this test has been removed shortly before this fix: https://hg.mozilla.org/mozilla-central/rev/815817bdeb1a62e45a19ad6305f1dc9f1ca074b2

Duplicate of this bug: 1672552
You need to log in before you can comment on or make changes to this bug.