1814367 - 570.85% amazon loadtime (Android) regression on Fri January 27 2023

Status: RESOLVED DUPLICATE of bug 1814835

(Testing :: geckodriver, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a browsertime performance regression from push 0ec98bd3d5b3213bb403fc16d4168e5fb44d5abb. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
623%	amazon loadtime	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,262.75 -> 9,126.96
571%	amazon loadtime	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,260.29 -> 8,454.62
525%	cnn-ampstories loadtime	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,580.17 -> 9,880.79
487%	amazon fcp	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	811.71 -> 4,763.17
475%	microsoft-support loadtime	android-hw-a51-11-0-aarch64-shippable-qr	cold fission webrender	2,459.33 -> 14,142.83
461%	cnn-ampstories PerceptualSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,620.75 -> 9,087.50
421%	cnn-ampstories LastVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,842.08 -> 9,604.75
409%	cnn-ampstories SpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,792.08 -> 9,124.33
407%	amazon FirstVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	939.50 -> 4,765.67
385%	espn fcp	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	1,143.46 -> 5,541.00
...	...	...	...	...	...
220%	youtube ContentfulSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	872.83 -> 2,789.33
197%	espn LastVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	4,025.42 -> 11,943.25
180%	espn LastVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold fission webrender	4,420.33 -> 12,369.25
97%	dailymail LastVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	2,607.25 -> 5,126.00
39%	imgur loadtime	macosx1015-64-shippable-qr	cold fission webrender	1,565.44 -> 2,169.58	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
11%	wikipedia fcp	android-hw-a51-11-0-aarch64-shippable-qr	cold fission webrender	970.21 -> 859.17
9%	amazon fcp	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	217.71 -> 199.21
6%	amazon loadtime	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	367.56 -> 346.62
3%	amazon PerceptualSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	649.38 -> 628.75

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #0)

Perfherder has detected a browsertime performance regression from push 0ec98bd3d5b3213bb403fc16d4168e5fb44d5abb. As author of one of the patches included in that push, we need your help to address this regression.

This push was a backout so it's not clear to me how this could produce a performance regression. Is it still present on latest mozilla-central?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Please note that the added reference to bug 1811545 doesn't make any changes in Marionette but is all test-only. So I do not see a connection.

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Thanks whimboo, I triggered some backfills and I'm waiting for the outcome. Sparky, you might want to know about this.