1734859 - Update office warm page load test to navigate to sub-page rather than about:blank

Status: RESOLVED FIXED

(Testing :: Performance, task, P1)

Description

[Kimberly Sereduck :kimberlythegeek]

+++ This bug was initially created as a clone of Bug #1724261 +++

https://bugzilla.mozilla.org/show_bug.cgi?id=1716825 landed a patch to navigate to a sub-page instead of about:blank for warm page load tests.

in this bug we need to:
1 - determine what secondary URL should be used for each test
2 - implement a secondary URL and re-record page load

Comment 1

[Kimberly Sereduck :kimberlythegeek]

Attachment: Bug 1734859: Update office warm page load test to navigate to sub-page rather than about:blank

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/67c7a27f4eff
Update office warm page load test to navigate to sub-page rather than about:blank r=perftest-reviewers,AlexandruIonescu

Comment 3

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/67c7a27f4eff

Comment 4

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #31997 (as of Thu, 21 Oct 2021 03:58:29 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
91%	office loadtime	linux1804-64-shippable-qr	warm webrender	833.73 -> 1,596.21
35%	office loadtime	macosx1015-64-shippable-qr	warm webrender	946.21 -> 1,276.92
29%	office loadtime	macosx1014-64-shippable-qr	warm webrender	1,159.12 -> 1,494.79
21%	office LastVisualChange	macosx1014-64-shippable-qr	warm webrender	1,820.00 -> 2,210.00
18%	office LastVisualChange	linux1804-64-shippable-qr	warm webrender	1,406.67 -> 1,653.33
15%	office LastVisualChange	macosx1015-64-shippable-qr	warm webrender	1,426.67 -> 1,646.67
10%	office LastVisualChange	windows10-64-shippable-qr	warm webrender	1,244.62 -> 1,367.00
9%	office loadtime	linux1804-64-shippable-qr	cold webrender	2,678.25 -> 2,921.96
6%	office LastVisualChange	macosx1014-64-shippable-qr	cold webrender	3,578.33 -> 3,810.00

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
57%	office loadtime	windows10-64-shippable-qr	cold webrender	2,224.62 -> 958.96
41%	office fnbpaint	windows10-64-shippable-qr	warm webrender	446.42 -> 261.92
35%	office fnbpaint	linux1804-64-shippable-qr	warm webrender	503.38 -> 327.17
34%	office fnbpaint	macosx1015-64-shippable-qr	warm webrender	651.96 -> 430.79
29%	office fnbpaint	macosx1014-64-shippable-qr	warm webrender	757.58 -> 534.54
...	...	...	...	...
4%	office fcp	linux1804-64-shippable-qr	warm webrender	645.38 -> 620.21

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=31997