1730961 - Update expedia page load test to navigate to sub-page rather than about:blank

Status: RESOLVED FIXED

(Testing :: Performance, task, P1)

Description

[Kimberly Sereduck :kimberlythegeek]

+++ This bug was initially created as a clone of Bug #1724261 +++

https://bugzilla.mozilla.org/show_bug.cgi?id=1716825 landed a patch to navigate to a sub-page instead of about:blank for warm page load tests.

in this bug we need to:
1 - determine what secondary URL should be used for each test
2 - implement a secondary URL and re-record page load

Comment 1

[Kimberly Sereduck :kimberlythegeek]

Attachment: Bug 1730961: Update expedia page load test to navigate to sub-page rather than about:blank

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5fa251d8908
Update expedia page load test to navigate to sub-page rather than about:blank r=perftest-reviewers,AlexandruIonescu

Comment 3

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/a5fa251d8908

Comment 4

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #31440 (as of Wed, 22 Sep 2021 06:50:42 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
84%	expedia LastVisualChange	linux1804-64-shippable-qr	warm webrender	1,240.00 -> 2,276.67
76%	expedia LastVisualChange	macosx1015-64-shippable-qr	warm webrender	1,176.67 -> 2,076.67
72%	expedia LastVisualChange	macosx1014-64-shippable-qr	warm webrender	1,506.67 -> 2,590.00
71%	expedia loadtime	linux1804-64-shippable-qr	warm webrender	1,251.67 -> 2,146.00
66%	expedia LastVisualChange	windows10-64-shippable-qr	warm webrender	1,254.00 -> 2,087.00
60%	expedia loadtime	macosx1014-64-shippable-qr	warm webrender	1,580.17 -> 2,524.88
55%	expedia loadtime	windows10-64-shippable-qr	warm webrender	1,253.88 -> 1,949.46
52%	expedia LastVisualChange	linux1804-64-shippable-qr	cold webrender	1,950.00 -> 2,973.33
52%	expedia loadtime	macosx1014-64-shippable-qr	cold webrender	3,950.67 -> 5,996.00
49%	expedia LastVisualChange	macosx1015-64-shippable-qr	cold webrender	1,946.67 -> 2,896.67
...	...	...	...	...
6%	expedia ContentfulSpeedIndex	linux1804-64-shippable-qr	cold webrender	604.00 -> 641.58
6%	expedia ContentfulSpeedIndex	windows10-64-shippable-qr	cold webrender	563.25 -> 597.92
5%	expedia FirstVisualChange	linux1804-64-shippable-qr	cold webrender	531.67 -> 560.00
5%	expedia fcp	windows10-64-shippable-qr	cold webrender	422.42 -> 444.71
4%	expedia fnbpaint	windows10-64-shippable-qr	cold webrender	440.67 -> 457.75

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
32%	expedia PerceptualSpeedIndex	windows10-64-shippable-qr	cold webrender	1,300.67 -> 890.33
31%	expedia PerceptualSpeedIndex	windows10-64-shippable-qr	warm webrender	848.50 -> 584.17
29%	expedia PerceptualSpeedIndex	linux1804-64-shippable-qr	cold webrender	1,354.33 -> 958.58
29%	expedia PerceptualSpeedIndex	macosx1014-64-shippable-qr	warm webrender	1,040.75 -> 740.67
28%	expedia PerceptualSpeedIndex	macosx1015-64-shippable-qr	warm webrender	838.42 -> 607.58
...	...	...	...	...
4%	expedia SpeedIndex	linux1804-64-shippable-qr	warm webrender	437.17 -> 421.33

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=31440

Comment 5

[Kimberly Sereduck :kimberlythegeek]

Accepting as new baseline

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

This bug was fixed, the associated Perfherder alert has been wontfixed.