1730532 - Update amazon page load test to navigate to sub-page rather than about:blank

Status: RESOLVED FIXED

(Testing :: Performance, task, P3)

Description

[Kimberly Sereduck :kimberlythegeek]

+++ This bug was initially created as a clone of Bug #1724261 +++

https://bugzilla.mozilla.org/show_bug.cgi?id=1716825 landed a patch to navigate to a sub-page instead of about:blank for warm page load tests.

in this bug we need to:
1 - determine what secondary URL should be used for each test
2 - implement a secondary URL and re-record page load

Comment 1

[Kimberly Sereduck :kimberlythegeek]

Attachment: Bug 1730532: Update amazon page load test to navigate to sub-page rather than about:blank

Comment 2

[Kimberly Sereduck :kimberlythegeek]

Fails only on profiled runs, error is "failed to load page, timed out" but when running locally the browser appears to crash when navigating from secondary url back to primary url

Comment 3

[Kimberly Sereduck :kimberlythegeek]

Attachment: WIP: Bug 1730532: mitm7 amazon secondary url

Depends on D127738

Comment 4

[Kimberly Sereduck :kimberlythegeek]

Attachment: Bug 1730532: Update amazon page load test to navigate to sub-page rather than about:blank

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/13fddbd2f5a6
Update amazon page load test to navigate to sub-page rather than about:blank r=perftest-reviewers,AlexandruIonescu

Comment 6

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/13fddbd2f5a6

Comment 7

[Alex Finder]

== Change summary for alert #32732 (as of Wed, 15 Dec 2021 19:14:13 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
32%	amazon fcp	linux1804-64-shippable-qr	warm webrender	177.33 -> 234.29
31%	amazon fcp	macosx1015-64-shippable-qr	fission warm webrender	131.92 -> 173.00
30%	amazon fnbpaint	macosx1014-64-shippable-qr	fission warm webrender	267.42 -> 348.46
29%	amazon fcp	macosx1015-64-shippable-qr	warm webrender	127.88 -> 165.46
29%	amazon dcf	macosx1014-64-shippable-qr	fission warm webrender	289.50 -> 373.71
29%	amazon fcp	macosx1014-64-shippable-qr	fission warm webrender	208.92 -> 268.75
28%	amazon fcp	macosx1014-64-shippable-qr	warm webrender	191.92 -> 246.17
25%	amazon fnbpaint	linux1804-64-shippable-qr	warm webrender	222.62 -> 279.04
25%	amazon dcf	linux1804-64-shippable-qr	warm webrender	229.88 -> 287.21
24%	amazon fnbpaint	macosx1014-64-shippable-qr	warm webrender	248.25 -> 308.79
...	...	...	...	...
5%	amazon LastVisualChange	linux1804-64-shippable-qr	cold webrender	1,811.56 -> 1,903.33
5%	welcome ContentfulSpeedIndex	macosx1014-64-shippable-qr	cold webrender	1,610.83 -> 1,687.83
4%	amazon LastVisualChange	linux1804-64-shippable-qr	warm webrender	1,463.33 -> 1,515.56
3%	amazon SpeedIndex	macosx1015-64-shippable-qr	fission warm webrender	254.17 -> 261.75
3%	amazon ContentfulSpeedIndex	macosx1015-64-shippable-qr	fission warm webrender	256.25 -> 263.25

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
16%	amazon loadtime	linux1804-64-shippable-qr	fission warm webrender	512.06 -> 431.88
14%	amazon LastVisualChange	macosx1014-64-shippable-qr	fission warm webrender	2,736.67 -> 2,366.67
11%	amazon dcf	macosx1015-64-shippable-qr	cold webrender	370.23 -> 327.71
11%	amazon fcp	macosx1015-64-shippable-qr	cold webrender	326.00 -> 290.46
11%	amazon FirstVisualChange	macosx1015-64-shippable-qr	cold webrender	478.33 -> 426.67
...	...	...	...	...
2%	amazon SpeedIndex	linux1804-64-shippable-qr	cold webrender	483.83 -> 473.08

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=32732