Closed Bug 1643263 Opened 2 years ago Closed 1 year ago

Refresh pageload snapshots (re-record to mitm5)

Categories

(Testing :: Performance, defect, P1)

defect

Tracking

(firefox80 fixed)

RESOLVED FIXED
mozilla80
Tracking Status
firefox80 --- fixed

People

(Reporter: alexandrui, Assigned: alexandrui)

References

Details

(Keywords: perf-alert)

Attachments

(11 files, 1 obsolete file)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review

Currently the recordings are using mitm4. They need to be updated to mitm5. This task includes also the update of mitm to 5.1.1 and re-recordings as they need to be tested together.

No longer depends on: 1643261
No longer depends on: 1643265
Duplicate of this bug: 1643261

This patch is updating the code to mitm 5.1.1 and mitmproxy manifest.

Attachment #9155878 - Attachment description: Bug 1643263 Re-record amazon, fandom, imgur, youtube, twitter r?#perftest,bebe → Bug 1643263 Re-record amazon, apple, bing-search r?#perftest,bebe
Attachment #9155877 - Attachment description: Bug 1643263 Update mitm from 5.0.1 to 5.1.1. → Bug 1643263 Update mitm from 5.0.1 to 5.1.1. r?#perftest,bebe
Depends on: 1643265
Priority: P2 → P1
Attachment #9158553 - Attachment description: Bug 1643263 Re-record microsoft, netflix r?#perftest,bebe → Bug 1643263 Re-record microsoft r?#perftest,bebe
Attachment #9158554 - Attachment description: Bug 1643263 Re-record pinterest, reddit, tumblr r?#perftest,bebe → Bug 1643263 Re-record reddit, tumblr r?#perftest,bebe
Attachment #9158556 - Attachment description: Bug 1643263 Re-record twitch, twitter, wikipedia r?#perftest,bebe → Bug 1643263 Re-record twitch, wikipedia r?#perftest,bebe
Attachment #9158557 - Attachment description: Bug 1643263 Re-record yandex, youtube r?#perftest,bebe → Bug 1643263 Re-record yandex r?#perftest,bebe
Attachment #9161567 - Attachment description: Bug 1643263 Re-record facebook, google-search, yahoo-news r?#perftest,bebe → Bug 1643263 Re-record facebook, google-search r?#perftest,bebe
Attachment #9161566 - Attachment is obsolete: true
Pushed by fstrugariu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/430d39e9301c
Update mitm from 5.0.1 to 5.1.1. r=perftest-reviewers,Bebe
https://hg.mozilla.org/integration/autoland/rev/f657d3d6b29d
Re-record amazon, apple, bing-search r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/a3ee21aedcbd
Re-record ebay, fandom, imdb r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/ed8761d44ec6
Re-record imgur, instagram, linkedin r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/96b8a48f36a5
Re-record microsoft r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/371928b72ff7
Re-record reddit, tumblr r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/634fd6b52636
Re-record twitch, wikipedia r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/9b83189a9f0a
Re-record yandex r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/21e7199214d1
Re-record facebook, google-search r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/f775f5979117
Re-record yahoo-mail, google-mail, facebook-redesign r=Bebe,perftest-reviewers
https://hg.mozilla.org/integration/autoland/rev/4e4ad424fce7
Remove raptor-tp6-1 (leftover) r=Bebe,perftest-reviewers

This is the first part of the rerecordings please observe the regression and offer input

== Change summary for alert #26516 (as of Thu, 16 Jul 2020 04:29:46 GMT) ==

Regressions:

151% raptor-tp6-google-firefox-cold loadtime windows10-64-shippable opt 425.33 -> 1,066.00
148% raptor-tp6-google-firefox-cold loadtime windows7-32-shippable opt 457.17 -> 1,132.67
46% raptor-tp6-facebook-firefox-cold loadtime windows7-32-shippable opt 1,880.67 -> 2,748.92
46% raptor-tp6-facebook-firefox-cold loadtime windows10-64-shippable opt 1,881.83 -> 2,744.17
46% raptor-tp6-facebook-firefox-cold fcp linux64-shippable-qr opt webrender 729.17 -> 1,062.25
46% raptor-tp6-facebook-firefox-cold fcp linux64-shippable opt 702.58 -> 1,022.25
44% raptor-tp6-wikipedia-firefox-cold loadtime windows7-32-shippable opt 1,351.33 -> 1,948.92
44% raptor-tp6-facebook-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,867.75 -> 2,691.50
43% raptor-tp6-wikipedia-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,305.17 -> 1,870.08
43% raptor-tp6-facebook-firefox-cold windows10-64-shippable opt 863.25 -> 1,235.84
43% raptor-tp6-facebook-redesign-firefox-cold loadtime macosx1014-64-shippable opt 2,065.83 -> 2,955.67
43% raptor-tp6-linkedin-firefox-cold fcp macosx1014-64-shippable opt 634.33 -> 904.33
43% raptor-tp6-imdb-firefox-cold fcp macosx1014-64-shippable opt 992.42 -> 1,414.17
42% raptor-tp6-facebook-firefox-cold windows7-32-shippable opt 865.90 -> 1,230.95
42% raptor-tp6-imdb-firefox-cold macosx1014-64-shippable opt 1,196.03 -> 1,693.23
41% raptor-tp6-facebook-firefox-cold windows10-64-shippable-qr opt webrender 851.76 -> 1,200.64
41% raptor-tp6-facebook-firefox-cold fcp windows10-64-shippable opt 692.50 -> 974.75
41% raptor-tp6-wikipedia-firefox-cold loadtime windows10-64-shippable opt 1,336.25 -> 1,879.50
39% raptor-tp6-facebook-firefox-cold fcp windows7-32-shippable opt 695.58 -> 969.75
39% raptor-tp6-facebook-firefox-cold fcp windows10-64-shippable-qr opt webrender 674.17 -> 935.75
38% raptor-tp6-linkedin-firefox-cold fcp windows7-32-shippable opt 377.25 -> 521.08
38% raptor-tp6-linkedin-firefox-cold fcp windows10-64-shippable opt 372.17 -> 512.58
37% raptor-tp6-imgur-firefox-cold loadtime macosx1014-64-shippable opt 3,725.08 -> 5,119.75
37% raptor-tp6-facebook-redesign-firefox-cold loadtime linux64-shippable-qr opt webrender 1,482.42 -> 2,026.42
37% raptor-tp6-imdb-firefox-cold loadtime windows7-32-shippable opt 1,433.42 -> 1,957.08
37% raptor-tp6-linkedin-firefox-cold fcp windows10-64-shippable-qr opt webrender 350.58 -> 478.67
36% raptor-tp6-facebook-firefox-cold linux64-shippable-qr opt webrender 994.67 -> 1,350.64
36% raptor-tp6-imdb-firefox-cold loadtime windows10-64-shippable opt 1,433.83 -> 1,945.17
36% raptor-tp6-facebook-firefox-cold linux64-shippable opt 966.04 -> 1,309.61
35% raptor-tp6-facebook-redesign-firefox-cold loadtime linux64-shippable opt 1,434.75 -> 1,941.92
34% raptor-tp6-wikipedia-firefox-cold windows7-32-shippable opt 1,251.02 -> 1,681.85
34% raptor-tp6-imdb-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,376.92 -> 1,844.42
34% raptor-tp6-imdb-firefox-cold linux64-shippable opt 823.47 -> 1,102.41
34% raptor-tp6-wikipedia-firefox-cold windows10-64-shippable opt 1,225.68 -> 1,638.85
34% raptor-tp6-wikipedia-firefox-cold windows10-64-shippable-qr opt webrender 1,241.57 -> 1,657.92
33% raptor-tp6-linkedin-firefox-cold fcp linux64-shippable opt 433.42 -> 576.42
33% raptor-tp6-facebook-redesign-firefox-cold macosx1014-64-shippable opt 1,213.11 -> 1,612.96
33% raptor-tp6-facebook-redesign-firefox-cold windows10-64-shippable opt 810.61 -> 1,076.27
33% raptor-tp6-linkedin-firefox-cold macosx1014-64-shippable opt 1,147.44 -> 1,521.92
33% raptor-tp6-wikipedia-firefox-cold fcp windows7-32-shippable opt 1,217.83 -> 1,615.08
33% raptor-tp6-linkedin-firefox-cold fcp linux64-shippable-qr opt webrender 441.58 -> 585.42
32% raptor-tp6-facebook-redesign-firefox-cold windows10-64-shippable-qr opt webrender 776.42 -> 1,026.77
32% raptor-tp6-imdb-firefox-cold fcp linux64-shippable opt 695.75 -> 919.08
32% raptor-tp6-facebook-redesign-firefox-cold windows7-32-shippable opt 803.87 -> 1,061.25
32% raptor-tp6-facebook-redesign-firefox-cold loadtime windows10-64-shippable opt 1,419.75 -> 1,873.00
32% raptor-tp6-imdb-firefox-cold loadtime macosx1014-64-shippable opt 2,221.42 -> 2,929.83
32% raptor-tp6-facebook-redesign-firefox-cold fcp windows10-64-shippable-qr opt webrender 799.42 -> 1,052.58
32% raptor-tp6-facebook-redesign-firefox-cold fcp windows10-64-shippable opt 903.67 -> 1,188.50
31% raptor-tp6-imdb-firefox-cold loadtime linux64-shippable-qr opt webrender 1,557.08 -> 2,038.25
31% raptor-tp6-facebook-redesign-firefox-cold fcp windows7-32-shippable opt 908.33 -> 1,188.67
31% raptor-tp6-wikipedia-firefox-cold fcp windows10-64-shippable opt 1,206.08 -> 1,578.00
31% raptor-tp6-facebook-redesign-firefox-cold loadtime windows7-32-shippable opt 1,401.42 -> 1,833.50
30% raptor-tp6-facebook-redesign-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,404.58 -> 1,831.92
30% raptor-tp6-imdb-firefox-cold loadtime linux64-shippable opt 1,521.17 -> 1,981.08
29% raptor-tp6-facebook-firefox-cold fcp macosx1014-64-shippable opt 1,053.75 -> 1,364.33
29% raptor-tp6-wikipedia-firefox-cold fcp linux64-shippable opt 1,453.92 -> 1,876.50
29% raptor-tp6-google-firefox-cold windows7-32-shippable opt 308.64 -> 398.17
29% raptor-tp6-wikipedia-firefox-cold linux64-shippable opt 1,476.70 -> 1,903.07
29% raptor-tp6-facebook-firefox-cold macosx1014-64-shippable opt 1,377.11 -> 1,772.61
29% raptor-tp6-wikipedia-firefox-cold loadtime linux64-shippable opt 1,555.58 -> 2,001.00
28% raptor-tp6-facebook-redesign-firefox-cold fcp linux64-shippable-qr opt webrender 887.33 -> 1,136.42
28% raptor-tp6-google-firefox-cold windows10-64-shippable opt 301.86 -> 385.96
27% raptor-tp6-facebook-redesign-firefox-cold fcp linux64-shippable opt 843.67 -> 1,070.33
26% raptor-tp6-facebook-redesign-firefox-cold fcp macosx1014-64-shippable opt 1,298.08 -> 1,639.25
26% raptor-tp6-facebook-redesign-firefox-cold linux64-shippable-qr opt webrender 809.01 -> 1,020.09
26% raptor-tp6-wikipedia-firefox-cold fcp macosx1014-64-shippable opt 1,676.00 -> 2,104.33
26% raptor-tp6-wikipedia-firefox-cold linux64-shippable-qr opt webrender 1,588.68 -> 1,993.79
25% raptor-tp6-linkedin-firefox-cold loadtime macosx1014-64-shippable opt 3,594.42 -> 4,506.00
25% raptor-tp6-wikipedia-firefox-cold loadtime linux64-shippable-qr opt webrender 1,641.08 -> 2,055.83
25% raptor-tp6-facebook-firefox-cold loadtime macosx1014-64-shippable opt 3,279.50 -> 4,105.25
24% raptor-tp6-facebook-redesign-firefox-cold linux64-shippable opt 790.10 -> 982.49
24% raptor-tp6-google-firefox-cold linux64-shippable opt 329.74 -> 409.48
22% raptor-tp6-imgur-firefox-cold macosx1014-64-shippable opt 1,341.06 -> 1,636.57
22% raptor-tp6-wikipedia-firefox-cold fcp linux64-shippable-qr opt webrender 1,669.25 -> 2,033.50
22% raptor-tp6-yahoo-mail-firefox-cold fcp macosx1014-64-shippable opt 470.75 -> 572.08
22% raptor-tp6-wikipedia-firefox-cold fcp windows10-64-shippable-qr opt webrender 1,412.67 -> 1,716.33
21% raptor-tp6-instagram-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,189.25 -> 1,444.08
21% raptor-tp6-yahoo-mail-firefox-cold macosx1014-64-shippable opt 736.46 -> 892.00
20% raptor-tp6-linkedin-firefox-cold windows7-32-shippable opt 710.27 -> 854.88
20% raptor-tp6-wikipedia-firefox-cold macosx1014-64-shippable opt 1,816.01 -> 2,175.60
20% raptor-tp6-linkedin-firefox-cold windows10-64-shippable opt 713.59 -> 853.05
19% raptor-tp6-amazon-firefox-cold fcp linux64-shippable opt 661.25 -> 788.25
19% raptor-tp6-google-mail-firefox-cold linux64-shippable-qr opt webrender 257.01 -> 306.17
19% raptor-tp6-linkedin-firefox-cold linux64-shippable opt 779.56 -> 926.21
18% raptor-tp6-yahoo-mail-firefox-cold loadtime macosx1014-64-shippable opt 1,244.58 -> 1,474.67
18% raptor-tp6-yahoo-mail-firefox-cold loadtime linux64-shippable-qr opt webrender 917.83 -> 1,086.83
18% raptor-tp6-instagram-firefox-cold loadtime windows10-64-shippable opt 1,191.00 -> 1,409.75
18% raptor-tp6-linkedin-firefox-cold windows10-64-shippable-qr opt webrender 701.42 -> 827.62
18% raptor-tp6-linkedin-firefox-cold linux64-shippable-qr opt webrender 814.46 -> 960.19
18% raptor-tp6-google-mail-firefox-cold linux64-shippable opt 252.93 -> 297.76
18% raptor-tp6-yahoo-mail-firefox-cold loadtime windows10-64-shippable-qr opt webrender 859.00 -> 1,010.08
17% raptor-tp6-yahoo-mail-firefox-cold linux64-shippable-qr opt webrender 518.65 -> 609.17
17% raptor-tp6-yahoo-mail-firefox-cold loadtime windows10-64-shippable opt 871.25 -> 1,023.08
17% raptor-tp6-yahoo-mail-firefox-cold linux64-shippable opt 489.08 -> 573.62
17% raptor-tp6-imgur-firefox-cold fcp macosx1014-64-shippable opt 853.83 -> 997.17
17% raptor-tp6-yahoo-mail-firefox-cold loadtime windows7-32-shippable opt 872.42 -> 1,017.42
16% raptor-tp6-yahoo-mail-firefox-cold loadtime linux64-shippable opt 885.17 -> 1,029.08
16% raptor-tp6-amazon-firefox-cold fcp windows10-64-shippable-qr opt webrender 655.71 -> 761.67
16% raptor-tp6-yahoo-mail-firefox-cold windows10-64-shippable-qr opt webrender 485.35 -> 562.17
16% raptor-tp6-yahoo-mail-firefox-cold windows10-64-shippable opt 473.68 -> 548.22
16% raptor-tp6-google-mail-firefox-cold loadtime macosx1014-64-shippable opt 938.88 -> 1,084.83
15% raptor-tp6-yahoo-mail-firefox-cold windows7-32-shippable opt 476.86 -> 549.18
15% raptor-tp6-yahoo-mail-firefox-cold fcp linux64-shippable opt 289.92 -> 333.42
15% raptor-tp6-linkedin-firefox-cold loadtime linux64-shippable opt 2,393.67 -> 2,748.25
14% raptor-tp6-google-mail-firefox-cold windows10-64-shippable-qr opt webrender 250.08 -> 284.87
14% raptor-tp6-google-mail-firefox-cold windows7-32-shippable opt 267.45 -> 304.39
13% raptor-tp6-yahoo-mail-firefox-cold fcp linux64-shippable-qr opt webrender 334.08 -> 378.92
13% raptor-tp6-amazon-firefox-cold fcp linux64-shippable-qr opt webrender 726.08 -> 821.25
13% raptor-tp6-google-mail-firefox-cold loadtime linux64-shippable opt 588.00 -> 663.42
13% raptor-tp6-imgur-firefox-cold windows10-64-shippable-qr opt webrender 793.31 -> 892.75
13% raptor-tp6-wikipedia-firefox-cold loadtime macosx1014-64-shippable opt 2,129.92 -> 2,396.50
12% raptor-tp6-google-mail-firefox-cold windows10-64-shippable opt 270.75 -> 303.49
12% raptor-tp6-google-mail-firefox-cold loadtime linux64-shippable-qr opt webrender 599.92 -> 671.00
12% raptor-tp6-tumblr-firefox-cold loadtime macosx1014-64-shippable opt 2,515.04 -> 2,804.83
11% raptor-tp6-linkedin-firefox-cold loadtime linux64-shippable-qr opt webrender 2,619.92 -> 2,916.33
11% raptor-tp6-google-mail-firefox-cold loadtime windows7-32-shippable opt 625.58 -> 694.17
11% raptor-tp6-yahoo-mail-firefox-cold fcp windows10-64-shippable opt 287.08 -> 318.42
11% raptor-tp6-reddit-firefox-cold fcp macosx1014-64-shippable opt 428.42 -> 474.08
11% raptor-tp6-yahoo-mail-firefox-cold fcp windows7-32-shippable opt 289.58 -> 320.33
10% raptor-tp6-google-mail-firefox-cold loadtime windows10-64-shippable-qr opt webrender 621.67 -> 685.25
10% raptor-tp6-reddit-firefox-cold loadtime macosx1014-64-shippable opt 4,051.71 -> 4,456.33
10% raptor-tp6-amazon-firefox-cold fcp windows10-64-shippable opt 688.33 -> 755.67
10% raptor-tp6-yahoo-mail-firefox-cold fcp windows10-64-shippable-qr opt webrender 325.25 -> 356.92
9% raptor-tp6-twitch-firefox-cold macosx1014-64-shippable opt 341.48 -> 373.26
9% raptor-tp6-bing-firefox-cold windows10-64-shippable-qr opt webrender 231.43 -> 252.30
9% raptor-tp6-google-mail-firefox-cold loadtime windows10-64-shippable opt 646.08 -> 704.25
9% raptor-tp6-bing-firefox-cold fcp linux64-shippable opt 247.21 -> 269.08
9% raptor-tp6-amazon-firefox-cold fcp windows7-32-shippable opt 694.08 -> 755.42
8% raptor-tp6-twitch-firefox-cold windows10-64-shippable-qr opt webrender 282.42 -> 305.72
8% raptor-tp6-fandom-firefox-cold confidence macosx1014-64-shippable opt 95.00 -> 87.33
8% raptor-tp6-reddit-firefox-cold loadtime linux64-shippable-qr opt webrender 2,868.33 -> 3,097.25
8% raptor-tp6-linkedin-firefox-cold loadtime windows7-32-shippable opt 2,416.33 -> 2,607.67
8% raptor-tp6-twitch-firefox-cold fcp windows10-64-shippable-qr opt webrender 158.58 -> 171.08
8% raptor-tp6-bing-firefox-cold windows10-64-shippable opt 237.75 -> 256.33
8% raptor-tp6-bing-firefox-cold windows7-32-shippable opt 238.61 -> 257.08
8% raptor-tp6-reddit-firefox-cold loadtime windows10-64-shippable opt 2,734.92 -> 2,944.58
7% raptor-tp6-linkedin-firefox-cold loadtime windows10-64-shippable-qr opt webrender 2,423.54 -> 2,603.58
7% raptor-tp6-bing-firefox-cold fcp windows10-64-shippable-qr opt webrender 236.67 -> 254.00
7% raptor-tp6-facebook-firefox-cold loadtime linux64-shippable opt 2,692.17 -> 2,889.25
7% raptor-tp6-linkedin-firefox-cold loadtime windows10-64-shippable opt 2,460.00 -> 2,638.58
7% raptor-tp6-bing-firefox-cold linux64-shippable-qr opt webrender 249.59 -> 267.50
7% raptor-tp6-amazon-firefox-cold linux64-shippable opt 838.67 -> 897.51
7% raptor-tp6-bing-firefox-cold fcp linux64-shippable-qr opt webrender 257.92 -> 275.83
7% raptor-tp6-ebay-firefox-cold loadtime windows10-64-shippable opt 859.29 -> 918.00
7% raptor-tp6-reddit-firefox-cold loadtime linux64-shippable opt 2,810.92 -> 3,001.58
7% raptor-tp6-bing-firefox-cold linux64-shippable opt 250.42 -> 267.27
7% raptor-tp6-facebook-firefox-cold loadtime linux64-shippable-qr opt webrender 2,787.83 -> 2,974.75
7% raptor-tp6-twitch-firefox-cold linux64-shippable-qr opt webrender 262.22 -> 279.76
7% raptor-tp6-ebay-firefox-cold loadtime linux64-shippable opt 925.33 -> 986.83
7% raptor-tp6-ebay-firefox-cold loadtime linux64-shippable-qr opt webrender 945.67 -> 1,007.33
6% raptor-tp6-ebay-firefox-cold loadtime windows10-64-shippable-qr opt webrender 835.08 -> 888.25
6% raptor-tp6-bing-firefox-cold loadtime windows10-64-shippable-qr opt webrender 303.75 -> 322.33
6% raptor-tp6-instagram-firefox-cold windows10-64-shippable opt 371.46 -> 393.92
6% raptor-tp6-ebay-firefox-cold loadtime windows7-32-shippable opt 855.88 -> 906.67
6% raptor-tp6-bing-firefox-cold loadtime windows7-32-shippable opt 306.33 -> 323.58
6% raptor-tp6-reddit-firefox-cold loadtime windows7-32-shippable opt 2,720.38 -> 2,871.00
5% raptor-tp6-reddit-firefox-cold loadtime windows10-64-shippable-qr opt webrender 2,687.42 -> 2,831.17
5% raptor-tp6-bing-firefox-cold loadtime windows10-64-shippable opt 307.33 -> 322.58
5% raptor-tp6-bing-firefox-cold fcp windows10-64-shippable opt 257.00 -> 269.67
5% raptor-tp6-amazon-firefox-cold linux64-shippable-qr opt webrender 880.76 -> 921.22
5% raptor-tp6-bing-firefox-cold fcp windows7-32-shippable opt 258.04 -> 269.67
4% raptor-tp6-bing-firefox-cold loadtime linux64-shippable-qr opt webrender 321.08 -> 333.50
4% raptor-tp6-microsoft-firefox-cold fcp windows10-64-shippable opt 468.33 -> 486.42
4% raptor-tp6-google-mail-firefox-cold fcp linux64-shippable opt 201.92 -> 209.33
4% raptor-tp6-google-mail-firefox-cold fcp linux64-shippable-qr opt webrender 205.88 -> 213.25
4% raptor-tp6-ebay-firefox-cold linux64-shippable-qr opt webrender 714.72 -> 740.00
3% raptor-tp6-microsoft-firefox-cold fcp windows10-64-shippable-qr opt webrender 510.38 -> 527.33
3% raptor-tp6-google-mail-firefox-cold fcp windows7-32-shippable opt 245.29 -> 253.17
2% raptor-tp6-bing-firefox-cold loadtime linux64-shippable opt 325.58 -> 333.58

Improvements:

34% raptor-tp6-tumblr-firefox-cold fcp linux64-shippable opt 718.58 -> 477.67
32% raptor-tp6-bing-firefox-cold confidence macosx1014-64-shippable opt 64.83 -> 85.42
31% raptor-tp6-tumblr-firefox-cold linux64-shippable opt 983.24 -> 677.60
29% raptor-tp6-tumblr-firefox-cold linux64-shippable-qr opt webrender 1,013.57 -> 721.16
28% raptor-tp6-tumblr-firefox-cold windows10-64-shippable opt 907.20 -> 649.54
28% raptor-tp6-tumblr-firefox-cold fcp windows10-64-shippable opt 656.92 -> 473.42
26% raptor-tp6-tumblr-firefox-cold windows7-32-shippable opt 894.79 -> 658.40
26% raptor-tp6-tumblr-firefox-cold fcp windows7-32-shippable opt 651.67 -> 484.08
24% raptor-tp6-tumblr-firefox-cold fcp linux64-shippable-qr opt webrender 745.83 -> 569.25
20% raptor-tp6-tumblr-firefox-cold fcp macosx1014-64-shippable opt 1,050.75 -> 839.42
20% raptor-tp6-tumblr-firefox-cold macosx1014-64-shippable opt 1,436.00 -> 1,152.03
20% raptor-tp6-tumblr-firefox-cold windows10-64-shippable-qr opt webrender 893.09 -> 717.89
18% raptor-tp6-reddit-firefox-cold windows10-64-shippable-qr opt webrender 736.65 -> 600.45
18% raptor-tp6-instagram-firefox-cold loadtime linux64-shippable opt 1,297.58 -> 1,057.92
17% raptor-tp6-amazon-firefox-cold confidence windows10-64-shippable opt 74.00 -> 86.92
17% raptor-tp6-instagram-firefox-cold loadtime linux64-shippable-qr opt webrender 1,284.58 -> 1,063.25
17% raptor-tp6-reddit-firefox-cold windows10-64-shippable opt 741.15 -> 615.44
17% raptor-tp6-reddit-firefox-cold windows7-32-shippable opt 736.72 -> 612.74
16% raptor-tp6-apple-firefox-cold confidence windows10-64-shippable opt 75.92 -> 88.00
16% raptor-tp6-google-firefox-cold confidence windows10-64-shippable opt 74.50 -> 86.25
16% raptor-tp6-apple-firefox-cold confidence windows7-32-shippable opt 76.08 -> 88.00
15% raptor-tp6-instagram-firefox-cold loadtime macosx1014-64-shippable opt 2,019.67 -> 1,710.92
15% raptor-tp6-apple-firefox-cold confidence windows10-64-shippable-qr opt webrender 76.75 -> 88.00
15% raptor-tp6-reddit-firefox-cold linux64-shippable-qr opt webrender 762.62 -> 651.42
14% raptor-tp6-reddit-firefox-cold macosx1014-64-shippable opt 1,093.18 -> 935.78
14% raptor-tp6-tumblr-firefox-cold confidence windows10-64-shippable-qr opt webrender 85.75 -> 98.00
14% raptor-tp6-tumblr-firefox-cold confidence macosx1014-64-shippable opt 86.83 -> 99.00
14% raptor-tp6-google-mail-firefox-cold confidence windows7-32-shippable opt 78.50 -> 89.42
14% raptor-tp6-tumblr-firefox-cold confidence windows10-64-shippable opt 86.33 -> 98.00
13% raptor-tp6-amazon-firefox-cold loadtime windows7-32-shippable opt 1,531.67 -> 1,326.17
13% raptor-tp6-twitch-firefox-cold loadtime windows10-64-shippable opt 1,170.75 -> 1,014.67
13% raptor-tp6-tumblr-firefox-cold confidence windows7-32-shippable opt 86.42 -> 97.92
13% raptor-tp6-reddit-firefox-cold linux64-shippable opt 732.03 -> 634.90
13% raptor-tp6-twitch-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,166.46 -> 1,013.67
13% raptor-tp6-tumblr-firefox-cold confidence linux64-shippable-qr opt webrender 86.83 -> 98.00
13% raptor-tp6-twitch-firefox-cold loadtime windows7-32-shippable opt 1,155.67 -> 1,007.58
13% raptor-tp6-tumblr-firefox-cold confidence linux64-shippable opt 87.00 -> 98.00
13% raptor-tp6-google-firefox-cold confidence windows7-32-shippable opt 76.42 -> 86.00
12% raptor-tp6-amazon-firefox-cold loadtime windows10-64-shippable opt 1,518.25 -> 1,328.67
12% raptor-tp6-amazon-firefox-cold confidence linux64-shippable opt 75.17 -> 84.50
12% raptor-tp6-google-mail-firefox-cold confidence macosx1014-64-shippable opt 80.00 -> 89.83
12% raptor-tp6-apple-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,058.25 -> 936.50
11% raptor-tp6-bing-firefox-cold confidence linux64-shippable-qr opt webrender 83.50 -> 93.00
11% raptor-tp6-google-mail-firefox-cold confidence windows10-64-shippable opt 80.08 -> 89.08
10% raptor-tp6-amazon-firefox-cold loadtime linux64-shippable-qr opt webrender 1,674.67 -> 1,499.83
10% raptor-tp6-microsoft-firefox-cold confidence linux64-shippable-qr opt webrender 80.75 -> 89.17
10% raptor-tp6-twitch-firefox-cold loadtime linux64-shippable-qr opt webrender 1,216.33 -> 1,090.67
10% raptor-tp6-amazon-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,481.46 -> 1,329.00
10% raptor-tp6-bing-firefox-cold confidence linux64-shippable opt 83.92 -> 92.50
10% raptor-tp6-microsoft-firefox-cold confidence macosx1014-64-shippable opt 80.42 -> 88.50
10% raptor-tp6-microsoft-firefox-cold confidence linux64-shippable opt 80.58 -> 88.67
10% raptor-tp6-amazon-firefox-cold loadtime linux64-shippable opt 1,618.08 -> 1,456.00
10% raptor-tp6-bing-firefox-cold confidence windows10-64-shippable opt 83.00 -> 91.17
10% raptor-tp6-bing-firefox-cold confidence windows7-32-shippable opt 83.00 -> 91.17
10% raptor-tp6-bing-firefox-cold confidence windows10-64-shippable-qr opt webrender 83.33 -> 91.42
9% raptor-tp6-amazon-firefox-cold fcp macosx1014-64-shippable opt 1,306.58 -> 1,184.42
9% raptor-tp6-yahoo-mail-firefox-cold confidence windows7-32-shippable opt 73.08 -> 79.58
9% raptor-tp6-google-firefox-cold loadtime linux64-shippable-qr opt webrender 1,212.42 -> 1,105.00
9% raptor-tp6-yahoo-mail-firefox-cold confidence windows10-64-shippable-qr opt webrender 72.83 -> 79.08
9% raptor-tp6-yandex-firefox-cold confidence windows10-64-shippable-qr opt webrender 77.62 -> 84.25
9% raptor-tp6-imdb-firefox-cold confidence windows10-64-shippable opt 85.71 -> 93.00
8% raptor-tp6-imdb-firefox-cold confidence windows10-64-shippable-qr opt webrender 85.33 -> 92.50
8% raptor-tp6-yandex-firefox-cold confidence windows10-64-shippable opt 77.25 -> 83.67
8% raptor-tp6-microsoft-firefox-cold confidence windows7-32-shippable opt 72.67 -> 78.58
8% raptor-tp6-microsoft-firefox-cold confidence windows10-64-shippable-qr opt webrender 70.71 -> 76.42
8% raptor-tp6-microsoft-firefox-cold confidence windows10-64-shippable opt 71.88 -> 77.67
8% raptor-tp6-yandex-firefox-cold confidence linux64-shippable opt 78.50 -> 84.58
8% raptor-tp6-yandex-firefox-cold confidence windows7-32-shippable opt 77.96 -> 84.00
8% raptor-tp6-imdb-firefox-cold confidence windows7-32-shippable opt 86.12 -> 92.67
8% raptor-tp6-linkedin-firefox-cold confidence windows10-64-shippable-qr opt webrender 82.75 -> 89.00
7% raptor-tp6-linkedin-firefox-cold confidence windows7-32-shippable opt 82.83 -> 88.92
7% raptor-tp6-google-firefox-cold confidence linux64-shippable opt 80.33 -> 86.17
7% raptor-tp6-google-firefox-cold confidence windows10-64-shippable-qr opt webrender 79.25 -> 85.00
7% raptor-tp6-twitch-firefox-cold confidence windows10-64-shippable-qr opt webrender 88.21 -> 94.58
7% raptor-tp6-yahoo-mail-firefox-cold confidence windows10-64-shippable opt 72.67 -> 77.83
7% raptor-tp6-linkedin-firefox-cold confidence windows10-64-shippable opt 82.92 -> 88.75
7% raptor-tp6-amazon-firefox-cold macosx1014-64-shippable opt 1,369.15 -> 1,273.64
7% raptor-tp6-yandex-firefox-cold confidence linux64-shippable-qr opt webrender 78.92 -> 84.42
7% raptor-tp6-yandex-firefox-cold confidence macosx1014-64-shippable opt 78.58 -> 84.00
7% raptor-tp6-google-firefox-cold confidence macosx1014-64-shippable opt 80.17 -> 85.58
7% raptor-tp6-yandex-firefox-cold windows10-64-shippable opt 620.56 -> 579.00
7% raptor-tp6-yandex-firefox-cold linux64-shippable-qr opt webrender 672.78 -> 628.47
7% raptor-tp6-yahoo-mail-firefox-cold confidence linux64-shippable-qr opt webrender 72.67 -> 77.42
7% raptor-tp6-amazon-firefox-cold confidence macosx1014-64-shippable opt 79.42 -> 84.58
6% raptor-tp6-twitch-firefox-cold confidence macosx1014-64-shippable opt 89.17 -> 94.92
6% raptor-tp6-yahoo-mail-firefox-cold confidence macosx1014-64-shippable opt 75.17 -> 79.92
6% raptor-tp6-reddit-firefox-cold confidence windows10-64-shippable opt 89.50 -> 94.92
6% raptor-tp6-apple-firefox-cold confidence linux64-shippable opt 92.42 -> 98.00
6% raptor-tp6-yandex-firefox-cold fcp windows10-64-shippable opt 313.54 -> 294.92
5% raptor-tp6-apple-firefox-cold confidence linux64-shippable-qr opt webrender 93.00 -> 98.00
5% raptor-tp6-apple-firefox-cold linux64-shippable-qr opt webrender 625.53 -> 592.96
5% raptor-tp6-yahoo-mail-firefox-cold confidence linux64-shippable opt 73.38 -> 77.08
5% raptor-tp6-linkedin-firefox-cold confidence macosx1014-64-shippable opt 83.17 -> 87.33
5% raptor-tp6-twitch-firefox-cold confidence linux64-shippable-qr opt webrender 89.50 -> 93.92
5% raptor-tp6-apple-firefox-cold confidence macosx1014-64-shippable opt 93.42 -> 98.00
5% raptor-tp6-yandex-firefox-cold fcp windows10-64-shippable-qr opt webrender 285.12 -> 271.42
5% raptor-tp6-facebook-redesign-firefox-cold confidence windows7-32-shippable opt 86.00 -> 90.00
5% raptor-tp6-facebook-redesign-firefox-cold confidence windows10-64-shippable opt 86.00 -> 89.92
5% raptor-tp6-google-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,118.62 -> 1,068.25
4% raptor-tp6-yandex-firefox-cold fcp linux64-shippable-qr opt webrender 318.00 -> 303.75
4% raptor-tp6-twitch-firefox-cold confidence linux64-shippable opt 90.00 -> 94.00
4% raptor-tp6-yandex-firefox-cold loadtime linux64-shippable-qr opt webrender 1,768.42 -> 1,690.50
4% raptor-tp6-google-mail-firefox-cold confidence linux64-shippable-qr opt webrender 81.00 -> 84.50
4% raptor-tp6-reddit-firefox-cold confidence windows7-32-shippable opt 90.58 -> 94.50
4% raptor-tp6-reddit-firefox-cold confidence windows10-64-shippable-qr opt webrender 90.54 -> 94.33
4% raptor-tp6-facebook-redesign-firefox-cold confidence windows10-64-shippable-qr opt webrender 86.00 -> 89.42
4% raptor-tp6-instagram-firefox-cold linux64-shippable opt 386.32 -> 371.04
4% raptor-tp6-ebay-firefox-cold confidence linux64-shippable-qr opt webrender 94.50 -> 98.00
4% raptor-tp6-facebook-redesign-firefox-cold confidence linux64-shippable-qr opt webrender 85.92 -> 89.00
4% raptor-tp6-ebay-firefox-cold confidence macosx1014-64-shippable opt 94.67 -> 98.00
4% raptor-tp6-ebay-firefox-cold confidence windows7-32-shippable opt 94.67 -> 98.00
3% raptor-tp6-facebook-redesign-firefox-cold confidence linux64-shippable opt 86.00 -> 89.00
3% raptor-tp6-linkedin-firefox-cold confidence linux64-shippable opt 84.00 -> 86.92
3% raptor-tp6-ebay-firefox-cold confidence windows10-64-shippable-qr opt webrender 94.83 -> 98.00
3% raptor-tp6-instagram-firefox-cold confidence macosx1014-64-shippable opt 93.00 -> 96.00
3% raptor-tp6-ebay-firefox-cold confidence windows10-64-shippable opt 95.00 -> 98.00
3% raptor-tp6-linkedin-firefox-cold confidence linux64-shippable-qr opt webrender 84.00 -> 86.50
3% raptor-tp6-imdb-firefox-cold confidence linux64-shippable opt 88.79 -> 91.42
3% raptor-tp6-tumblr-firefox-cold loadtime linux64-shippable-qr opt webrender 1,760.08 -> 1,710.92
3% raptor-tp6-google-firefox-cold confidence linux64-shippable-qr opt webrender 82.75 -> 84.92
2% raptor-tp6-facebook-redesign-firefox-cold confidence macosx1014-64-shippable opt 87.08 -> 89.00
2% raptor-tp6-instagram-firefox-cold confidence linux64-shippable opt 94.00 -> 96.00
2% raptor-tp6-instagram-firefox-cold confidence linux64-shippable-qr opt webrender 94.00 -> 96.00
2% raptor-tp6-instagram-firefox-cold confidence windows10-64-shippable opt 94.00 -> 96.00
2% raptor-tp6-instagram-firefox-cold confidence windows10-64-shippable-qr opt webrender 94.00 -> 96.00
2% raptor-tp6-instagram-firefox-cold confidence windows7-32-shippable opt 94.00 -> 96.00
2% raptor-tp6-wikipedia-firefox-cold confidence windows10-64-shippable opt 97.96 -> 100.00
2% raptor-tp6-wikipedia-firefox-cold confidence linux64-shippable opt 98.00 -> 100.00
2% raptor-tp6-wikipedia-firefox-cold confidence linux64-shippable-qr opt webrender 98.00 -> 100.00
2% raptor-tp6-wikipedia-firefox-cold confidence macosx1014-64-shippable opt 98.00 -> 100.00
2% raptor-tp6-wikipedia-firefox-cold confidence windows10-64-shippable-qr opt webrender 98.00 -> 100.00
2% raptor-tp6-wikipedia-firefox-cold confidence windows7-32-shippable opt 98.00 -> 100.00

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26516

Status: RESOLVED → REOPENED
Flags: needinfo?(gmierz2)
Flags: needinfo?(dave.hunt)
Flags: needinfo?(aionescu)
Resolution: FIXED → ---

:bebe/:aionescu, if you can verify that all the recordings are working properly, then we should accept these changes as we are now testing a newer ("different") web page. I see that confidence has increased in most cases which is a good sign.

Flags: needinfo?(gmierz2)

(In reply to Greg Mierzwinski [:sparky] from comment #18)

:bebe/:aionescu, if you can verify that all the recordings are working properly, then we should accept these changes as we are now testing a newer ("different") web page. I see that confidence has increased in most cases which is a good sign.

I agree with Greg. If you're confident that there are no regressions related to how or what we're measuring thenwe should accept these changes as a new baseline.

Flags: needinfo?(dave.hunt)
See Also: → 1654190
Flags: needinfo?(aionescu)

I am confident.

Status: REOPENED → RESOLVED
Closed: 1 year ago1 year ago
Resolution: --- → FIXED

updated for desktop snapshots

You need to log in before you can comment on or make changes to this bug.