Closed Bug 1695685 Opened 5 years ago Closed 5 years ago

Use ffmpeg recorder on windows10 browsertime pageload tests.

Categories

(Testing :: Raptor, defect, P1)

Product:
Testing
Component:
Raptor
Type:
defect

Tracking

(firefox88 fixed)

Status:
RESOLVED FIXED
Milestone:
88 Branch
Tracking Flags:
Tracking Status
firefox88 --- fixed

People

(Reporter: sparky, Assigned: sparky)

Details

(Keywords: perf-alert)

Attachments

(1 file)

Bug 1695685 - Use the ffmpeg recorder in browsertime pageload tests. r?#perftest
48 bytes, text/x-phabricator-request
Details | Review

There's currently an issue with the Firefox window recorder on windows where there are white frames being painted after recording is started but before the orange frame is painted and this is causing issues with the visual metrics calculations. This bug is to use the ffmpeg recorder instead since it doesn't have this issue: https://treeherder.mozilla.org/jobs?repo=try&revision=861d88a4ea7efe423a9587c5d9378d3cd43a57d6

This patch switches us from using the Firefox window reorder to the ffmpeg recorder because of some issues that give bad visual metrics values.

Attachment #9206090 - Attachment description: Bug 1695685 - Use the ffmpeg recorder in browsertime pageload tests. r?#perftest → Bug 1695685 - Use the ffmpeg recorder in browsertime windows pageload tests. r?#perftest
Attachment #9206090 - Attachment description: Bug 1695685 - Use the ffmpeg recorder in browsertime windows pageload tests. r?#perftest → Bug 1695685 - Use the ffmpeg recorder in browsertime pageload tests. r?#perftest
Pushed by gmierz2@outlook.com: https://hg.mozilla.org/integration/autoland/rev/e321c447a380 Use the ffmpeg recorder in browsertime pageload tests. r=perftest-reviewers,AlexandruIonescu

https://hg.mozilla.org/mozilla-central/rev/e321c447a380

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox88: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 88 Branch

This impacted a lot of Windows 10 tests:

== Change summary for alert #29005 (as of Tue, 02 Mar 2021 19:24:28 GMT) ==

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
4132% wikipedia FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 1,692.83
3287% wikipedia ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 50.25 -> 1,702.08
3205% wikipedia SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 51.50 -> 1,702.17
2909% wikipedia PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 56.67 -> 1,705.00
1930% facebook FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 53.33 -> 1,082.92
1885% imdb FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 66.67 -> 1,323.25
1516% ebay FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 646.25
1355% imdb FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 581.83
1336% tumblr FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 574.42
1303% imdb SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 42.75 -> 599.92
1249% imdb ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 43.33 -> 584.50
1197% imdb PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 46.17 -> 599.00
1086% paypal FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 53.33 -> 632.67
1041% google-slides FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 66.67 -> 760.83
1023% imdb ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 121.00 -> 1,358.75
939% imgur FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 63.33 -> 657.83
931% yahoo-mail FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 412.50
931% fandom FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 56.67 -> 584.08
780% facebook SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 130.58 -> 1,149.00
744% imdb SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 169.75 -> 1,432.75
738% paypal FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 335.17
660% bing-search FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 304.17
659% google-search FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 46.67 -> 354.08
655% yandex FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 301.92
652% yahoo-mail FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 300.83
649% facebook PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 158.08 -> 1,183.42
646% microsoft FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 80.00 -> 597.00
605% microsoft SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 84.92 -> 598.33
604% netflix FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 40.00 -> 281.58
572% microsoft PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 89.33 -> 600.33
569% apple FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 267.67
568% pinterest FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 133.33 -> 890.83
563% microsoft ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 90.50 -> 600.42
518% linkedin FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 43.33 -> 268.00
507% google-sheets FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 120.00 -> 727.83
495% amazon FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 153.33 -> 911.75
493% google-docs FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 237.08
485% google-slides FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 233.83
460% apple SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 49.00 -> 274.42
455% yahoo-mail SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 80.08 -> 444.17
452% imdb PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 273.33 -> 1,509.67
426% apple PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 53.00 -> 279.00
424% office FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 213.33 -> 1,117.25
409% amazon ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 189.67 -> 965.42
407% google-sheets FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 46.67 -> 236.75
403% bing-search FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 201.33
403% ebay FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 201.00
403% fandom FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 201.00
402% tumblr FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 200.83
399% amazon PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 190.50 -> 950.25
384% yahoo-mail SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 65.67 -> 317.75
382% google-search PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 73.50 -> 354.17
367% google-docs FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 95.00 -> 443.83
365% apple ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 60.08 -> 279.33
360% amazon SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 206.92 -> 952.08
354% amazon FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 80.00 -> 363.00
345% paypal ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 173.42 -> 771.17
342% paypal ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 92.58 -> 408.83
338% tumblr SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 154.25 -> 676.25
335% apple FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 100.00 -> 435.42
334% google-search SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 81.58 -> 354.25
333% office SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 274.75 -> 1,189.08
314% office PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 291.42 -> 1,206.92
313% google-search ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 85.83 -> 354.17
309% bing-search ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 83.83 -> 342.58
306% amazon SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 95.33 -> 387.42
304% facebook-redesign FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 283.33 -> 1,145.00
297% paypal PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 204.67 -> 812.92
294% bing-search ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 55.58 -> 219.25
293% amazon PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 98.33 -> 386.08
286% facebook ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 348.42 -> 1,344.75
285% amazon ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 104.08 -> 400.42
271% fandom ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 60.08 -> 222.75
268% ebay ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 242.67 -> 893.75
265% reddit FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 90.00 -> 328.42
263% facebook-redesign PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 325.92 -> 1,181.67
257% google-sheets ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 248.75 -> 887.42
252% apple PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 134.50 -> 473.25
250% pinterest FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 58.33 -> 204.00
250% fandom PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 229.83 -> 803.67
246% facebook-redesign SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 343.92 -> 1,190.08
239% facebook-redesign ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 371.67 -> 1,259.92
234% office ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 462.25 -> 1,545.92
234% fandom ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 279.58 -> 933.25
234% paypal SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 252.50 -> 842.58
230% wikipedia PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 320.33 -> 1,056.17
230% wikipedia FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 320.00 -> 1,055.00
230% wikipedia SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 321.08 -> 1,058.42
230% wikipedia ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 320.42 -> 1,055.92
228% ebay PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 291.83 -> 957.58
222% yahoo-mail ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 172.67 -> 556.50
221% tumblr PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 248.50 -> 798.00
215% youtube FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 160.00 -> 504.08
214% apple ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 232.50 -> 729.58
212% apple SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 178.25 -> 556.75
209% google-search ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 76.67 -> 236.83
209% google-search FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 76.67 -> 236.83
209% google-search LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 76.67 -> 236.83
209% google-search PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 76.67 -> 236.83
209% google-search SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 76.67 -> 236.83
206% yahoo-mail ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 125.00 -> 382.58
204% bing-search SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 125.75 -> 382.42
201% wikipedia LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 503.33 -> 1,516.67
177% fandom SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 320.58 -> 888.58
177% ebay SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 357.83 -> 991.33
176% tumblr ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 341.17 -> 941.17
174% reddit FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 73.33 -> 201.25
167% yahoo-mail PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 231.92 -> 619.67
165% google-sheets PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 416.67 -> 1,104.50
163% yahoo-news FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 486.67 -> 1,279.33
160% google-sheets SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 365.00 -> 949.25
158% google-slides PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 464.00 -> 1,198.58
158% yahoo-news PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 516.17 -> 1,333.00
158% google-sheets LastVisualChange windows10-64-shippable nocondprof warm 2,390.00 -> 6,164.42
157% paypal PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 193.33 -> 496.08
154% imdb LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 366.67 -> 931.33
153% yahoo-news SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 541.25 -> 1,370.42
151% imdb LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 826.67 -> 2,075.17
150% pinterest ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 583.25 -> 1,460.83
150% google-sheets ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 124.50 -> 310.92
150% yahoo-mail PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 169.42 -> 422.75
144% yahoo-news ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 585.58 -> 1,426.42
136% google-slides ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 633.75 -> 1,497.25
136% google-slides SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 488.92 -> 1,153.83
132% microsoft FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 86.67 -> 201.00
129% microsoft SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 88.50 -> 202.92
129% wikipedia LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,313.33 -> 3,006.83
128% youtube LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 2,230.00 -> 5,091.17
128% microsoft PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 89.33 -> 203.92
127% microsoft ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 90.17 -> 204.67
125% paypal SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 242.67 -> 546.17
114% yandex ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 269.50 -> 575.58
112% tumblr SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 128.67 -> 272.25
106% outlook FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 261.67 -> 539.50
102% linkedin FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 295.00 -> 594.58
99% bing-search PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 352.75 -> 701.75
97% yahoo-news FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 200.00 -> 393.00
96% office FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 360.00 -> 703.83
94% office SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 383.92 -> 744.25
93% pinterest ContentfulSpeedIndex windows10-64-shippable nocondprof warm 305.42 -> 590.83
92% office PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 392.83 -> 753.58
92% yahoo-news PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 209.92 -> 402.33
90% office ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 501.75 -> 951.42
89% yandex PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 362.00 -> 685.83
88% google-sheets SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 221.58 -> 417.58
87% google-mail FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 663.33 -> 1,240.42
86% google-mail SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 668.17 -> 1,246.00
85% google-mail PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 681.33 -> 1,259.75
85% office LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,236.67 -> 2,284.42
84% fandom SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 176.50 -> 325.08
83% yandex SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 378.25 -> 691.58
83% youtube LastVisualChange windows10-64-shippable cold nocondprof 2,780.00 -> 5,080.50
81% google-mail ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 692.33 -> 1,256.25
77% facebook-redesign LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,070.00 -> 1,892.83
74% netflix ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 473.67 -> 825.50
74% youtube ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 810.08 -> 1,409.92
74% pinterest PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 956.00 -> 1,662.00
73% reddit SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 180.67 -> 312.33
71% tumblr ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 230.83 -> 394.58
71% fandom PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 215.00 -> 366.58
70% yahoo-news SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 253.17 -> 430.75
70% yahoo-news ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 260.75 -> 442.00
69% paypal LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 953.33 -> 1,607.92
68% ebay ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 288.83 -> 485.67
68% yandex FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 40.00 -> 67.00
67% reddit SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 387.67 -> 646.00
67% linkedin ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,176.83 -> 1,961.00
66% yandex ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 43.08 -> 71.67
65% fandom LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 720.00 -> 1,190.83
65% google-mail FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 836.67 -> 1,380.83
64% twitch ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 910.00 -> 1,494.75
64% youtube PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 839.08 -> 1,377.58
64% youtube LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 2,730.00 -> 4,470.92
63% netflix SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 376.58 -> 614.92
63% netflix PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 461.08 -> 751.83
63% office LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 830.00 -> 1,353.00
61% pinterest SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,120.25 -> 1,801.00
59% google-docs PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,114.38 -> 1,775.42
58% reddit ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 744.67 -> 1,180.00
57% facebook ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 310.58 -> 488.83
57% yandex SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 46.33 -> 72.92
57% amazon LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,343.33 -> 2,107.75
57% tumblr PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 208.42 -> 326.92
56% reddit PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 480.58 -> 751.00
54% google-docs ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,377.71 -> 2,126.17
51% google-mail LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,020.00 -> 1,544.00
51% google-mail ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,375.08 -> 2,079.33
51% microsoft LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 510.00 -> 770.33
50% youtube SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 858.25 -> 1,284.92
49% google-mail PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,124.17 -> 1,672.08
49% google-sheets LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 3,100.00 -> 4,609.25
48% facebook-redesign ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 334.42 -> 495.50
48% facebook-redesign FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 316.67 -> 468.83
48% reddit ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 252.17 -> 372.00
48% facebook LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 2,723.33 -> 4,016.92
47% microsoft LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 270.00 -> 396.83
47% ebay LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,420.00 -> 2,082.83
47% facebook-redesign PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 327.67 -> 480.25
46% reddit PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 240.50 -> 350.50
45% facebook-redesign SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 338.00 -> 491.75
45% google-sheets PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 357.17 -> 516.50
43% facebook SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 254.58 -> 364.33
42% outlook PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 827.92 -> 1,174.50
42% linkedin PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,464.29 -> 2,072.33
41% google-slides SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 382.75 -> 540.58
39% google-slides PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 379.33 -> 527.92
38% linkedin SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,319.00 -> 1,818.67
38% google-docs SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,211.17 -> 1,667.58
37% facebook PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 293.75 -> 403.00
37% yandex PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 73.58 -> 100.50
36% google-slides ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 527.00 -> 717.92
36% twitch PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 913.08 -> 1,239.50
35% paypal LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,000.00 -> 1,353.67
35% fandom LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 400.00 -> 540.83
33% google-mail SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,459.33 -> 1,947.50
33% amazon LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 913.33 -> 1,218.42
33% twitter ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,256.25 -> 1,672.00
33% twitch ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,143.67 -> 1,521.83
33% instagram ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 569.17 -> 756.92
32% twitch ContentfulSpeedIndex windows10-64-shippable nocondprof warm 1,001.92 -> 1,327.00
29% google-docs ContentfulSpeedIndex windows10-64-shippable nocondprof warm 1,265.67 -> 1,633.92
29% google-docs ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,812.50 -> 2,329.58
26% outlook FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 211.67 -> 267.50
26% twitch SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 780.46 -> 982.58
26% linkedin ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,062.92 -> 1,334.42
25% apple LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 500.00 -> 623.50
25% outlook ContentfulSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,496.21 -> 1,865.17
24% reddit ContentfulSpeedIndex windows10-64-shippable cold nocondprof 926.25 -> 1,153.08
24% google-slides LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 3,903.33 -> 4,848.50
24% tumblr LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 620.00 -> 769.67
24% instagram LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 680.00 -> 843.83
23% facebook-redesign LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 783.33 -> 964.83
23% pinterest LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 3,800.00 -> 4,675.83
23% tumblr LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,363.33 -> 1,676.33
23% linkedin PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,083.42 -> 1,330.92
23% linkedin ContentfulSpeedIndex windows10-64-shippable nocondprof warm 1,122.33 -> 1,375.92
22% yandex LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,658.33 -> 2,023.42
21% instagram SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 510.08 -> 617.83
21% instagram LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,080.00 -> 1,304.92
21% outlook SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 2,125.08 -> 2,565.00
20% twitter SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 1,376.83 -> 1,649.50
19% linkedin SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,024.42 -> 1,220.75
19% instagram SpeedIndex windows10-64-shippable-qr cold nocondprof webrender 936.17 -> 1,111.00
19% google-docs PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,104.75 -> 1,310.58
18% outlook SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 995.92 -> 1,177.92
18% instagram ContentfulSpeedIndex windows10-64-shippable cold nocondprof 672.42 -> 791.33
18% instagram PerceptualSpeedIndex windows10-64-shippable-qr cold nocondprof webrender 585.08 -> 688.33
17% yahoo-news LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 4,843.33 -> 5,683.67
17% yahoo-mail LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 856.67 -> 1,002.58
17% facebook LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,810.00 -> 2,117.83
15% yahoo-news LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 6,053.33 -> 6,984.92
15% linkedin ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,781.67 -> 2,052.92
15% google-mail LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 3,576.67 -> 4,120.08
15% ebay PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,109.58 -> 1,276.25
14% instagram ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 418.04 -> 478.58
14% twitter LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 2,326.67 -> 2,648.00
13% netflix windows10-64-shippable nocondprof warm 168.17 -> 190.54
13% linkedin LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 2,998.33 -> 3,396.00
13% twitch ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,193.83 -> 1,343.83
12% youtube SpeedIndex windows10-64-shippable cold nocondprof 1,353.04 -> 1,512.83
12% pinterest ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,368.46 -> 1,530.00
12% linkedin LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 2,200.00 -> 2,456.92
11% reddit LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,960.00 -> 2,179.33
11% yahoo-mail LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 1,520.00 -> 1,690.00
11% google-docs SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,030.17 -> 1,142.17
11% google-docs ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,384.50 -> 1,534.08
11% yahoo-news LastVisualChange windows10-64-shippable nocondprof warm 5,120.00 -> 5,669.58
10% twitter ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 822.25 -> 906.17
10% ebay SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 1,769.42 -> 1,941.25
8% outlook LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 2,981.67 -> 3,227.17
7% twitch fcp windows10-64-shippable-qr nocondprof warm webrender 56.35 -> 60.38
7% twitter SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 855.42 -> 914.42
7% imdb loadtime windows10-64-shippable-qr cold nocondprof webrender 1,944.88 -> 2,072.50
6% yandex LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 438.33 -> 465.58
6% twitter ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,583.17 -> 1,680.58
6% netflix LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,025.00 -> 1,087.08
6% netflix ContentfulSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 534.58 -> 566.83
6% google-docs LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 6,050.00 -> 6,413.50
6% netflix windows10-64-shippable cold nocondprof 406.00 -> 430.20
6% netflix loadtime windows10-64-shippable cold nocondprof 762.12 -> 805.21
6% facebook windows10-64-shippable cold nocondprof 1,269.03 -> 1,339.21
6% twitter LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 1,493.33 -> 1,575.92
6% facebook fcp windows10-64-shippable cold nocondprof 969.12 -> 1,022.50
5% facebook loadtime windows10-64-shippable cold nocondprof 2,890.21 -> 3,047.79
5% tumblr loadtime windows10-64-shippable nocondprof warm 582.54 -> 614.04
5% reddit loadtime windows10-64-shippable-qr nocondprof warm webrender 2,010.06 -> 2,113.21
5% netflix loadtime windows10-64-shippable nocondprof warm 488.12 -> 513.08
5% pinterest LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 3,716.67 -> 3,906.50
5% facebook-redesign windows10-64-shippable cold nocondprof 1,045.67 -> 1,099.04
5% google-sheets loadtime windows10-64-shippable-qr cold nocondprof webrender 1,804.00 -> 1,891.75
5% reddit loadtime windows10-64-shippable-qr cold nocondprof webrender 2,773.17 -> 2,903.71
4% facebook-redesign FirstVisualChange windows10-64-shippable nocondprof warm 481.67 -> 502.58
4% twitch loadtime windows10-64-shippable-qr cold nocondprof webrender 940.54 -> 981.33
4% facebook loadtime windows10-64-shippable nocondprof warm 1,085.54 -> 1,131.62
4% facebook-redesign loadtime windows10-64-shippable nocondprof warm 483.42 -> 503.58
4% twitch windows10-64-shippable-qr nocondprof warm webrender 199.51 -> 207.81
4% twitch loadtime windows10-64-shippable-qr nocondprof warm webrender 735.00 -> 764.00
4% facebook-redesign PerceptualSpeedIndex windows10-64-shippable nocondprof warm 493.83 -> 512.50
4% facebook-redesign SpeedIndex windows10-64-shippable nocondprof warm 504.12 -> 522.83
4% tumblr loadtime windows10-64-shippable-qr nocondprof warm webrender 553.88 -> 574.08
4% imdb fcp windows10-64-shippable-qr nocondprof warm webrender 536.88 -> 555.79
4% reddit windows10-64-shippable nocondprof warm 416.72 -> 431.35
3% reddit loadtime windows10-64-shippable cold nocondprof 2,834.04 -> 2,925.67
3% office fcp windows10-64-shippable-qr cold nocondprof webrender 1,038.21 -> 1,071.62
3% twitch LastVisualChange windows10-64-shippable-qr nocondprof warm webrender 5,880.00 -> 6,062.92
3% facebook loadtime windows10-64-shippable-qr nocondprof warm webrender 1,078.06 -> 1,111.38
3% twitter loadtime windows10-64-shippable nocondprof warm 616.10 -> 634.81
3% youtube windows10-64-shippable-qr nocondprof warm webrender 476.41 -> 490.75
3% twitter loadtime windows10-64-shippable-qr cold nocondprof webrender 1,736.60 -> 1,788.04
3% facebook-redesign windows10-64-shippable nocondprof warm 333.70 -> 342.95
3% google-mail loadtime windows10-64-shippable-qr nocondprof warm webrender 520.40 -> 534.46
2% youtube fcp windows10-64-shippable-qr nocondprof warm webrender 208.46 -> 213.62
2% apple loadtime windows10-64-shippable nocondprof warm 242.35 -> 247.50
2% tumblr windows10-64-shippable nocondprof warm 257.70 -> 263.16

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
91% google-search LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 4,216.67 -> 398.75
64% twitter FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 283.33 -> 100.75
64% youtube FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 650.00 -> 233.75
51% instagram FirstVisualChange windows10-64-shippable-qr nocondprof warm webrender 343.33 -> 167.00
50% twitter FirstVisualChange windows10-64-shippable-qr cold nocondprof webrender 466.67 -> 234.67
42% bing-search LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 2,556.67 -> 1,471.50
29% reddit LastVisualChange windows10-64-shippable-qr cold nocondprof webrender 4,016.67 -> 2,849.17
16% apple LastVisualChange windows10-64-shippable nocondprof warm 738.33 -> 616.67
16% google-search ContentfulSpeedIndex windows10-64-shippable nocondprof warm 280.00 -> 234.33
16% google-search FirstVisualChange windows10-64-shippable nocondprof warm 280.00 -> 234.33
16% google-search LastVisualChange windows10-64-shippable nocondprof warm 280.00 -> 234.33
16% google-search PerceptualSpeedIndex windows10-64-shippable nocondprof warm 280.00 -> 234.33
16% google-search SpeedIndex windows10-64-shippable nocondprof warm 280.00 -> 234.33
16% apple ContentfulSpeedIndex windows10-64-shippable nocondprof warm 307.12 -> 257.42
15% outlook FirstVisualChange windows10-64-shippable nocondprof warm 276.67 -> 234.17
15% google-slides FirstVisualChange windows10-64-shippable nocondprof warm 243.33 -> 206.50
15% tumblr PerceptualSpeedIndex windows10-64-shippable nocondprof warm 395.50 -> 337.42
14% apple SpeedIndex windows10-64-shippable nocondprof warm 284.17 -> 244.67
14% apple FirstVisualChange windows10-64-shippable nocondprof warm 275.00 -> 236.92
14% apple PerceptualSpeedIndex windows10-64-shippable nocondprof warm 288.71 -> 249.17
11% youtube SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 939.08 -> 835.00
9% google-docs FirstVisualChange windows10-64-shippable cold nocondprof 486.67 -> 443.17
8% tumblr SpeedIndex windows10-64-shippable nocondprof warm 303.25 -> 278.42
8% netflix ContentfulSpeedIndex windows10-64-shippable nocondprof warm 600.75 -> 554.42
8% twitch SpeedIndex windows10-64-shippable-qr nocondprof warm webrender 546.92 -> 505.08
7% netflix SpeedIndex windows10-64-shippable nocondprof warm 428.67 -> 398.33
7% netflix PerceptualSpeedIndex windows10-64-shippable nocondprof warm 571.96 -> 533.67
6% paypal FirstVisualChange windows10-64-shippable nocondprof warm 361.67 -> 338.25
6% twitch PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 459.62 -> 431.08
6% instagram PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 444.88 -> 419.33
5% paypal PerceptualSpeedIndex windows10-64-shippable nocondprof warm 528.79 -> 499.92
5% twitter PerceptualSpeedIndex windows10-64-shippable-qr nocondprof warm webrender 650.42 -> 615.25
5% paypal SpeedIndex windows10-64-shippable nocondprof warm 581.38 -> 552.58

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=29005

Keywords: perf-alert
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: