Closed Bug 1697112 Opened 2 months ago Closed 2 months ago

4.55 - 18.83% apple ContentfulSpeedIndex ... tumblr SpeedIndex (windows10-64-shippable) regression on push 6ea4d69aa5c6c7064d3b4a195bf96617baa3aebf (Thu March 4 2021)

Categories

(Testing :: mozperftest, defect)

defect

Tracking

(firefox-esr78 unaffected, firefox86 unaffected, firefox87 unaffected, firefox88 affected)

RESOLVED WONTFIX
Tracking Status
firefox-esr78 --- unaffected
firefox86 --- unaffected
firefox87 --- unaffected
firefox88 --- affected

People

(Reporter: igoldan, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a browsertime performance regression from push 6ea4d69aa5c6c7064d3b4a195bf96617baa3aebf. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
19% apple ContentfulSpeedIndex windows10-64-shippable nocondprof warm 257.50 -> 306.00
18% apple LastVisualChange windows10-64-shippable nocondprof warm 616.83 -> 730.00
18% tumblr PerceptualSpeedIndex windows10-64-shippable nocondprof warm 336.83 -> 398.58
18% google-search ContentfulSpeedIndex windows10-64-shippable nocondprof warm 234.31 -> 276.67
18% google-search FirstVisualChange windows10-64-shippable nocondprof warm 234.31 -> 276.67
18% google-search LastVisualChange windows10-64-shippable nocondprof warm 234.31 -> 276.67
18% google-search PerceptualSpeedIndex windows10-64-shippable nocondprof warm 234.31 -> 276.67
18% google-search SpeedIndex windows10-64-shippable nocondprof warm 234.31 -> 276.67
18% reddit FirstVisualChange windows10-64-shippable nocondprof warm 201.14 -> 236.67
16% bing-search FirstVisualChange windows10-64-shippable nocondprof warm 201.23 -> 233.33
15% apple SpeedIndex windows10-64-shippable nocondprof warm 244.75 -> 282.58
15% apple FirstVisualChange windows10-64-shippable nocondprof warm 237.08 -> 273.33
15% outlook FirstVisualChange windows10-64-shippable nocondprof warm 234.29 -> 270.00
15% apple PerceptualSpeedIndex windows10-64-shippable nocondprof warm 249.33 -> 287.17
11% microsoft LastVisualChange windows10-64-shippable cold nocondprof 723.33 -> 800.00
10% tumblr SpeedIndex windows10-64-shippable nocondprof warm 278.67 -> 306.25
8% netflix ContentfulSpeedIndex windows10-64-shippable nocondprof warm 553.86 -> 598.58
7% netflix SpeedIndex windows10-64-shippable nocondprof warm 398.71 -> 425.83
7% reddit SpeedIndex windows10-64-shippable nocondprof warm 308.38 -> 329.25
6% netflix PerceptualSpeedIndex windows10-64-shippable nocondprof warm 535.43 -> 567.33
6% tumblr LastVisualChange windows10-64-shippable nocondprof warm 815.60 -> 863.33
6% reddit PerceptualSpeedIndex windows10-64-shippable nocondprof warm 354.93 -> 375.08
5% facebook PerceptualSpeedIndex windows10-64-shippable nocondprof warm 380.50 -> 400.17
5% tumblr ContentfulSpeedIndex windows10-64-shippable nocondprof warm 411.52 -> 430.25

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
23% google-docs ContentfulSpeedIndex windows10-64-shippable nocondprof warm 1,642.67 -> 1,261.42
19% reddit ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,151.93 -> 929.17
17% linkedin ContentfulSpeedIndex windows10-64-shippable nocondprof warm 1,379.50 -> 1,143.75
14% linkedin ContentfulSpeedIndex windows10-64-shippable cold nocondprof 2,052.42 -> 1,770.33
7% reddit loadtime windows10-64-shippable nocondprof warm 2,185.23 -> 2,033.88
6% netflix loadtime windows10-64-shippable nocondprof warm 511.78 -> 483.46
5% facebook fcp windows10-64-shippable cold nocondprof 1,017.05 -> 963.75
5% facebook windows10-64-shippable cold nocondprof 1,332.92 -> 1,263.16
5% facebook loadtime windows10-64-shippable cold nocondprof 3,038.72 -> 2,885.12
5% twitter ContentfulSpeedIndex windows10-64-shippable cold nocondprof 1,678.67 -> 1,594.33
5% twitch loadtime windows10-64-shippable nocondprof warm 772.86 -> 737.00
4% google-docs loadtime windows10-64-shippable nocondprof warm 918.08 -> 877.21
4% facebook loadtime windows10-64-shippable nocondprof warm 1,127.05 -> 1,080.42
4% twitter loadtime windows10-64-shippable nocondprof warm 637.08 -> 611.21
4% reddit loadtime windows10-64-shippable cold nocondprof 2,926.98 -> 2,809.21
4% tumblr loadtime windows10-64-shippable nocondprof warm 614.33 -> 592.42
3% facebook-redesign loadtime windows10-64-shippable nocondprof warm 504.08 -> 487.88
3% reddit windows10-64-shippable nocondprof warm 430.52 -> 417.06
3% wikipedia loadtime windows10-64-shippable nocondprof warm 1,017.56 -> 986.54
3% twitter ContentfulSpeedIndex windows10-64-shippable nocondprof warm 928.00 -> 899.75
3% imdb loadtime windows10-64-shippable nocondprof warm 904.40 -> 879.67
2% twitter windows10-64-shippable cold nocondprof 442.76 -> 433.00

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(gmierz2)

I guess this will most likely be a wontfix.

Yup, this is a wontfix.

Status: NEW → RESOLVED
Closed: 2 months ago
Flags: needinfo?(gmierz2)
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.