7 - 24% raptor-tp6-slides-firefox-cold loadtime (windows10-64-shippable, windows10-64-shippable-qr, windows7-32-shippable) regression on push fc5808ee37a18acef7ff69eba861ccc8f24c3ae2 (Wed June 10 2020)
Categories
(Core :: Performance, defect)
Tracking
()
Tracking | Status | |
---|---|---|
firefox-esr68 | --- | unaffected |
firefox-esr78 | --- | unaffected |
firefox77 | --- | unaffected |
firefox78 | --- | unaffected |
firefox79 | + | wontfix |
People
(Reporter: Bebe, Unassigned)
References
(Regression)
Details
(Keywords: perf, perf-alert, regression)
Perfherder has detected a raptor performance regression from push fc5808ee37a18acef7ff69eba861ccc8f24c3ae2. As author of one of the patches included in that push, we need your help to address this regression.
== Change summary for alert #26217 (as of Mon, 15 Jun 2020 04:11:39 GMT) ==
Regressions:
24% raptor-tp6-apple-firefox-cold not-replayed linux64-shippable-qr opt 90.38 -> 112.00
17% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable-qr opt 1,930.25 -> 2,257.83
17% raptor-tp6-slides-firefox-cold loadtime windows10-64-shippable opt 1,985.67 -> 2,315.17
14% raptor-tp6-slides-firefox-cold loadtime windows7-32-shippable opt 1,973.83 -> 2,246.83
7% raptor-tp6-apple-firefox-cold not-replayed windows10-64-shippable opt 339.62 -> 363.50
7% raptor-tp6-apple-firefox-cold not-replayed windows7-32-shippable opt 340.12 -> 362.50
Improvements:
64% raptor-tp6-slides-firefox-cold fcp windows10-64-shippable opt 1,224.33 -> 440.25
63% raptor-tp6-slides-firefox-cold fcp windows7-32-shippable opt 1,201.00 -> 445.33
63% raptor-tp6-docs-firefox-cold fcp windows10-64-shippable opt 1,066.67 -> 399.83
62% raptor-tp6-docs-firefox-cold fcp windows7-32-shippable opt 1,050.00 -> 398.75
62% raptor-tp6-netflix-firefox-cold fcp windows10-64-shippable opt 559.00 -> 214.00
61% raptor-tp6-slides-firefox-cold fcp linux64-shippable-qr opt 1,215.92 -> 470.00
61% raptor-tp6-docs-firefox-cold fcp windows10-64-shippable-qr opt 1,001.58 -> 390.17
61% raptor-tp6-slides-firefox-cold fcp linux64-shippable opt 1,156.00 -> 450.92
61% raptor-tp6-netflix-firefox-cold fcp windows7-32-shippable opt 548.92 -> 216.17
60% raptor-tp6-slides-firefox-cold fcp windows10-64-shippable-qr opt 1,245.00 -> 498.17
60% raptor-tp6-slides-firefox-cold fcp macosx1014-64-shippable opt 1,908.50 -> 764.17
60% raptor-tp6-docs-firefox-cold fcp linux64-shippable-qr opt 1,058.08 -> 426.50
59% raptor-tp6-netflix-firefox-cold fcp windows10-64-shippable-qr opt 565.17 -> 229.50
58% raptor-tp6-docs-firefox-cold fcp linux64-shippable opt 1,021.33 -> 427.75
58% raptor-tp6-netflix-firefox-cold fcp linux64-shippable-qr opt 614.17 -> 259.58
57% raptor-tp6-netflix-firefox-cold fcp macosx1014-64-shippable opt 717.67 -> 309.33
56% raptor-tp6-netflix-firefox-cold fcp linux64-shippable opt 575.08 -> 254.67
55% raptor-tp6-docs-firefox-cold fcp macosx1014-64-shippable opt 1,552.25 -> 699.67
47% raptor-tp6-youtube-firefox-cold fcp linux64-shippable-qr opt 893.08 -> 470.33
47% raptor-tp6-youtube-firefox-cold fcp linux64-shippable opt 855.92 -> 455.42
43% raptor-tp6-youtube-firefox-cold fcp macosx1014-64-shippable opt 1,350.67 -> 765.17
39% raptor-tp6-docs-firefox-cold windows10-64-shippable opt 1,130.75 -> 685.62
39% raptor-tp6-docs-firefox-cold windows7-32-shippable opt 1,111.17 -> 679.52
39% raptor-tp6-netflix-firefox-cold windows10-64-shippable-qr opt 726.71 -> 445.11
38% raptor-tp6-netflix-firefox-cold windows10-64-shippable opt 723.56 -> 447.24
38% raptor-tp6-docs-firefox-cold windows10-64-shippable-qr opt 1,079.60 -> 667.63
38% raptor-tp6-netflix-firefox-cold windows7-32-shippable opt 721.60 -> 447.30
37% raptor-tp6-docs-firefox-cold linux64-shippable-qr opt 1,143.49 -> 724.32
36% raptor-tp6-docs-firefox-cold linux64-shippable opt 1,115.16 -> 714.92
35% raptor-tp6-netflix-firefox-cold macosx1014-64-shippable opt 968.10 -> 624.87
35% raptor-tp6-netflix-firefox-cold linux64-shippable-qr opt 789.85 -> 512.98
34% raptor-tp6-sheets-firefox-cold fcp windows10-64-shippable opt 770.33 -> 505.75
34% raptor-tp6-slides-firefox-cold windows10-64-shippable-qr opt 1,200.99 -> 791.50
34% raptor-tp6-netflix-firefox-cold linux64-shippable opt 767.47 -> 505.99
34% raptor-tp6-sheets-firefox-cold fcp windows7-32-shippable opt 771.17 -> 512.67
33% raptor-tp6-slides-firefox-cold windows10-64-shippable opt 1,218.34 -> 814.20
33% raptor-tp6-slides-firefox-cold linux64-shippable-qr opt 1,223.55 -> 817.94
33% raptor-tp6-slides-firefox-cold windows7-32-shippable opt 1,202.33 -> 805.29
32% raptor-tp6-slides-firefox-cold linux64-shippable opt 1,177.63 -> 804.00
31% raptor-tp6-slides-firefox-cold windows10-64-shippable-qr opt 1,147.17 -> 791.72
30% raptor-tp6-sheets-firefox-cold fcp linux64-shippable opt 758.83 -> 535.00
29% raptor-tp6-slides-firefox-cold macosx1014-64-shippable opt 1,817.24 -> 1,286.69
28% raptor-tp6-docs-firefox-cold macosx1014-64-shippable opt 1,549.93 -> 1,122.07
27% raptor-tp6-sheets-firefox-cold fcp windows10-64-shippable-qr opt 877.75 -> 637.50
25% raptor-tp6-docs-firefox-cold not-replayed windows10-64-shippable opt 72.58 -> 54.67
24% raptor-tp6-sheets-firefox-cold fcp linux64-shippable-qr opt 869.75 -> 657.08
23% raptor-tp6-sheets-firefox-cold windows7-32-shippable opt 895.57 -> 689.66
23% raptor-tp6-sheets-firefox-cold fcp macosx1014-64-shippable opt 1,100.25 -> 850.33
23% raptor-tp6-sheets-firefox-cold windows10-64-shippable opt 896.16 -> 693.20
22% raptor-tp6-docs-firefox-cold not-replayed windows10-64-shippable-qr opt 69.58 -> 54.50
20% raptor-tp6-sheets-firefox-cold linux64-shippable opt 892.15 -> 715.25
19% raptor-tp6-docs-firefox-cold not-replayed windows7-32-shippable opt 66.83 -> 54.17
17% raptor-tp6-sheets-firefox-cold windows10-64-shippable-qr opt 907.07 -> 749.66
17% raptor-tp6-fandom-firefox-cold fcp windows10-64-shippable opt 637.50 -> 529.67
17% raptor-tp6-sheets-firefox-cold linux64-shippable-qr opt 929.88 -> 774.17
15% raptor-tp6-fandom-firefox-cold fcp windows7-32-shippable opt 629.29 -> 534.83
14% raptor-tp6-fandom-firefox-cold fcp windows10-64-shippable-qr opt 696.17 -> 598.08
14% raptor-tp6-fandom-firefox-cold fcp linux64-shippable-qr opt 736.17 -> 633.08
13% raptor-tp6-fandom-firefox-cold fcp linux64-shippable opt 682.00 -> 594.42
13% raptor-tp6-youtube-firefox-cold linux64-shippable-qr opt 761.58 -> 665.58
12% raptor-tp6-youtube-firefox-cold linux64-shippable opt 747.29 -> 654.37
12% raptor-tp6-pinterest-firefox-cold windows10-64-shippable opt 881.83 -> 773.37
12% raptor-tp6-youtube-firefox-cold macosx1014-64-shippable opt 1,160.96 -> 1,018.73
12% raptor-tp6-sheets-firefox-cold macosx1014-64-shippable opt 1,313.34 -> 1,152.68
11% raptor-tp6-linkedin-firefox-cold windows10-64-shippable-qr opt 772.46 -> 688.52
11% raptor-tp6-docs-firefox-cold not-replayed linux64-shippable-qr opt 85.75 -> 76.75
10% raptor-tp6-linkedin-firefox-cold windows7-32-shippable opt 786.16 -> 704.97
10% raptor-tp6-pinterest-firefox-cold windows10-64-shippable-qr opt 837.73 -> 751.64
10% raptor-tp6-linkedin-firefox-cold linux64-shippable opt 857.84 -> 773.19
9% raptor-tp6-linkedin-firefox-cold windows10-64-shippable opt 780.56 -> 708.00
9% raptor-tp6-linkedin-firefox-cold linux64-shippable-qr opt 887.77 -> 806.70
9% raptor-tp6-fandom-firefox-cold windows10-64-shippable-qr opt 790.49 -> 718.73
9% raptor-tp6-fandom-firefox-cold windows10-64-shippable opt 782.12 -> 711.95
9% raptor-tp6-linkedin-firefox-cold loadtime windows7-32-shippable opt 2,640.17 -> 2,406.50
9% raptor-tp6-linkedin-firefox-cold loadtime windows10-64-shippable-qr opt 2,650.38 -> 2,421.42
8% raptor-tp6-fandom-firefox-cold windows7-32-shippable opt 776.56 -> 712.32
8% raptor-tp6-linkedin-firefox-cold loadtime linux64-shippable opt 2,607.00 -> 2,392.25
8% raptor-tp6-fandom-firefox-cold linux64-shippable-qr opt 866.53 -> 795.81
8% raptor-tp6-apple-firefox-cold windows10-64-shippable opt 593.26 -> 547.48
8% raptor-tp6-linkedin-firefox-cold loadtime windows10-64-shippable opt 2,647.00 -> 2,442.58
7% raptor-tp6-fandom-firefox-cold linux64-shippable opt 848.50 -> 787.14
7% raptor-tp6-linkedin-firefox-cold loadtime linux64-shippable-qr opt 2,803.58 -> 2,607.92
7% raptor-tp6-docs-firefox-cold not-replayed linux64-shippable opt 81.29 -> 75.67
6% raptor-tp6-slides-firefox-cold replayed linux64-shippable opt 2,122.25 -> 2,258.42
6% raptor-tp6-linkedin-firefox-cold macosx1014-64-shippable opt 1,169.49 -> 1,094.62
6% raptor-tp6-yahoo-news-firefox-cold fcp windows7-32-shippable opt 776.58 -> 727.58
6% raptor-tp6-yahoo-news-firefox-cold windows10-64-shippable opt 876.18 -> 825.21
6% raptor-tp6-yahoo-news-firefox-cold fcp linux64-shippable-qr opt 914.71 -> 861.75
6% raptor-tp6-yahoo-news-firefox-cold windows7-32-shippable opt 873.91 -> 824.02
6% raptor-tp6-slides-firefox-cold replayed linux64-shippable-qr opt 2,194.96 -> 2,318.00
5% raptor-tp6-sheets-firefox-cold loadtime linux64-shippable opt 1,594.17 -> 1,506.83
5% raptor-tp6-yahoo-news-firefox-cold linux64-shippable-qr opt 1,006.55 -> 952.59
5% raptor-tp6-paypal-firefox-cold fcp windows7-32-shippable opt 597.83 -> 566.33
5% raptor-tp6-sheets-firefox-cold loadtime windows10-64-shippable opt 1,634.00 -> 1,550.25
5% raptor-tp6-sheets-firefox-cold loadtime windows7-32-shippable opt 1,640.50 -> 1,557.42
5% raptor-tp6-sheets-firefox-cold loadtime linux64-shippable-qr opt 1,609.17 -> 1,528.92
5% raptor-tp6-yahoo-news-firefox-cold linux64-shippable opt 978.17 -> 933.12
5% raptor-tp6-pinterest-firefox-cold loadtime windows7-32-shippable opt 1,836.83 -> 1,752.25
4% raptor-tp6-yahoo-mail-firefox-cold loadtime linux64-shippable opt 932.42 -> 891.00
3% raptor-tp6-yahoo-mail-firefox-cold loadtime linux64-shippable-qr opt 946.21 -> 914.42
3% raptor-tp6-docs-firefox-cold replayed windows10-64-shippable opt 552.75 -> 570.50
3% raptor-tp6-yahoo-news-firefox-cold loadtime linux64-shippable opt 1,412.75 -> 1,367.75
3% raptor-tp6-docs-firefox-cold confidence windows10-64-shippable opt 87.96 -> 90.75
3% raptor-tp6-docs-firefox-cold confidence windows10-64-shippable-qr opt 88.17 -> 90.83
3% raptor-tp6-pinterest-firefox-cold loadtime linux64-shippable opt 1,962.67 -> 1,906.92
3% raptor-tp6-twitch-firefox-cold loadtime windows7-32-shippable opt 1,184.08 -> 1,150.83
3% raptor-tp6-docs-firefox-cold replayed windows10-64-shippable-qr opt 555.83 -> 570.50
3% raptor-tp6-yahoo-mail-firefox-cold windows7-32-shippable opt 485.80 -> 473.63
2% raptor-tp6-yahoo-news-firefox-cold loadtime linux64-shippable-qr opt 1,441.25 -> 1,406.08
2% raptor-tp6-twitch-firefox-cold loadtime linux64-shippable-qr opt 1,258.04 -> 1,228.75
2% raptor-tp6-docs-firefox-cold replayed windows7-32-shippable opt 558.50 -> 571.25
2% raptor-tp6-twitch-firefox-cold loadtime linux64-shippable opt 1,212.08 -> 1,184.83
2% raptor-tp6-yahoo-mail-firefox-cold linux64-shippable opt 498.46 -> 487.72
2% raptor-tp6-docs-firefox-cold confidence windows7-32-shippable opt 88.88 -> 90.75
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26217
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.
For more information on performance sheriffing please see our FAQ.
Reporter | ||
Updated•4 years ago
|
Updated•4 years ago
|
Reporter | ||
Updated•4 years ago
|
Comment 1•4 years ago
|
||
Set release status flags based on info from the regressing bug 1606652
Comment 2•4 years ago
|
||
Investigating the loadtime regression in raptor-tp6-slides-firefox-cold. However, I don't think there is anything actionable for the not-replayed regressions.
Updated•4 years ago
|
Comment 3•4 years ago
|
||
Hi Florian, can you please help generate profiles for the raptor-tp6-slides-firefox-cold regression before and after the change? I cannot reproduce this locally and not sure how to get a profile from CI. Thanks.
Reporter | ||
Comment 4•4 years ago
|
||
you can find the profiles here:
To stream line the process in the future you can generate the gecko profiles in treeherder:
- Select the job you want to generate the profiles for
- Click the 3 dots on the lower left side of the screen
- Click Create Gecko profiles
The profiles will be available in a short period of time as <job name>-p
Comment 5•4 years ago
|
||
Thanks Florin. The loadtime for https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit is actually the same in both cases, but the time between the above url and https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit?usp=sharing is the difference for the regression. With the patch, there are 4 setTimeout calls that coincidentally end up being scheduled after the /edit onload event which take a total time of 318ms: https://share.firefox.dev/2Ysgd6y
I think this is just an unlucky scheduling coincidence, and not much that can be done here other than possibly forcing setTimeout deferral until after on load, but that could have other negative consequences. I believe given the other improvements, including a 64% improvement in fcp, a small regression in loadtime for a unique case can be tolerated especially since the fcp improvement will have a bigger impact on visual completeness than loadtime will. For those reasons, I'm resolving this as wontfix. Please reopen if you believe otherwise.
Updated•4 years ago
|
Description
•