1561529 - 3.45 - 66.35% raptor-tp6-sheets-firefox / raptor-tp6-slides-firefox regression on push f14d8d4a913ee7ef995f39c3da530969438a860d (Tue June 25 2019)

Status: RESOLVED WONTFIX

(Testing :: Raptor, defect, P1)

Description

[Florin Strugariu [:Bebe]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=f14d8d4a913ee7ef995f39c3da530969438a860d

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

66% raptor-tp6-sheets-firefox fcp macosx1014-64-shippable opt 428.54 -> 712.88
61% raptor-tp6-sheets-firefox macosx1014-64-shippable opt 511.66 -> 822.85
58% raptor-tp6-sheets-firefox fcp windows10-64-shippable-qr opt 262.71 -> 414.33
55% raptor-tp6-sheets-firefox fcp windows7-32-shippable opt 273.08 -> 421.96
53% raptor-tp6-sheets-firefox fcp windows10-64-shippable opt 274.21 -> 418.50
52% raptor-tp6-sheets-firefox windows10-64-shippable-qr opt 323.10 -> 491.11
51% raptor-tp6-sheets-firefox windows7-32-shippable opt 323.28 -> 486.60
50% raptor-tp6-sheets-firefox windows10-64-shippable opt 325.01 -> 488.09
49% raptor-tp6-sheets-firefox fcp linux64-shippable opt 262.75 -> 392.75
49% raptor-tp6-sheets-firefox fcp linux64-shippable-qr opt 272.46 -> 406.88
47% raptor-tp6-sheets-firefox linux64-shippable opt 310.63 -> 455.93
47% raptor-tp6-sheets-firefox linux64-shippable-qr opt 322.78 -> 473.64
33% raptor-tp6-sheets-firefox loadtime macosx1014-64-shippable opt 1,063.58 -> 1,409.21
27% raptor-tp6-sheets-firefox loadtime windows10-64-shippable-qr opt 724.54 -> 919.79
26% raptor-tp6-sheets-firefox loadtime windows10-64-shippable opt 726.79 -> 918.58
26% raptor-tp6-sheets-firefox loadtime linux64-shippable-qr opt 685.62 -> 866.21
26% raptor-tp6-sheets-firefox loadtime windows7-32-shippable opt 720.42 -> 906.88
25% raptor-tp6-sheets-firefox loadtime linux64-shippable opt 674.75 -> 843.71
15% raptor-tp6-slides-firefox fcp linux64-shippable-qr opt 750.79 -> 866.38
14% raptor-tp6-slides-firefox fcp linux64-shippable opt 713.17 -> 814.62
12% raptor-tp6-slides-firefox fcp windows10-64-shippable opt 797.67 -> 893.25
12% raptor-tp6-slides-firefox fcp windows10-64-shippable-qr opt 804.50 -> 898.33
10% raptor-tp6-slides-firefox fcp windows7-32-shippable opt 795.92 -> 874.46
3% raptor-tp6-slides-firefox linux64-shippable opt 726.38 -> 751.46

Improvements:

34% raptor-tp6-docs-firefox loadtime windows10-64-shippable opt 1,157.00 -> 759.46
34% raptor-tp6-docs-firefox loadtime windows7-32-shippable opt 1,105.46 -> 731.29
34% raptor-tp6-docs-firefox fcp windows10-64-shippable opt 1,070.50 -> 708.62
33% raptor-tp6-docs-firefox fcp windows7-32-shippable opt 1,037.33 -> 695.38
31% raptor-tp6-docs-firefox fcp windows10-64-shippable-qr opt 1,052.46 -> 728.42
31% raptor-tp6-docs-firefox loadtime windows10-64-shippable-qr opt 1,103.62 -> 766.29
29% raptor-tp6-docs-firefox loadtime linux64-shippable-qr opt 1,046.62 -> 741.79
29% raptor-tp6-docs-firefox fcp linux64-shippable opt 937.88 -> 664.92
29% raptor-tp6-docs-firefox fcp linux64-shippable-qr opt 989.25 -> 703.42
28% raptor-tp6-docs-firefox loadtime linux64-shippable opt 988.12 -> 708.12
27% raptor-tp6-docs-firefox fcp macosx1014-64-shippable opt 1,494.71 -> 1,086.21
27% raptor-tp6-docs-firefox loadtime macosx1014-64-shippable opt 1,582.96 -> 1,153.83
25% raptor-tp6-docs-firefox windows10-64-shippable opt 939.12 -> 700.13
25% raptor-tp6-docs-firefox windows7-32-shippable opt 915.58 -> 683.39
22% raptor-tp6-docs-firefox windows10-64-shippable-qr opt 915.56 -> 713.44
20% raptor-tp6-docs-firefox linux64-shippable opt 821.87 -> 655.52
19% raptor-tp6-docs-firefox linux64-shippable-qr opt 848.61 -> 690.95
18% raptor-tp6-docs-firefox macosx1014-64-shippable opt 1,308.78 -> 1,076.22

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=21594

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Florin Strugariu [:Bebe]]

This is a framework change that we won't fix