Bug 1675774 Comment 13 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

== Change summary for alert #27627 (as of Fri, 13 Nov 2020 02:29:23 GMT) ==

### Regressions:

| **Ratio** | **Suite** | **Test** | **Platform** | **Options** | **Absolute values (old vs new)**| 
|--|--|--|--|--|--| 
| 26% | raptor-tp6-twitter-firefox-cold | loadtime | windows7-32-shippable |  | 1,427.17 -> 1,794.33 |
| 25% | raptor-tp6-twitter-firefox-cold | loadtime | windows10-64-shippable |  | 1,441.58 -> 1,797.42 |
| 24% | raptor-tp6-twitter-firefox-cold | loadtime | windows10-64-shippable-qr | webrender | 1,441.83 -> 1,793.08 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows10-64-shippable-qr | webrender | 2,752.50 -> 2,979.92 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows10-64-shippable |  | 2,737.50 -> 2,961.75 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows7-32-shippable |  | 2,709.08 -> 2,927.67 |
| 6% | raptor-tp6-twitter-firefox-cold |  | windows10-64-shippable |  | 454.24 -> 483.12 |
| 6% | raptor-tp6-twitter-firefox-cold |  | windows7-32-shippable |  | 457.21 -> 483.95 |
| 5% | raptor-tp6-twitter-firefox-cold |  | windows10-64-shippable-qr | webrender | 450.16 -> 474.49 |
| 3% | raptor-tp6-reddit-firefox-cold |  | windows7-32-shippable |  | 644.61 -> 661.35 |

### Improvements:

| **Ratio** | **Suite** | **Test** | **Platform** | **Options** | **Absolute values (old vs new)**| 
|--|--|--|--|--|--| 
| 115% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows10-64-shippable |  | 45.17 -> 97.00 |
| 113% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 45.50 -> 97.00 |
| 111% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows7-32-shippable |  | 45.92 -> 97.00 |
| 34% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 71.83 -> 96.00 |
| 33% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows10-64-shippable |  | 72.00 -> 96.00 |
| 33% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows7-32-shippable |  | 72.17 -> 96.00 |
| 30% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows7-32-shippable |  | 72.00 -> 93.25 |
| 27% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 72.00 -> 91.58 |
| 27% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows10-64-shippable |  | 73.83 -> 93.58 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows10-64-shippable |  | 62.25 -> 78.08 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows7-32-shippable |  | 62.08 -> 77.58 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 62.75 -> 78.33 |
| 12% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows10-64-shippable |  | 76.58 -> 85.67 |
| 11% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows7-32-shippable |  | 79.00 -> 88.00 |
| 11% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 76.50 -> 85.08 |
| 11% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows7-32-shippable |  | 85.67 -> 94.92 |
| 10% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 85.83 -> 94.58 |
| 10% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows10-64-shippable |  | 85.92 -> 94.33 |
| 9% | raptor-tp6-netflix-firefox-cold | fcp | windows10-64-shippable-qr | webrender | 330.50 -> 301.67 |
| 8% | raptor-tp6-youtube-firefox-cold | fcp | windows10-64-shippable |  | 515.31 -> 476.08 |
| 7% | raptor-tp6-amazon-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 80.20 -> 85.58 |
| 7% | raptor-tp6-youtube-firefox-cold | fcp | windows10-64-shippable-qr | webrender | 493.58 -> 460.58 |
| 6% | raptor-tp6-youtube-firefox-cold | fcp | windows7-32-shippable |  | 515.35 -> 482.00 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows10-64-shippable |  | 85.33 -> 90.67 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 86.00 -> 91.25 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows7-32-shippable |  | 86.00 -> 91.25 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows10-64-shippable |  | 86.00 -> 91.17 |
| 6% | raptor-tp6-amazon-firefox-cold | replay-confidence | windows7-32-shippable |  | 80.90 -> 85.67 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows7-32-shippable |  | 85.08 -> 90.00 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 85.83 -> 90.58 |
| 5% | raptor-tp6-youtube-firefox-cold |  | windows10-64-shippable-qr | webrender | 968.49 -> 920.64 |
| 5% | raptor-tp6-youtube-firefox-cold |  | windows7-32-shippable |  | 989.73 -> 942.20 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows7-32-shippable |  | 92.42 -> 96.00 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 92.50 -> 96.00 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows10-64-shippable |  | 92.69 -> 96.17 |
| 4% | raptor-tp6-google-firefox-cold | replay-confidence | windows7-32-shippable |  | 83.00 -> 86.00 |
| 4% | raptor-tp6-microsoft-firefox-cold | replay-confidence | windows7-32-shippable |  | 74.89 -> 77.58 |
| 3% | raptor-tp6-google-firefox-cold | replay-confidence | windows10-64-shippable |  | 83.00 -> 85.75 |
| 3% | raptor-tp6-google-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 82.80 -> 85.50 |
| 3% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows10-64-shippable |  | 91.00 -> 93.33 |
| 3% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows7-32-shippable |  | 82.00 -> 84.08 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows10-64-shippable |  | 80.00 -> 82.00 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 80.00 -> 82.00 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows7-32-shippable |  | 80.00 -> 82.00 |
| 2% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows10-64-shippable |  | 82.00 -> 84.00 |
| 2% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 82.00 -> 84.00 |
| 2% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 90.84 -> 93.00 |
| 2% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows7-32-shippable |  | 90.80 -> 92.92 |
| 2% | raptor-tp6-facebook-redesign-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 87.00 -> 88.83 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows10-64-shippable |  | 98.00 -> 100.00 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 98.00 -> 100.00 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows7-32-shippable |  | 98.00 -> 100.00 |

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27627
(In reply to Andreea Pavel [:apavel] from comment #6)
> Backed out for performance regressions (bug 1676630)
> 
> Backout: https://hg.mozilla.org/integration/autoland/rev/88f6a6c2a328a0abc4421e5428b9b1c7a527c8b3

== Change summary for alert #27627 (as of Fri, 13 Nov 2020 02:29:23 GMT) ==

### Regressions:

| **Ratio** | **Suite** | **Test** | **Platform** | **Options** | **Absolute values (old vs new)**| 
|--|--|--|--|--|--| 
| 26% | raptor-tp6-twitter-firefox-cold | loadtime | windows7-32-shippable |  | 1,427.17 -> 1,794.33 |
| 25% | raptor-tp6-twitter-firefox-cold | loadtime | windows10-64-shippable |  | 1,441.58 -> 1,797.42 |
| 24% | raptor-tp6-twitter-firefox-cold | loadtime | windows10-64-shippable-qr | webrender | 1,441.83 -> 1,793.08 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows10-64-shippable-qr | webrender | 2,752.50 -> 2,979.92 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows10-64-shippable |  | 2,737.50 -> 2,961.75 |
| 8% | raptor-tp6-reddit-firefox-cold | loadtime | windows7-32-shippable |  | 2,709.08 -> 2,927.67 |
| 6% | raptor-tp6-twitter-firefox-cold |  | windows10-64-shippable |  | 454.24 -> 483.12 |
| 6% | raptor-tp6-twitter-firefox-cold |  | windows7-32-shippable |  | 457.21 -> 483.95 |
| 5% | raptor-tp6-twitter-firefox-cold |  | windows10-64-shippable-qr | webrender | 450.16 -> 474.49 |
| 3% | raptor-tp6-reddit-firefox-cold |  | windows7-32-shippable |  | 644.61 -> 661.35 |

### Improvements:

| **Ratio** | **Suite** | **Test** | **Platform** | **Options** | **Absolute values (old vs new)**| 
|--|--|--|--|--|--| 
| 115% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows10-64-shippable |  | 45.17 -> 97.00 |
| 113% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 45.50 -> 97.00 |
| 111% | raptor-tp6-paypal-firefox-cold | replay-confidence | windows7-32-shippable |  | 45.92 -> 97.00 |
| 34% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 71.83 -> 96.00 |
| 33% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows10-64-shippable |  | 72.00 -> 96.00 |
| 33% | raptor-tp6-instagram-firefox-cold | replay-confidence | windows7-32-shippable |  | 72.17 -> 96.00 |
| 30% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows7-32-shippable |  | 72.00 -> 93.25 |
| 27% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 72.00 -> 91.58 |
| 27% | raptor-tp6-twitch-firefox-cold | replay-confidence | windows10-64-shippable |  | 73.83 -> 93.58 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows10-64-shippable |  | 62.25 -> 78.08 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows7-32-shippable |  | 62.08 -> 77.58 |
| 25% | raptor-tp6-yahoo-mail-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 62.75 -> 78.33 |
| 12% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows10-64-shippable |  | 76.58 -> 85.67 |
| 11% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows7-32-shippable |  | 79.00 -> 88.00 |
| 11% | raptor-tp6-netflix-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 76.50 -> 85.08 |
| 11% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows7-32-shippable |  | 85.67 -> 94.92 |
| 10% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 85.83 -> 94.58 |
| 10% | raptor-tp6-reddit-firefox-cold | replay-confidence | windows10-64-shippable |  | 85.92 -> 94.33 |
| 9% | raptor-tp6-netflix-firefox-cold | fcp | windows10-64-shippable-qr | webrender | 330.50 -> 301.67 |
| 8% | raptor-tp6-youtube-firefox-cold | fcp | windows10-64-shippable |  | 515.31 -> 476.08 |
| 7% | raptor-tp6-amazon-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 80.20 -> 85.58 |
| 7% | raptor-tp6-youtube-firefox-cold | fcp | windows10-64-shippable-qr | webrender | 493.58 -> 460.58 |
| 6% | raptor-tp6-youtube-firefox-cold | fcp | windows7-32-shippable |  | 515.35 -> 482.00 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows10-64-shippable |  | 85.33 -> 90.67 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 86.00 -> 91.25 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows7-32-shippable |  | 86.00 -> 91.25 |
| 6% | raptor-tp6-yahoo-news-firefox-cold | replay-confidence | windows10-64-shippable |  | 86.00 -> 91.17 |
| 6% | raptor-tp6-amazon-firefox-cold | replay-confidence | windows7-32-shippable |  | 80.90 -> 85.67 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows7-32-shippable |  | 85.08 -> 90.00 |
| 6% | raptor-tp6-linkedin-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 85.83 -> 90.58 |
| 5% | raptor-tp6-youtube-firefox-cold |  | windows10-64-shippable-qr | webrender | 968.49 -> 920.64 |
| 5% | raptor-tp6-youtube-firefox-cold |  | windows7-32-shippable |  | 989.73 -> 942.20 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows7-32-shippable |  | 92.42 -> 96.00 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 92.50 -> 96.00 |
| 4% | raptor-tp6-youtube-firefox-cold | replay-confidence | windows10-64-shippable |  | 92.69 -> 96.17 |
| 4% | raptor-tp6-google-firefox-cold | replay-confidence | windows7-32-shippable |  | 83.00 -> 86.00 |
| 4% | raptor-tp6-microsoft-firefox-cold | replay-confidence | windows7-32-shippable |  | 74.89 -> 77.58 |
| 3% | raptor-tp6-google-firefox-cold | replay-confidence | windows10-64-shippable |  | 83.00 -> 85.75 |
| 3% | raptor-tp6-google-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 82.80 -> 85.50 |
| 3% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows10-64-shippable |  | 91.00 -> 93.33 |
| 3% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows7-32-shippable |  | 82.00 -> 84.08 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows10-64-shippable |  | 80.00 -> 82.00 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 80.00 -> 82.00 |
| 3% | raptor-tp6-pinterest-firefox-cold | replay-confidence | windows7-32-shippable |  | 80.00 -> 82.00 |
| 2% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows10-64-shippable |  | 82.00 -> 84.00 |
| 2% | raptor-tp6-yandex-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 82.00 -> 84.00 |
| 2% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 90.84 -> 93.00 |
| 2% | raptor-tp6-imdb-firefox-cold | replay-confidence | windows7-32-shippable |  | 90.80 -> 92.92 |
| 2% | raptor-tp6-facebook-redesign-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 87.00 -> 88.83 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows10-64-shippable |  | 98.00 -> 100.00 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows10-64-shippable-qr | webrender | 98.00 -> 100.00 |
| 2% | raptor-tp6-outlook-firefox-cold | replay-confidence | windows7-32-shippable |  | 98.00 -> 100.00 |

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27627

Back to Bug 1675774 Comment 13