Closed Bug 1649100 Opened 4 years ago Closed 4 years ago

3.63% Browsertime vismet regression on push 8aff011594bfdeb0a7d364ebf4b18ba755477319 (Tue June 23 2020)

Tracking

(firefox79 wontfix)

Status:

RESOLVED WONTFIX

Tracking Flags:

Tracking

Status

firefox79

---

wontfix

People

(Reporter: Bebe, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Florin Strugariu [:Bebe]

Reporter

Description

•

4 years ago

Perfherder has detected a browsertime performance regression from push 8aff011594bfdeb0a7d364ebf4b18ba755477319. As author of one of the patches included in that push, we need your help to address this regression.

== Change summary for alert #26322 (as of Wed, 24 Jun 2020 13:16:04 GMT) ==

Regressions:

41% reddit ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 2,959.08 -> 4,160.17
35% youtube-watch ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 1,229.46 -> 1,655.25
30% reddit ContentfulSpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt cold 1,534.67 -> 1,994.83
21% instagram ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 812.58 -> 983.17
11% ebay-kleinanzeigen-search ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 1,945.96 -> 2,162.17
10% reddit loadtime android-hw-g5-7-0-arm7-api-16-shippable opt cold 4,779.81 -> 5,242.58
7% youtube ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 1,042.71 -> 1,117.25
6% reddit LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 6,293.33 -> 6,701.58
6% reddit SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,399.50 -> 5,738.00
5% bing-search-restaurants ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 670.88 -> 707.25
5% youtube-watch LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 2,506.67 -> 2,622.08
4% bing-search-restaurants FirstVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 636.67 -> 661.33
3% imdb LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 10,301.67 -> 10,634.83

Improvements:

65% cnn videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 65% bbc videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 65% jianshu videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 65% microsoft-support videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 65% amazon-search videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 65% allrecipes videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% ebay-kleinanzeigen-search videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable 64% google-search-restaurants videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable 64% web-de videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% imdb videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% reddit videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% cnn-ampstories videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% facebook-cristiano videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% booking videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% ebay-kleinanzeigen videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% bing videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% google videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% instagram videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% youtube videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% youtube-watch videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% amazon videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% stackoverflow videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% wikipedia videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% espn videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% google-maps videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 64% bing-search-restaurants videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable 64% facebook videoRecordingStart android-hw-g5-7-0-arm7-api-16-shippable opt cold 21% allrecipes SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 18% allrecipes SpeedIndex android-hw-p2-8-0-android-aarch64-shippable opt cold 10% bing ContentfulSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 10% google-maps LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 7% facebook-cristiano videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% bing-search-restaurants videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable 7% web-de videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% jianshu videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% microsoft-support videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% amazon-search videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% youtube videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% ebay-kleinanzeigen-search videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable 7% instagram videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% youtube-watch videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% google videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% allrecipes videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% facebook videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% cnn-ampstories videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% ebay-kleinanzeigen videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% google-maps videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% imdb videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% reddit videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% cnn videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% google-search-restaurants videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable 7% amazon videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% stackoverflow videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% booking videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 7% espn videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 6% bbc videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 6% bing videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 6% wikipedia videoRecordingStart android-hw-p2-8-0-android-aarch64-shippable opt cold 2,080.00 -> 735.00
2,080.00 -> 735.42
2,080.00 -> 736.92
2,080.00 -> 736.92
2,080.00 -> 737.08
2,080.00 -> 738.00
opt cold 2,080.00 -> 738.58
opt cold 2,080.00 -> 738.67
2,080.00 -> 738.83
2,080.00 -> 739.00
2,080.00 -> 739.08
2,080.00 -> 739.25
2,080.00 -> 739.25
2,080.00 -> 739.58
2,080.00 -> 740.08
2,080.00 -> 740.25
2,080.00 -> 740.50
2,080.00 -> 740.50
2,080.00 -> 741.25
2,076.67 -> 740.33
2,073.33 -> 739.67
2,080.00 -> 742.58
2,080.00 -> 743.58
2,080.00 -> 743.83
2,080.00 -> 744.25
opt cold 2,076.67 -> 744.42
2,080.00 -> 748.17
5,536.92 -> 4,388.00
2,439.71 -> 1,989.75
1,172.21 -> 1,058.08
7,056.67 -> 6,380.00
2,040.00 -> 1,903.58
opt cold 2,040.00 -> 1,904.00
2,040.00 -> 1,904.42
2,040.00 -> 1,904.92
2,040.00 -> 1,904.92
2,040.00 -> 1,905.17
2,040.00 -> 1,905.25
opt cold 2,040.00 -> 1,905.58
2,040.00 -> 1,905.83
2,040.00 -> 1,905.75
2,040.00 -> 1,906.00
2,040.00 -> 1,906.17
2,040.00 -> 1,906.33
2,040.00 -> 1,906.50
2,040.00 -> 1,906.50
2,040.00 -> 1,906.50
2,040.00 -> 1,906.67
2,040.00 -> 1,906.58
2,040.00 -> 1,906.75
opt cold 2,040.00 -> 1,906.75
2,040.00 -> 1,906.92
2,040.00 -> 1,906.92
2,036.67 -> 1,904.17
2,040.00 -> 1,907.33
2,040.00 -> 1,907.67
2,040.00 -> 1,907.67
2,040.00 -> 1,907.75

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26322

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Greg Mierzwinski [:sparky]

Comment 1

•

4 years ago

We can resolve this as WONTFIX since these are caused by a temporary tooling change (a known shift to our baseline).

Alexandru Ionescu (needinfo me) [:alexandrui]

Updated

•

4 years ago

Severity: -- → S3

Status: NEW → RESOLVED

Closed: 4 years ago

Priority: -- → P1

Resolution: --- → WONTFIX

Ryan VanderMeulen [:RyanVM]

Updated

•

4 years ago

status-firefox79: affected → wontfix

You need to log in before you can comment on or make changes to this bug.

Bugzilla

Quick Search

3.63% Browsertime vismet regression on push 8aff011594bfdeb0a7d364ebf4b18ba755477319 (Tue June 23 2020)

Categories

(Testing :: Performance, defect, P1)

Tracking

(firefox79 wontfix)

People

(Reporter: Bebe, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Crash Data

Security

(public)

User Story

Description

Comment 1

Updated

Updated