1702703 - 3165.64 - 38.43% sunspider / wasm-godot-optimizing + 47 more (Linux, OSX, Windows) regression on Wed March 31 2021

Status: RESOLVED WONTFIX

(Testing :: Raptor, defect, P3)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a browsertime performance regression from push 5efe021dc3900b93ef93bd8026e5ab0a9c6c6e4f. As author of one of the patches included in that push, we need your help to address this regression.

== Change summary for alert #29549 (as of Fri, 02 Apr 2021 04:13:57 GMT) ==

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
3166%	sunspider		macosx1015-64-shippable	nocondprof	5.27 -> 171.96
3152%	sunspider		macosx1015-64-shippable-qr	nocondprof webrender	5.23 -> 170.18
3137%	sunspider		linux1804-64-shippable	nocondprof	6.77 -> 219.04
3106%	sunspider		linux1804-64-shippable-qr	nocondprof webrender	6.92 -> 221.86
3024%	sunspider		windows10-64-shippable-qr	nocondprof webrender	6.19 -> 193.32
2978%	sunspider		windows10-64-shippable	nocondprof	6.22 -> 191.50
1449%	wasm-misc-baseline		macosx1015-64-shippable	nocondprof	5,184.64 -> 80,313.69
1447%	wasm-misc-baseline		macosx1015-64-shippable-qr	nocondprof webrender	5,195.10 -> 80,342.02
1441%	wasm-misc-baseline		linux1804-64-shippable	nocondprof	6,424.54 -> 99,012.86
1426%	wasm-misc-baseline		linux1804-64-shippable-qr	nocondprof webrender	6,460.18 -> 98,599.80
1397%	wasm-misc-baseline		windows10-64-shippable	nocondprof	4,912.86 -> 73,538.68
1385%	wasm-misc-baseline		windows10-64-shippable-qr	nocondprof webrender	4,945.11 -> 73,434.03
1308%	wasm-misc		macosx1015-64-shippable	nocondprof	3,272.92 -> 46,093.98
1307%	wasm-misc		macosx1015-64-shippable-qr	nocondprof webrender	3,278.54 -> 46,118.30
1289%	wasm-misc		linux1804-64-shippable	nocondprof	3,527.61 -> 48,986.13
1273%	wasm-misc		linux1804-64-shippable-qr	nocondprof webrender	3,576.46 -> 49,120.68
1250%	wasm-misc		windows10-64-shippable	nocondprof	3,138.64 -> 42,362.48
1245%	wasm-misc		windows10-64-shippable-qr	nocondprof webrender	3,145.42 -> 42,294.81
1088%	wasm-misc-optimizing		macosx1015-64-shippable-qr	nocondprof webrender	3,883.24 -> 46,139.25
1080%	wasm-misc-optimizing		macosx1015-64-shippable	nocondprof	3,893.83 -> 45,962.20
1070%	wasm-misc-optimizing		linux1804-64-shippable	nocondprof	4,196.06 -> 49,077.77
1069%	wasm-misc-optimizing		linux1804-64-shippable-qr	nocondprof webrender	4,194.85 -> 49,047.25
1059%	wasm-misc-optimizing		windows10-64-shippable-qr	nocondprof webrender	3,640.37 -> 42,192.97
1058%	wasm-misc-optimizing		windows10-64-shippable	nocondprof	3,636.43 -> 42,118.17
1055%	wasm-misc-optimizing		windows10-64-shippable-qr	nocondprof webrender	3,662.02 -> 42,283.80
53%	wasm-godot		windows10-64-shippable	nocondprof	552.90 -> 844.87
52%	unity-webgl		windows7-32-shippable	nocondprof	3,773.41 -> 1,794.33
52%	wasm-godot		macosx1015-64-shippable-qr	nocondprof webrender	359.20 -> 547.30
52%	wasm-godot		windows10-64-shippable-qr	nocondprof webrender	552.14 -> 840.52
52%	unity-webgl		macosx1015-64-shippable	nocondprof	4,000.25 -> 1,915.92
52%	unity-webgl		windows10-64-shippable	nocondprof	4,067.01 -> 1,951.58
52%	unity-webgl		windows10-64-shippable-qr	nocondprof webrender	4,046.33 -> 1,945.83
52%	unity-webgl		macosx1015-64-shippable-qr	nocondprof webrender	4,025.20 -> 1,936.08
51%	unity-webgl		linux1804-64-shippable-qr	nocondprof webrender	3,262.64 -> 1,590.67
51%	unity-webgl		linux1804-64-shippable	nocondprof	3,250.16 -> 1,596.67
49%	wasm-godot		macosx1015-64-shippable	nocondprof	362.96 -> 542.15
49%	wasm-godot-optimizing		linux1804-64-shippable-qr	nocondprof webrender	889.09 -> 1,323.74
49%	wasm-godot-optimizing		linux1804-64-shippable	nocondprof	868.79 -> 1,291.31
48%	wasm-godot		linux1804-64-shippable	nocondprof	366.06 -> 542.66
48%	wasm-godot-baseline		macosx1015-64-shippable	nocondprof	342.49 -> 507.10
46%	wasm-godot-baseline		macosx1015-64-shippable-qr	nocondprof webrender	339.23 -> 494.99
46%	wasm-godot-baseline		windows10-64-shippable-qr	nocondprof webrender	493.19 -> 718.20
45%	wasm-godot-baseline		linux1804-64-shippable	nocondprof	329.45 -> 476.65
44%	wasm-godot-baseline		linux1804-64-shippable-qr	nocondprof webrender	343.10 -> 495.18
43%	wasm-godot		linux1804-64-shippable-qr	nocondprof webrender	389.48 -> 557.60
42%	wasm-godot-optimizing		macosx1015-64-shippable	nocondprof	889.06 -> 1,258.67
41%	wasm-godot-optimizing		macosx1015-64-shippable-qr	nocondprof webrender	886.58 -> 1,247.40
40%	wasm-godot-optimizing		windows10-64-shippable	nocondprof	1,123.32 -> 1,570.23
38%	wasm-godot-optimizing		windows10-64-shippable-qr	nocondprof webrender	1,134.70 -> 1,570.74

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
596%	stylebench		windows10-64-shippable-qr	nocondprof webrender	6.98 -> 48.60
544%	stylebench		windows10-64-shippable	nocondprof	7.19 -> 46.31
463%	stylebench		linux1804-64-shippable-qr	nocondprof webrender	7.63 -> 42.96
449%	stylebench		linux1804-64-shippable	nocondprof	7.73 -> 42.47
379%	stylebench		macosx1015-64-shippable-qr	nocondprof webrender	8.69 -> 41.67
364%	stylebench		macosx1015-64-shippable	nocondprof	8.96 -> 41.53

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=29549

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1701639

Comment 2

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Seems like Bug 1702689 didn't bring with it notable performance changes so I would close this as WONTFIX as the results prior this regression weren't valid anyway.

Comment 3

[Greg Mierzwinski [:sparky]]

Closing as WONTFIX based on comment #2.