1807938 - Beta 109 Performance changes when we unset EARLY_BETA_OR_EARLIER

Status: RESOLVED WONTFIX

(Testing :: Performance, defect, P3)

Comment 1

[Andrej (:aglavic)]

== Change summary for alert #36646 (as of Thu, 29 Dec 2022 20:21:06 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
213%	glterrain	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	2.59 -> 8.08
188%	glterrain	linux1804-64-shippable-qr	e10s fission stylo webrender	2.91 -> 8.39
44%	glterrain	macosx1015-64-shippable-qr	e10s fission stylo webrender-sw	0.91 -> 1.30
36%	glterrain	macosx1015-64-shippable-qr	e10s fission stylo webrender	1.00 -> 1.36
25%	glterrain	windows10-64-shippable-qr	e10s fission stylo webrender-sw	1.09 -> 1.36

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
19%	perf_reftest_singletons style-sharing-style-attr.html	macosx1015-64-shippable-qr	e10s fission stylo webrender	4.21 -> 3.40
18%	perf_reftest_singletons style-sharing.html	macosx1015-64-shippable-qr	e10s fission stylo webrender	4.07 -> 3.34
16%	perf_reftest_singletons style-sharing.html	windows10-64-shippable-qr	e10s fission stylo webrender	4.79 -> 4.00
16%	perf_reftest_singletons style-sharing.html	linux1804-64-shippable-qr	e10s fission stylo webrender	5.45 -> 4.58
15%	perf_reftest_singletons style-sharing-style-attr.html	windows10-64-shippable-qr	e10s fission stylo webrender	4.85 -> 4.14
...	...	...	...	...
2%	tp5o	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	195.65 -> 190.98

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=36646

Comment 2

[Alex Finder]

== Change summary for alert #36630 (as of Wed, 28 Dec 2022 16:05:57 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
160%	microsoft PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	171.83 -> 446.75	Before/After
153%	microsoft PerceptualSpeedIndex	windows10-64-shippable-qr	fission warm webrender	185.08 -> 467.58	Before/After
147%	microsoft LastVisualChange	linux1804-64-shippable-qr	fission warm webrender	326.67 -> 806.67	Before/After
121%	microsoft LastVisualChange	windows10-64-shippable-qr	fission warm webrender	362.56 -> 800.00	Before/After
45%	microsoft LastVisualChange	linux1804-64-shippable-qr	cold fission webrender	710.00 -> 1,026.67	Before/After
42%	microsoft LastVisualChange	windows10-64-shippable-qr	cold fission webrender	644.70 -> 913.75	Before/After
26%	microsoft PerceptualSpeedIndex	windows10-64-shippable-qr	cold fission webrender	400.05 -> 503.42	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
13%	amazon FirstVisualChange	linux1804-64-shippable-qr	fission warm webrender	276.67 -> 240.00	Before/After
12%	amazon PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	289.75 -> 254.08	Before/After
12%	amazon ContentfulSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	290.00 -> 255.58	Before/After
12%	amazon SpeedIndex	linux1804-64-shippable-qr	fission warm webrender	286.00 -> 252.33	Before/After
7%	amazon fcp (geomean)	linux1804-64-shippable-qr	fission warm webrender	196.54 -> 183.16	Before/After
...	...	...	...	...	...
4%	amazon LastVisualChange (geomean)	linux1804-64-shippable-qr	fission warm webrender	1,110.09 -> 1,071.07	Before/After

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=36630

Comment 3

[Alex Finder]

== Change summary for alert #36653 (as of Thu, 29 Dec 2022 18:38:51 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
8%	Base Content Explicit	linux1804-64-shippable-qr	fission	7,593,797.33 -> 7,012,506.67
7%	Base Content Explicit	windows10-64-2004-shippable-qr	fission	8,194,544.00 -> 7,614,960.00

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=36653

Comment 4

[Alex Finder]

== Change summary for alert #36656 (as of Thu, 29 Dec 2022 21:55:53 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
21%	twitter FirstVisualChange	linux1804-64-shippable-qr	cold fission webrender	165.00 -> 200.00	Before/After
17%	instagram FirstVisualChange	linux1804-64-shippable-qr	cold fission webrender	240.00 -> 280.00	Before/After
14%	instagram fcp	macosx1015-64-shippable-qr	cold fission webrender	177.54 -> 202.83	Before/After
13%	instagram fcp	macosx1015-64-shippable-qr	bytecode-cached cold fission webrender	177.96 -> 201.29	Before/After
12%	instagram fcp	windows10-64-shippable-qr	cold fission webrender	190.68 -> 213.54	Before/After
10%	instagram LastVisualChange	windows10-64-shippable-qr	cold fission webrender	1,317.59 -> 1,455.50	Before/After
10%	instagram fcp	windows10-64-shippable-qr	bytecode-cached cold fission webrender	196.19 -> 216.29	Before/After
10%	instagram fcp	linux1804-64-shippable-qr	cold fission webrender	221.26 -> 243.42	Before/After
10%	instagram LastVisualChange	windows10-64-shippable-qr	bytecode-cached cold fission webrender	1,319.25 -> 1,449.75	Before/After
9%	instagram LastVisualChange	linux1804-64-shippable-qr	cold fission webrender	1,320.00 -> 1,440.00	Before/After
...	...	...	...	...	...
4%	facebook fcp	windows10-64-shippable-qr	cold fission webrender	831.88 -> 866.96	Before/After
4%	facebook PerceptualSpeedIndex	windows10-64-shippable-qr	cold fission webrender	840.82 -> 876.00	Before/After
4%	instagram PerceptualSpeedIndex	macosx1015-64-shippable-qr	bytecode-cached cold fission webrender	647.00 -> 673.92	Before/After
4%	facebook fcp	linux1804-64-shippable-qr	cold fission webrender	810.06 -> 843.33	Before/After
4%	facebook ContentfulSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	826.76 -> 860.08	Before/After

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=36656