1879080 - Beta 123 Performance changes when we unset EARLY_BETA_OR_EARLIER

Status: RESOLVED WONTFIX

(Testing :: Performance, defect, P5)

Comment 1

[Florin Bilt]

== Change summary for alert #41346 (as of Tue, 06 Feb 2024 11:34:59 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
26%	Base Content Explicit	linux1804-64-shippable-qr	fission	9,710,266.67 -> 7,142,757.33
25%	Base Content Explicit	macosx1015-64-shippable-qr	fission	10,298,554.67 -> 7,701,520.00
24%	Base Content Explicit	windows11-64-2009-shippable-qr	fission	10,366,650.67 -> 7,849,829.33
13%	Base Content Resident Unique Memory	windows11-64-2009-shippable-qr	fission	10,232,661.33 -> 8,923,306.67
10%	Base Content Resident Unique Memory	linux1804-64-shippable-qr	fission	14,965,248.00 -> 13,474,816.00

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41346

Comment 2

[Florin Bilt]

== Change summary for alert #41348 (as of Tue, 06 Feb 2024 11:52:18 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
168%	microsoft LastVisualChange	windows10-64-shippable-qr	fission warm webrender	288.44 -> 774.28	Before/After
157%	microsoft LastVisualChange	linux1804-64-shippable-qr	fission warm webrender	321.89 -> 826.52	Before/After
152%	microsoft PerceptualSpeedIndex	windows10-64-shippable-qr	fission warm webrender	175.78 -> 442.98	Before/After
140%	microsoft LastVisualChange	macosx1015-64-shippable-qr	fission warm webrender	345.17 -> 829.64	Before/After
135%	microsoft PerceptualSpeedIndex	macosx1015-64-shippable-qr	fission warm webrender	193.42 -> 453.99	Before/After
117%	microsoft PerceptualSpeedIndex	linux1804-64-shippable-qr	fission warm webrender	221.55 -> 479.65	Before/After
86%	microsoft LastVisualChange	windows10-64-shippable-qr	cold fission webrender	474.89 -> 885.21	Before/After
81%	microsoft PerceptualSpeedIndex	macosx1015-64-shippable-qr	cold fission webrender	280.16 -> 506.67	Before/After
74%	microsoft LastVisualChange	macosx1015-64-shippable-qr	cold fission webrender	524.49 -> 912.61	Before/After
74%	microsoft PerceptualSpeedIndex	windows10-64-shippable-qr	cold fission webrender	307.43 -> 533.67	Before/After
55%	microsoft PerceptualSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	412.55 -> 639.41	Before/After
47%	microsoft LastVisualChange	linux1804-64-shippable-qr	cold fission webrender	735.75 -> 1,082.80	Before/After
19%	microsoft SpeedIndex	windows10-64-shippable-qr	fission warm webrender	189.97 -> 225.31	Before/After
15%	microsoft ContentfulSpeedIndex	windows10-64-shippable-qr	fission warm webrender	167.89 -> 192.83	Before/After
10%	microsoft SpeedIndex	windows10-64-shippable-qr	cold fission webrender	384.39 -> 422.82	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
34%	expedia fcp	macosx1015-64-shippable-qr	fission warm webrender	215.84 -> 141.80	Before/After
27%	expedia fcp	linux1804-64-shippable-qr	fission warm webrender	223.43 -> 163.84	Before/After
17%	amazon fcp	windows10-64-shippable-qr	fission warm webrender	83.32 -> 68.86	Before/After
16%	amazon FirstVisualChange	windows10-64-shippable-qr	bytecode-cached fission warm webrender	119.58 -> 100.50	Before/After
11%	yahoo-mail fcp	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	152.55 -> 136.41	Before/After
...	...	...	...	...	...
2%	google-mail FirstVisualChange	macosx1015-64-shippable-qr	fission warm webrender	815.05 -> 798.63	Before/After

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41348

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

This bug has been marked as a regression. Setting status flag for Nightly to affected.

Comment 4

[Andrej (:aglavic)]

== Change summary for alert #41339 (as of Tue, 06 Feb 2024 16:49:08 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
6%	about_preferences_basic	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	148.76 -> 158.34

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
9%	tp5o responsiveness	windows10-64-shippable-qr	e10s fission stylo webrender-sw	0.97 -> 0.88
9%	perf_reftest_singletons tiny-traversal-singleton.html	macosx1015-64-shippable-qr	e10s fission stylo webrender	712.86 -> 650.02
8%	a11yr	windows10-64-shippable-qr	e10s fission stylo webrender-sw	379.95 -> 350.99
8%	perf_reftest_singletons tiny-traversal-singleton.html	windows10-64-shippable-qr	e10s fission stylo webrender	738.50 -> 682.85
6%	cross_origin_pageload	windows10-64-shippable-qr	e10s fission stylo webrender-sw	249.14 -> 233.48
...	...	...	...	...
2%	sessionrestore_no_auto_restore	windows10-64-shippable-qr	e10s fission stylo webrender-sw	450.75 -> 441.25

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41339