Open Bug 1878067 Opened 1 year ago Updated 1 year ago

Regressions/improvements caused by commits modifying release flags/bumping Firefox 123

Categories

(Testing :: Performance, defect, P5)

Product:
Testing
Component:
Performance
Version:
Firefox 123
Type:
defect

Tracking

(Not tracked)

People

(Reporter: bacasandrei, Unassigned)

Details

(Keywords: perf-alert)

We have encountered some performance alerts that have no bugs for the data points that caused them. Please find the alert summaries in the comments.

This bug was created for tracking purposes.

No longer blocks: 1692139
No longer depends on: 1686704

== Change summary for alert #41183 (awsy) (as of Thu, 25 Jan 2024 08:26:26 GMT) ==

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
35% Base Content Explicit linux1804-64-shippable-qr fission 7,152,997.33 -> 9,687,056.00
34% Base Content Explicit macosx1015-64-shippable-qr fission 7,743,674.67 -> 10,401,808.00
33% Base Content Explicit windows11-64-2009-shippable-qr fission 7,810,917.33 -> 10,365,968.00
16% Base Content Resident Unique Memory windows11-64-2009-shippable-qr fission 8,854,016.00 -> 10,253,994.67
12% Base Content Resident Unique Memory linux1804-64-shippable-qr fission 13,368,832.00 -> 14,959,616.00

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41183

== Change summary for alert #41158 (build_metrics) (as of Wed, 24 Jan 2024 17:17:11 GMT) ==

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
8% compiler_metrics num_static_constructors osx-aarch64-shippable aarch64 95.00 -> 103.00
8% compiler_metrics num_static_constructors osx-shippable 95.00 -> 103.00
3% installer size osx-shippable nightly 80,328,042.00 -> 82,798,231.17
2% installer size osx-aarch64-shippable aarch64 nightly 76,489,480.25 -> 77,944,965.42

Improvements:

Ratio Test Platform Options Absolute values (old vs new)
8% build times linux64-shippable instrumented taskcluster-projects/970387039909/machineTypes/c2-standard-16 1,945.80 -> 1,799.81

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41158

== Change summary for alert #41173 (browsertime) (as of Thu, 25 Jan 2024 01:12:41 GMT) ==

Regressions:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
219% espn largestContentfulPaint macosx1015-64-shippable-qr cold fission webrender 387.98 -> 1,237.44 Before/After
72% expedia fcp macosx1015-64-shippable-qr fission warm webrender 140.65 -> 241.65 Before/After
14% motionmark-animometer-ramp windows10-64-shippable-qr fission webrender 750.37 -> 645.56 Before/After
13% espn FirstVisualChange macosx1015-64-shippable-qr cold fission webrender 558.90 -> 632.47 Before/After
10% expedia largestContentfulPaint macosx1015-64-shippable-qr fission warm webrender 59.05 -> 65.10 Before/After
6% motionmark-htmlsuite-ramp windows10-64-shippable-qr fission webrender 609.96 -> 572.37 Before/After
6% yahoo-mail loadtime windows10-64-shippable-qr bytecode-cached fission warm webrender 256.72 -> 271.22 Before/After
6% motionmark-htmlsuite-ramp macosx1015-64-shippable-qr fission webrender 637.51 -> 601.67 Before/After
6% motionmark-animometer windows10-64-shippable-qr fission webrender 46.98 -> 44.36 Before/After
5% yahoo-mail loadtime macosx1015-64-shippable-qr bytecode-cached fission warm webrender 214.43 -> 225.40 Before/After
... ... ... ... ... ...
2% imgur loadtime windows10-64-shippable-qr fission warm webrender 463.30 -> 474.19 Before/After
2% nytimes SpeedIndex windows10-64-shippable-qr fission warm webrender 948.98 -> 970.14 Before/After
2% instagram largestContentfulPaint windows10-64-shippable-qr fission warm webrender 548.94 -> 560.97 Before/After
2% wikipedia LastVisualChange windows10-64-shippable-qr cold fission webrender 1,375.00 -> 1,403.87 Before/After
2% cnn LastVisualChange windows10-64-shippable-qr bytecode-cached fission warm webrender 909.07 -> 927.87 Before/After

Improvements:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
63% microsoft LastVisualChange macosx1015-64-shippable-qr fission warm webrender 831.50 -> 305.32 Before/After
63% microsoft LastVisualChange windows10-64-shippable-qr fission warm webrender 769.01 -> 287.85 Before/After
61% microsoft LastVisualChange linux1804-64-shippable-qr fission warm webrender 833.95 -> 324.18 Before/After
60% microsoft PerceptualSpeedIndex windows10-64-shippable-qr fission warm webrender 446.98 -> 177.84 Before/After
60% microsoft PerceptualSpeedIndex macosx1015-64-shippable-qr fission warm webrender 452.54 -> 181.36 Before/After
... ... ... ... ... ...
2% nytimes loadtime linux1804-64-shippable-qr fission warm webrender 976.39 -> 955.27 Before/After

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41173

== Change summary for alert #41156 (talos) (as of Wed, 24 Jan 2024 15:41:53 GMT) ==

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
10% tp5n nonmain_startup_fileio windows11-64-2009-shippable-qr e10s fission stylo webrender 842,359.00 -> 928,665.42
10% perf_reftest_singletons tiny-traversal-singleton.html macosx1015-64-shippable-qr e10s fission stylo webrender 645.35 -> 710.66
9% sessionrestore macosx1015-64-shippable-qr e10s fission stylo webrender-sw 801.08 -> 875.83
9% sessionrestore_no_auto_restore macosx1015-64-shippable-qr e10s fission stylo webrender-sw 817.17 -> 893.42
9% perf_reftest_singletons tiny-traversal-singleton.html windows10-64-shippable-qr e10s fission stylo webrender 659.70 -> 717.66
9% perf_reftest_singletons coalesce-2.html macosx1015-64-shippable-qr e10s fission stylo webrender 116.85 -> 126.83
8% perf_reftest_singletons coalesce-2.html windows10-64-shippable-qr e10s fission stylo webrender 113.87 -> 122.47
7% a11yr windows10-64-shippable-qr e10s fission stylo webrender 351.79 -> 377.13
7% perf_reftest_singletons coalesce-1.html macosx1015-64-shippable-qr e10s fission stylo webrender 153.95 -> 164.78
7% perf_reftest_singletons coalesce-1.html windows10-64-shippable-qr e10s fission stylo webrender 150.87 -> 161.40
... ... ... ... ...
2% ts_paint windows10-64-shippable-qr e10s fission stylo webrender 415.33 -> 425.67
2% ts_paint windows10-64-shippable-qr e10s fission stylo webrender-sw 396.62 -> 406.42
2% tp5n nonmain_startup_fileio windows11-64-2009-shippable-qr e10s fission stylo webrender-sw 843,847.58 -> 863,481.00
2% startup_about_home_paint windows10-64-shippable-qr e10s fission stylo webrender-sw 572.76 -> 584.33
2% tp5o_scroll windows10-64-shippable-qr e10s fission stylo webrender 1.71 -> 1.75

Improvements:

Ratio Test Platform Options Absolute values (old vs new)
17% offscreencanvas_webcodecs_main_2d_vp9 offscreencanvas_webcodecs_main_2d_vp9 Mean time across 100 frames: windows10-64-shippable-qr e10s fission stylo webrender 13.60 -> 11.24
17% offscreencanvas_webcodecs_worker_2d_vp9 offscreencanvas_webcodecs_worker_2d_vp9 Mean time across 100 frames: windows10-64-shippable-qr e10s fission stylo webrender 13.31 -> 11.07
16% offscreencanvas_webcodecs_main_2d_av1 offscreencanvas_webcodecs_main_2d_av1 Mean time across 100 frames: windows10-64-shippable-qr e10s fission stylo webrender 13.86 -> 11.69
15% offscreencanvas_webcodecs_worker_2d_av1 offscreencanvas_webcodecs_worker_2d_av1 Mean time across 100 frames: windows10-64-shippable-qr e10s fission stylo webrender 13.62 -> 11.58
13% perf_reftest_singletons slow-selector-2.html linux1804-64-shippable-qr e10s fission stylo webrender 0.90 -> 0.78
... ... ... ... ...
3% offscreencanvas_webcodecs_worker_2d_av1 offscreencanvas_webcodecs_worker_2d_av1 Mean time across 100 frames: linux1804-64-shippable-qr e10s fission stylo webrender 12.90 -> 12.56

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=41156

Severity: -- → S4
Priority: -- → P5
You need to log in before you can comment on or make changes to this bug.