1859077 - 1%-10% regressions on multiple AWFY-Speedometer2 benchmarks around 5Oct2023

Status: RESOLVED WONTFIX

(Core :: Performance, defect)

Description

[Mayank Bansal]

Overall SP2 improved by 2.3%, but some sub-tests regressed by 10%.
See the following :

10% on AngularJS-TodoMVC/CompletingAllItems : https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912833,1768774334&series=mozilla-central,3735788,1,13&series=autoland,3912833,1,13&timerange=5184000&zoom=1696433187815,1696575785291,30.90598676962011,45.84501168980319

11% on AngularJS-TodoMVC/CompletingAllItems/Sync: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912835,1768774336&series=mozilla-central,3735790,1,13&series=autoland,3912835,1,13&timerange=5184000&zoom=1696393953062,1696595561171,28.180649614158803,41.396984312008414

6% on BackboneJS-TodoMVC/Adding100Items :https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912840,1768774341&series=mozilla-central,3735795,1,13&series=autoland,3912840,1,13&timerange=5184000&zoom=1696379235494,1696639852925,28.5977041715851,41.364903535877964

1.4% on Flight-TodoMVC/Adding100Items/Sync: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912882,1768774383&series=mozilla-central,3735837,1,13&series=autoland,3912882,1,13&timerange=5184000&zoom=1696379297658,1696644836230,32.157229084407575,42.22434136793893

1-10% on Preact-TodoMVC/CompletingAllItems/Async: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912904,1768923770&series=mozilla-central,3735859,1,13&series=autoland,3912904,1,13&timerange=5184000&zoom=1696349850403,1696979283556,2.5930239287170704,6.362746399496117

Some regression on Preact-TodoMVC/DeletingItems/Async : https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&series=mozilla-central,3735862,1,13&series=autoland,3912907,1,13&timerange=5184000&zoom=1696359754106,1697078724569,0.5545842701897903,2.103196871631286

11% on VanillaJS-TodoMVC/Adding100Items: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&series=mozilla-central,3735905,1,13&series=autoland,3912962,1,13&timerange=5184000&zoom=1696413908303,1696645035265,24.255559793958486,50.66381310776636

12% on VanillaJS-TodoMVC/Adding100Items/Sync : https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&selected=3912964,1768774454&series=mozilla-central,3735907,1,13&series=autoland,3912964,1,13&timerange=5184000&zoom=1696359844046,1696635231243,25.014624044427684,47.23099781722415

Regression range : https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=def7651a06fb17bb344eaf48cc6daa6047be722e&tochange=5cb270a4e6b6f0aa6cd6865b7f8daf75ad10263c

Which correspond to the recent gc/cc changes. Since these are timing changes (some tests improve, some regress), these regressions dont look too interesting. But filing in case there is anything that turns out to be an actual regression.

Comment 1

[Emilio Cobos Álvarez (:emilio)]

Olli can you look to see if there's something worth digging into here and close if not? Might be worth bisecting further from this range if we can?

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1856577

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

There isn't much to do here. Scheduling things differently does usually end up regressing something and improving other things.
And the actual results depend on the hardware.
More this kinds of changes are coming.