1910420 - 7.27 - 2.04% speedometer Flight-TodoMVC/CompletingAllItems/Sync / speedometer Flight-TodoMVC + 16 more (Linux, OSX, Windows) regression on Wed July 24 2024

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect, P1)

Description

[Florin Bilt]

Perfherder has detected a browsertime performance change from push 1848dbf727acb2d711f4c75ab6ad4e97a7941282.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
7%	speedometer Flight-TodoMVC/CompletingAllItems/Sync	linux1804-64-nightlyasrelease-qr	fission webrender	37.94 -> 40.70	Before/After
7%	speedometer Flight-TodoMVC/CompletingAllItems	linux1804-64-nightlyasrelease-qr	fission webrender	41.71 -> 44.51	Before/After
5%	speedometer Flight-TodoMVC/CompletingAllItems	windows11-64-shippable-qr	fission webrender	17.08 -> 18.00	Before/After
5%	speedometer Flight-TodoMVC/CompletingAllItems/Sync	linux1804-64-shippable-qr	fission webrender	37.36 -> 39.36	Before/After
5%	speedometer Flight-TodoMVC/CompletingAllItems	linux1804-64-shippable-qr	fission webrender	40.97 -> 43.16	Before/After
4%	speedometer Flight-TodoMVC/CompletingAllItems/Sync	windows11-64-shippable-qr	fission webrender	15.19 -> 15.77	Before/After
3%	speedometer Flight-TodoMVC/CompletingAllItems	macosx1015-64-nightlyasrelease-qr	fission webrender	36.16 -> 37.27	Before/After
3%	speedometer Flight-TodoMVC/CompletingAllItems/Sync	macosx1015-64-nightlyasrelease-qr	fission webrender	33.27 -> 34.28	Before/After
3%	speedometer Flight-TodoMVC/CompletingAllItems/Sync	macosx1015-64-shippable-qr	fission webrender	32.85 -> 33.78	Before/After
3%	speedometer Flight-TodoMVC/CompletingAllItems	macosx1015-64-shippable-qr	fission webrender	35.76 -> 36.71	Before/After
...	...	...	...	...	...
2%	speedometer EmberJS-Debug-TodoMVC/DeletingItems/Sync	macosx1400-64-shippable-qr	fission webrender	57.73 -> 59.15	Before/After
2%	speedometer EmberJS-Debug-TodoMVC/DeletingItems	macosx1400-64-shippable-qr	fission webrender	58.06 -> 59.47	Before/After
2%	speedometer Flight-TodoMVC/DeletingItems/Sync	macosx1015-64-shippable-qr	fission webrender	26.67 -> 27.31	Before/After
2%	speedometer Flight-TodoMVC	macosx1015-64-nightlyasrelease-qr	fission webrender	101.16 -> 103.26	Before/After
2%	speedometer Flight-TodoMVC	macosx1015-64-shippable-qr	fission webrender	99.75 -> 101.78	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
5%	speedometer AngularJS-TodoMVC/DeletingAllItems/Sync	linux1804-64-shippable-qr	fission webrender	118.10 -> 112.56	Before/After
5%	speedometer AngularJS-TodoMVC/DeletingAllItems	linux1804-64-shippable-qr	fission webrender	122.81 -> 117.20	Before/After
4%	speedometer Elm-TodoMVC/DeletingItems	linux1804-64-shippable-qr	fission webrender	68.36 -> 65.81	Before/After
4%	speedometer Elm-TodoMVC/DeletingItems/Sync	linux1804-64-shippable-qr	fission webrender	67.17 -> 64.72	Before/After
4%	speedometer AngularJS-TodoMVC/DeletingAllItems/Sync	linux1804-64-nightlyasrelease-qr	fission webrender	118.77 -> 114.49	Before/After
...	...	...	...	...	...
2%	speedometer EmberJS-Debug-TodoMVC/DeletingItems/Sync	windows11-64-shippable-qr	fission webrender	103.88 -> 101.79	Before/After

As author of one of the patches included in that push, we need your help to address this regression.
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 1517

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1907891

Comment 2

[Jan de Mooij [:jandem]]

Thanks for the report. Bug 1910544 might help a bit with the regressions.

Comment 3

[Jan de Mooij [:jandem]]

After bug 1910544 landed, a lot of these graphs are back to what they were before. Eg the EmberJS-Debug regression is fixed and it might now be a little better than it was before bug 1907891:

https://treeherder.mozilla.org/perfherder/graphs?timerange=1209600&series=autoland,5042023,1,13

I looked into the Flight-TodoMVC regression and I don't see anything obvious. It's possible we nursery-allocate more string characters now so we have an extra minor GC during this test. I'll investigate more but if that's the issue there's not a lot we can do.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1907891

Comment 6

[Jan de Mooij [:jandem]]

I did some experiments on Try for the Flight-TodoMVC test. The regression is largest on the Linux64 Nightly-as-release build and a hacky patch to disable nursery-allocation of string characters for js::StringBuffer makes Flight-TodoMVC/CompletingAllItems/Sync 5% faster. Other things I tried had no effect on this test.

This confirms that the issue is most likely that we changed where a nursery GC happens on Speedometer 2 because the nursery can fill up faster now. There isn't a simple fix for this that won't regress performance in other cases.

jonco is working on adding a different allocator for object slots/elements and that could potentially be used for string characters too, instead of allocating relatively long strings in the nursery.