1800806 - Maps and Sets are slower than native Objects or Arrays

Status: UNCONFIRMED

(Core :: JavaScript Engine, defect, P2)

Description

[Marco Trevisan (Treviño)]

Attachment: signalTestsMapsSets.js

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

I've been porting a signaling library that is part of gjs to use Map's and Set's for some operations that I was expecting to be faster in such cases (see https://gitlab.gnome.org/3v1n0/gjs/-/commits/signals-connect-after-v1), but that has not been the case (and in fact in the linked git log I eventually end up reverting such changes).

So I've moved them to a standalone runner for better testing, that I've attached here.

Run the script with an embedded mozjs (gjs or just use js, but may loose precision in performances analysis), and notice how implementation based on simple objects and arrays are faster than when using Map() or Set() objects.

Actual results:

In all the operations where I was expecting the Maps and/or Sets to provide some performance gain, we're actually getting slower results, leading us to keep using the basic objects.

See the details:

===================== dict =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 5.024 ms
emitAllSignals(nConnections=15000): 7787.400 ms
emitAllSignalsRandomly(nConnections=15000): 7197.527 ms
disconnectAllSignalsOneShot(nConnections=15000): 1.323 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 1.830 ms
disconnectAllSignalsRepeatedly(nConnections=5000): 1.022 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 1.462 ms
disconnectAllSignalsRandomly(nConnections=5000): 0.940 ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 14.733 ms
emitAllSignals(nConnections=500): 973.882 ms
emitAllSignalsRandomly(nConnections=500): 941.034 ms
disconnectAllSignalsRandomly(nConnections=50000): 13.089 ms

===================== map =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 9.126 ms
emitAllSignals(nConnections=15000): 18274.894 ms
emitAllSignalsRandomly(nConnections=15000): 18174.224 ms
disconnectAllSignalsOneShot(nConnections=15000): 1.693 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 2.682 ms
disconnectAllSignalsRepeatedly(nConnections=5000): 1.282 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 2.324 ms
disconnectAllSignalsRandomly(nConnections=5000): 1.464 ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 36.718 ms
emitAllSignals(nConnections=500): 2420.513 ms
emitAllSignalsRandomly(nConnections=500): 2422.254 ms
disconnectAllSignalsRandomly(nConnections=50000): 17.322 ms

===================== dict+array =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 16.931 ms
emitAllSignals(nConnections=15000): 17.322 ms
emitAllSignalsRandomly(nConnections=15000): 16.755 ms
disconnectAllSignalsOneShot(nConnections=15000): 1.302 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 4.274 ms
disconnectAllSignalsRepeatedly(nConnections=5000): 4504.640 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 3.592 ms
disconnectAllSignalsRandomly(nConnections=5000): 4570.978 ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 23.738 ms
emitAllSignals(nConnections=500): 2.580 ms
emitAllSignalsRandomly(nConnections=500): 2.443 ms
disconnectAllSignalsRandomly(nConnections=50000): 4676.315 ms

===================== map+set =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 35.327 ms
emitAllSignals(nConnections=15000): 20.758 ms
emitAllSignalsRandomly(nConnections=15000): 23.135 ms
disconnectAllSignalsOneShot(nConnections=15000): 1.017 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 7.305 ms
disconnectAllSignalsRepeatedly(nConnections=5000): 1725.360 ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 7.908 ms
disconnectAllSignalsRandomly(nConnections=5000): 1721.221 ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 33.797 ms
emitAllSignals(nConnections=500): 8.727 ms
emitAllSignalsRandomly(nConnections=500): 6.938 ms
disconnectAllSignalsRandomly(nConnections=50000): 3120.079 ms

For reference, the results I'm getting in node are way faster (if you ignore the dict ones, that is not what we care about here):

===================== dict =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 4.733ms
emitAllSignals(nConnections=15000): 49.581s
emitAllSignalsRandomly(nConnections=15000): 50.896s
disconnectAllSignalsOneShot(nConnections=15000): 3.988ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 1.372ms
disconnectAllSignalsRepeatedly(nConnections=5000): 3.097ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 0.551ms
disconnectAllSignalsRandomly(nConnections=5000): 1.331ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 5.298ms
emitAllSignals(nConnections=500): 8.377s
emitAllSignalsRandomly(nConnections=500): 8.400s
disconnectAllSignalsRandomly(nConnections=50000): 13.418ms

===================== map =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 5.045ms
emitAllSignals(nConnections=15000): 4.885s
emitAllSignalsRandomly(nConnections=15000): 4.766s
disconnectAllSignalsOneShot(nConnections=15000): 0.801ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 2.358ms
disconnectAllSignalsRepeatedly(nConnections=5000): 1.235ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 1.137ms
disconnectAllSignalsRandomly(nConnections=5000): 1.674ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 10.084ms
emitAllSignals(nConnections=500): 748.483ms
emitAllSignalsRandomly(nConnections=500): 719.87ms
disconnectAllSignalsRandomly(nConnections=50000): 13.807ms

===================== dict+array =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 13.167ms
emitAllSignals(nConnections=15000): 20.066ms
emitAllSignalsRandomly(nConnections=15000): 8.459ms
disconnectAllSignalsOneShot(nConnections=15000): 5.459ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 2.596ms
disconnectAllSignalsRepeatedly(nConnections=5000): 10.522s
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 2.851ms
disconnectAllSignalsRandomly(nConnections=5000): 10.357s
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 7.147ms
emitAllSignals(nConnections=500): 1.198ms
emitAllSignalsRandomly(nConnections=500): 2.103ms
disconnectAllSignalsRandomly(nConnections=50000): 6.983s

===================== map+set =====================
connectToManyDifferentSignals(nConnections=15000, nHandlers=1): 10.377ms
emitAllSignals(nConnections=15000): 20.171ms
emitAllSignalsRandomly(nConnections=15000): 7.961ms
disconnectAllSignalsOneShot(nConnections=15000): 0.821ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 6.84ms
disconnectAllSignalsRepeatedly(nConnections=5000): 690.084ms
--------------------
connectToManyDifferentSignals(nConnections=5000, nHandlers=1): 4.63ms
disconnectAllSignalsRandomly(nConnections=5000): 784.338ms
--------------------
connectToManyDifferentSignals(nConnections=500, nHandlers=100): 22.162ms
emitAllSignals(nConnections=500): 3.826ms
emitAllSignalsRandomly(nConnections=500): 3.22ms
disconnectAllSignalsRandomly(nConnections=50000): 1.755s

---

Also see some graphs are available at https://gitlab.gnome.org/3v1n0/gjs/-/snippets/3608 based on the results of the mentioned branch.

Expected results:

We were expecting maps/sets to be faster in most of insert and lookup operations while accepting a performance gap on iterating over them.

But in general, we were expecting better results than using normal Object's or Arrays.

Comment 1

[matti.roloux]

We've been running into this as well. Quite annoying to have to write browser detection code to switch between Maps and native objects.

Comment 2

[Mayank Bansal]

(In reply to matti.roloux from comment #1)

We've been running into this as well. Quite annoying to have to write browser detection code to switch between Maps and native objects.

If you have specific testcases that are slow in Firefox, please file a new bug(and attach the testcases) blocking this bug. Once this bug is fixed, we will re-evaluate that. If anything else is slow there, that can checked too.
Thanks!

Comment 3

[Jan de Mooij [:jandem]]

I'm currently working on Map and Set performance as part of bug 1851662 so this should get better the coming weeks.

I agree with Mayank that sharing tests representative of your workload would be useful.

Comment 4

[Mayank Bansal]

Can you please attach a testcase that reproduces the slowness?

Comment 5

[Mayank Bansal]

Matti: I have filed Bug 1953343 and Bug 1953345 for two performance issues I found on Seats.io