1689948 - Creating many small SharedArrayBuffers can easily exceed MaximumLiveMappedBuffers

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, task, P3)

Description

[Jan de Mooij [:jandem]]

It looks like all SharedArrayBuffers go through js::MapBufferMemory where we check against MaximumLiveMappedBuffers. This limit (1000 on most platforms) is easily exceeded when creating a large number of small SABs.

Someone reported this on Matrix because they ran into this.

Comment 1

[Lars T Hansen [:lth]]

We have several mechanisms that are supposed to work together to avoid OOMing prematurely - limits + vm checks + last-ditch GC - but this system never worked well, it was bolted on.

Random small SABs should not count against that limit I think; we care mostly about the large (6GB, soon to be 8GB) buffers?

Comment 2

[Jan de Mooij [:jandem]]

(In reply to Lars T Hansen [:lth] from comment #1)

Random small SABs should not count against that limit I think; we care mostly about the large (6GB, soon to be 8GB) buffers?

I believe the original limit was to prevent exhausting the relatively small number-of-mappings on Linux, because Wasm used different protection flags for different regions in the allocation (and so the kernel couldn't fuse these mappings). Later on this limit was also used to prevent address space exhaustion. Both of these problems don't apply to small SABs, as far as I can tell (?).

Lars, is there a reason we need to use mmap instead of malloc for plain SABs not allocated for Wasm memory? calloc is what we use for plain ArrayBuffers and it could also be more efficient (both memory usage and performance) when code is doing many small allocations.

Comment 3

[Jan de Mooij [:jandem]]

Lars and I discussed this a bit more on Matrix. To summarize, I think there are two pieces to look into:

1. Consider removing the mapped-buffer-limit or only keeping a crude limit for --fuzzing-safe if it bothers fuzzing. We should look at the history of this code to see what changed when, and for what reason.

2. For non-Wasm SharedArrayBuffer consider a mode where we use calloc instead of mmap. Lars isn't sure whether page alignment is required for SABs but maybe for asm.js.

Comment 4

[Lars T Hansen [:lth]]

It looks like we still support shared memory for asm.js. We could rip that out. (We could rip asm.js out for that matter.)

Comment 5

[Lars T Hansen [:lth]]

I take that back. It looks like there's vestigial code in AsmJS.cpp for shared memory, but I don't think shared memory's supported any more.

Comment 7

[Jan de Mooij [:jandem]]

Patches in bug 1778077 implement 2) from comment 3.

Comment 8

[Jan de Mooij [:jandem]]

Let's close this. Bug 1778077 fixed the problem for non-Wasm SABs and we also improved the max-buffer heuristic for Wasm in bug 1773225.