1756030 - Photoshop, LibreOffice wasm compilation OOMs with devtools open

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, defect, P2)

Description

[Ryan Hunt [:rhunt]]

STR:

1. Open up devtools
2. Navigate to photoshop
3. Observe 'Module.instantiateWasm callback failed with error: out of memory' in console
4. Errors cascade

This is likely an issue with debug code (baseline) taking up too much space.

Comment 1

[Lars T Hansen [:lth]]

When running without the devtools, we see this for the main allocation:

Requesting 216268800 bytes of executable memory

(in a sea of 64KB requests and a few others) which is PRETTY DARN GOOD as this is baseline code and the multiplier for baseline x64 code is 3.5 and 64MB * 3.5 = 229MB. A little later we see

Requesting 192479232 bytes of executable memory

which is the Ion code, which is a little high, as the estimate would be 161MB.

With the devtools open, and the fix for bug 1715459 applied, I initially I see this:

Requesting 387055616 bytes of executable memory

which is a lot but should perhaps be manageable. But then things continue:

Requesting 387055616 bytes of executable memory
Requesting 387055616 bytes of executable memory
Requesting 387055616 bytes of executable memory
Requesting 387055616 bytes of executable memory
Requesting 387055616 bytes of executable memory
 ==> FAILED @ 1
Requesting 387055616 bytes of executable memory
 ==> FAILED @ 1
Requesting 387055616 bytes of executable memory
 ==> FAILED @ 1
Requesting 387055616 bytes of executable memory
 ==> FAILED @ 1

(where @1 is just the first nullptr return from ProcessExecutableMemory::allocate) and while this looks a little sinister, I think what's happening is that we try to clone the debug code for every thread because every thread will need to control its own breakpoints. I will try to verify that.

Comment 2

[Lars T Hansen [:lth]]

The initial failures are all problems with cloning. On my machine I see four successful clones before we start failing; at this point we should be close to 2GB so this is totally reasonable. Then a bunch of clones fail. Then it becomes a little chaotic; it could look like the application is trying to restart compilation, but of course this all fails, and then there's a loop of compile/fail, with some cloning failures.

Obviously not creating debug code when the console is open will fix this, but then when the debugger is opened we will be back to the problem, and we need to think about how to solve it.

The debug code we generate is patchable: a NOP is patched to a call. This is not the only option available. The call could be hardwired (say, to a location in TLS or to a location in a table or to a handler that itself figures out what's going on) and in this case the code would not need to be cloned. The main risk here is that the debug code would be larger and slower, but almost everything beats having the current problem, where the code doesn't work at all. If we expect applications to have increasingly large numbers of threads (I saw one the other week with 23 threads, and that too was a large program) then this problem will only get worse.

Comment 3

[Lars T Hansen [:lth]]

Piling on:

LibreOffice limited demo (https://wasm-test.libreoffice.org/): 600MB of debug code, fails on third clone
LibreOffice less limited demo (https://lab.allotropia.de/wasm/): 755MB of debug code, fails on second clone

Again, all these numbers are with bug 1715459 fixed; with that unfixed, code size would be about 33% higher.

Comment 4

[Lars T Hansen [:lth]]

Oh, and for the photoshop STR in comment 0, you must upload an image from the local computer before wasm compilation starts.

Comment 5

[Lars T Hansen [:lth]]

Coupled with bug 1719615, effectively all tools use is impossible on sites with large wasm blobs => S2.

Comment 6

[Lars T Hansen [:lth]]

This is fixed now, as we don't clone code for debugging any longer. The two primary related bugs, not turning on debug mode just because the console is open, and using less space for stack maps, are also important to fix but don't really block this any longer.