1676441 - Allow Wasm memories to grow past 2GB on 64-bit systems

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, enhancement, P2)

Description

[Lars T Hansen [:lth]]

Various test cases for wasm memories > 2GB, and tweaks and infrastructure to make them pass.

Comment 1

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Allow up to 65534 pages + test cases. r?rhunt

Comment 2

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Going from 65534 to 65536 is hard, WIP

Depends on D96602

Comment 3

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - CreateWasmSpecificBuffer can accept more than UINT32_MAX initial size

Depends on D96602

Comment 4

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Upgrade to BufferSize::get everywhere. r?rhunt

Depends on D98969

Comment 5

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - boundsCheck32 fixes (WIP)

Depends on D98970

Comment 6

[Lars T Hansen [:lth]]

The plan for asm.js is to restrict it to 2GB, so that we do not have to vet the semantics + the implementation to handle more. This has been implemented as a check during linking, and iiuc a link failure means we'll just fall back to JS, so asm.js programs will be able to use the larger heaps, but they will be run as JS, not translated to wasm.

Comment 7

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Bounds checking part 1: Setup

Make the bounds check limit field platform-dependent and tidy up some
naming.

Make sure asm.js tests are run with explicit bounds checking also on
64-bit platforms.

Depends on D98970

Comment 8

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Bounds checking for 4GB heaps. r?rhunt

Add wasmBoundsCheck64() APIs that will take a 64-bit index and 64-bit
limit, to be used on all 64-bit platforms.

Document how the bounds checking primitives are supposed to be used.

Depends on D99097

Comment 9

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Bounds checking part 3: Baseline support

Use the new 64-bit API on 64-bit systems (after widening the index
appropriately) and the 32-bit API on 32-bit systems.

Depends on D99098

Comment 10

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Bounds checking part 4: Ion support

Update the Ion pipeline to support 64-bit bounds for wasm on 64-bit
platforms, and 32-bit bounds for asm.js everywhere and wasm on 32-bit
platforms.

Depends on D99102

Comment 11

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Bounds checking part 5: Cranelift support, WIP

Depends on D99103

Comment 12

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Initial notes about 65536 pages

Depends on D99104

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cb02a960ee5c
Allow up to 65534 pages + test cases. r=rhunt
https://hg.mozilla.org/integration/autoland/rev/ce483363652d
Upgrade to BufferSize::get everywhere.  r=rhunt

Comment 15

[Lars T Hansen [:lth]]

Attachment: Bug 1676441 - Allow 64K pages. r?rhunt

Increase the max number of allocatable pages on 64-bit from 65534 to
65536, except when Cranelift is present (because Cranelift depends on
the bounds check limit being 32 bits and we don't want to deal with
updating Cranelift right now).

Change TlsData::boundsCheckLimit from uint32_t to uintptr_t, so that
it can be 64 bits on 64-bit systems and can represent a 4GB heapsize.

Update test cases so that boundary conditions are handled properly,
for example, i32(65536*65536) == 0, so some test cases should not be
run unless the max page count is below 64K.

Note, this patch does not update the compilers, so the bounds check
limit continues to be read as a 32-bit quantity. This results in some
failures, which go away when the compilers are updated (next patch).

Comment 16

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/cb02a960ee5c
https://hg.mozilla.org/mozilla-central/rev/ce483363652d

Comment 17

[Lars T Hansen [:lth]]

Looking at the MIPS64 code in a little more detail, I'm not at all certain that it can handle more than 2GB heap. For index values >= 2GB, the natural 64-bit datum on MIPS64 is sign-extended, but that won't work properly when considered as an offset into the heap array. Since I'm not interested in investing more time in that platform, and since the MIPS64 simulator doesn't really work and I can't test anything, I'm going to try to tweak these patches slightly to clamp memory sizes at 2GB for MIPS64. I don't think this will be hard.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c6b934f5ec43
Allow 64K pages. r=rhunt
https://hg.mozilla.org/integration/autoland/rev/293371b99fca
Bounds checking for 4GB heaps. r=rhunt

Comment 19

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/c6b934f5ec43
https://hg.mozilla.org/mozilla-central/rev/293371b99fca