1686652 - Generalize the SIMD wormhole and make available on non-Nightly

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, enhancement, P1)

Description

[Lars T Hansen [:lth]]

The wormhole must be generalized to x86+x64 and Ion+baseline.

Comment 1

[Lars T Hansen [:lth]]

Attachment: Bug 1686652 - Generalize the SIMD wormhole. r?yury

Generalize the wormhole to both x86 and x64, and both baseline and Ion.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c11d0b4deea8
Generalize the SIMD wormhole.  r=yury

Comment 3

[Noemi Erli[:noemi_erli]]

Backed out changeset c11d0b4deea8 (Bug 1686652) for causing failures in wormhole-x64-ion-codegen.js CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&selectedTaskRun=P1L0oddQSBaJT7eaVql4yg.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&classifiedState=classified&revision=c11d0b4deea88cb903cfbb7d57dc30d8649a6759

Failure log: https://treeherder.mozilla.org/logviewer?job_id=327744357&repo=autoland&lineNumber=24536

Backout: https://hg.mozilla.org/integration/autoland/rev/fcc478645578d8298e671091781342f335a5369d

Comment 4

[Lars T Hansen [:lth]]

A stale test case: it creates a combination of switches that breaks a test, but the combination is not allowed.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d0bbe10f929f
Generalize the SIMD wormhole.  r=yury

Comment 6

[Dorel Luca [:dluca]]

Backed out changeset d0bbe10f929f (bug 1686652) for Spidermonkey failure in js/src/jit-test/tests/wasm/simd/wormhole-gating.js. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer?job_id=327971841&repo=autoland&lineNumber=11667

Push with failures:
https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&revision=d0bbe10f929f9f29dba624bf18e5c9807dccdefb

Backout:
https://hg.mozilla.org/integration/autoland/rev/dd61cb5a9d848ecaaf04787c48cdef0d8afd18d0

Comment 7

[Lars T Hansen [:lth]]

Good grief, how hard can this be?

Comment 8

[Lars T Hansen [:lth]]

The problem here is that the moz.configure changes raced with some other changes; the changes here need to be stronger to exclude the arm64 simulator. Taking the opportunity to further tune the guards in the test case for clarity.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a3a8f2b9d254
Generalize the SIMD wormhole.  r=yury

Comment 10

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b9549541b848
Backed out changeset a3a8f2b9d254 for Linting failure on moz.configure . CLOSED TREE

Comment 11

[Narcis Beleuzu [:NarcisB]]

Backed out for Linting failure on moz.configure

Backout link: https://hg.mozilla.org/integration/autoland/rev/b9549541b848280bd182d8eda54bd11f1ab8e525
Log link: https://treeherder.mozilla.org/logviewer?job_id=327983173&repo=autoland&lineNumber=118

Comment 12

[Lars T Hansen [:lth]]

Targeting FF88 for an experiment.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/58936f707b78
Generalize the SIMD wormhole.  r=yury

Comment 14

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 58936f707b78 (bug 1686652) for wormhole related failures.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=bg8dyARaTFOLUjPp0K4tFQ.0&fromchange=58936f707b78c204f1dcff131691afd60485eed6&searchStr=build&tochange=79b3fae614f2e77f867d93eca95abc03faaa87bd

Backout link: https://hg.mozilla.org/integration/autoland/rev/79b3fae614f2e77f867d93eca95abc03faaa87bd

Failure log:
https://treeherder.mozilla.org/logviewer?job_id=328211878&repo=autoland&lineNumber=456
``
...
[task 2021-01-29T14:10:45.801Z] checking bindgen cflags... -x c++ -fno-sized-deallocation -fno-aligned-new -DTRACING=1 -DIMPL_LIBXUL -DMOZILLA_INTERNAL_API -DRUST_BINDGEN -DOS_POSIX=1 -DOS_LINUX=1 -std=gnu++17 -m32
[task 2021-01-29T14:10:45.804Z] ERROR: SIMD wormhole currently supported on native x86/x64 only
[task 2021-01-29T14:10:45.850Z] Traceback (most recent call last):
[task 2021-01-29T14:10:45.850Z] File "/builds/worker/checkouts/gecko/js/src/devtools/automation/autospider.py", line 514, in <module>
[task 2021-01-29T14:10:45.850Z] check=True,
[task 2021-01-29T14:10:45.850Z] File "/builds/worker/checkouts/gecko/js/src/devtools/automation/autospider.py", line 429, in run_command
[task 2021-01-29T14:10:45.850Z] raise subprocess.CalledProcessError(status, command, output=stderr)
[task 2021-01-29T14:10:45.850Z] subprocess.CalledProcessError: Command '['sh', '-c', '/builds/worker/checkouts/gecko/js/src/configure --enable-stdcxx-compat --disable-gold --enable-simulator=arm --target=i686-pc-linux --enable-rust-simd --enable-optimize --enable-debug --target=i686-pc-linux --enable-nspr-build --prefix=/builds/worker/workspace/obj-spider/dist']' returned non-zero exit status 1.
[task 2021-01-29T14:10:45.857Z] + BUILD_STATUS=1

Comment 15

[Lars T Hansen [:lth]]

Apart from yet another backout (this is probably a personal record) we're going to need other changes here. Minimally:

• the wasm compilation should take an options bag which asks for simd and simd_wormhole for the compilation without being set globally
• the fact that the wormhole is enabled needs to be recorded with serialized code because it's an attribute of the code

Additionally, we should discuss whether there should be a new preference that can be set by an extension to enable the recognition of the options bag. It may be that this is required to avoid WPT failures but that's TBD; it may be that it's desirable to have it regardless, so as to avoid exposing experimental content to the web more than for those that choose to use this functionality.

Additionally, it's possible javascript.options.wasm_simd_wormhole should go away, though it may be desirable to keep it for development?

Comment 16

[Lars T Hansen [:lth]]

Attachment: Bug 1686652 - Clean up SIMD metadata and features. r?rhunt

This just tidies up the wormhole switch, introducing a getter for it
on the moduleEnv. As a drive-by fix, the v128Enabled switch in the
code metadata is removed, since nobody's using it. (At least I think
so. It does not seem to be part of anything we use to determine
whether cached code is valid, for example.)

Depends on D101709

Comment 17

[Lars T Hansen [:lth]]

Attachment: Bug 1686652 - Make the wormhole cleverer. r?rhunt

This introduces cleaner logic around selecting the SIMD wormhole. The
large comment block in WasmConstants.h explains it in full, but in
summary this is what happens:

• The switch j.o.wasm_simd_wormhole now serves as a signal that the
  wormhole may be requested through other means for individual
  compilations by content, it does not directly enable functionality.

• Content can pass an options bag to wasm compilation functions
  requesting simd + the simd wormhole; the j.o.wasm_simd switch may be
  false but is overridden in this case. The bag is ignored if the
  j.o.wasm_simd_wormhole pref is not set.

The intent of this mechanism is that privileged extensions will set
the j.o.wasm_simd_wormhole switch and then request compilation of wasm
code with the {simdWormhole:true} options bag, thus enabling simd
functionality (with the wormhole opcodes, too) in release browsers
for these extensions before simd ships. We do it this way because
enabling j.o.wasm_simd exposes content in a way we don't want to
expose it.

Depends on D103693

Comment 18

[Lars T Hansen [:lth]]

Patches finally green on try with acceptable logic and updated logic for feature enablement (see commit msg). There is a placeholder for "privileged content" that currently always returns false.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e595bdcf782b
Generalize the SIMD wormhole.  r=yury
https://hg.mozilla.org/integration/autoland/rev/6c5025b25cde
Clean up SIMD metadata and features. r=rhunt
https://hg.mozilla.org/integration/autoland/rev/63e8e8ea1578
Make the wormhole cleverer. r=rhunt

Comment 20

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/e595bdcf782b
https://hg.mozilla.org/mozilla-central/rev/6c5025b25cde
https://hg.mozilla.org/mozilla-central/rev/63e8e8ea1578