1561521 - Implement non-nullable references

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, enhancement, P3)

Description

[Lars T Hansen [:lth]]

As part of developing a separate proposal for typed function references, AR has proposed that the (ref T) syntax be reserved for non-nullable types and that (optref T) is instead used for nullable types. This will have some wide-ranging effects to text decoding and internally in the engine, I expect.

Comment 1

[Ryan Hunt [:rhunt] (on leave until early May)]

I'm going to take this bug.

Comment 2

[Ryan Hunt [:rhunt] (on leave until early May)]

Current patch stack that implements this [1]. Not posting for review while Lars is out.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=e7c40feff8f520ac9687e3bbcc69b63cad6398e7

Comment 3

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Vendor the latest wat. r?lth

Comment 4

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Automatically replace all uses of anyref with externref. r?lth

When we removed subtyping, we kept the anyref text syntax as an alias of
externref. This commit completes the migration over to externref. A future
commit will switch uses of externref which should be anyref back.

Depends on D85052

Comment 5

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Rename (ref opt T) to (ref null T). r?lth

The nullable reference syntax has changed from opt to null.

Depends on D85053

Comment 6

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Remove tests that assert one-level imports are invalid text. r?lth

Wat now implements the module-linking proposal which includes one-level imports.

This commit removes tests of the text format which asserted these were not-valid.

Depends on D85054

Comment 7

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Rewrite (ref.null $reftype) to (ref.null $heaptype). r?lth

The function-references spec refined ref.null to take a heaptype, which has a
slightly different text format for non-nullable references. This commit updates
tests to the new syntax.

Depends on D85055

Comment 8

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Revert to anyref in tests that require it. r?lth

The tests in this commit were switched to externref automatically, but
actually do require a universal supertype. This commit switches them back.

Depends on D85056

Comment 9

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Add wasm/ref-types/ folder, move non-gc specific tests there. r?lth

The tests in wasm/gc are a mix of reftypes/gc tests. This commit adds a
ref-types folder and moves relevant tests over.

Depends on D85057

Comment 10

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Rename ref-types/ tests that actually test externref. r?lth

Follow-up to the previous commit that moved these tests over to wasm/ref-types,
this commit renames the tests from 'anyref' to 'externref'.

Depends on D85059

Comment 11

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Clean up disabled tests for gc/ref-types. r?lth

This commit finishes the split of wasm/gc and wasm/ref-types tests by splitting
the disabled-ref.js test into a GC test and a ref-type test.

Depends on D85060

Comment 12

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Tests: Change tests that rely on old test messages. r?lth

The following commits will change test messages for a couple tests, and this
commit fixes the breakage.

Depends on D85061

Comment 13

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Rename TypeCode/ValType/RefType::Any to Extern. r?lth

This commit renames the TypeCode/ValType/RefType occurrences of 'any' to 'extern'
to match what they actually represent. All public API facing occurrences have
already been switched to 'externref' so this should only be aligning the
internal names with their extern usage.

Comments that only apply to 'externref' code paths are renamed as well.

Depends on D85062

Comment 14

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Generalize table/element code for future reference-types. r?lth

wasm::Table uses wasm::TableKind to determine the element type of the table,
which makes it difficult to extend tables to support more reference types. This
commit drops TableKind in favor of directly storing the element RefType, along
with a flag for whether this table is forAsmJS, which influences the funcref
ABI. RefType is extended to return the existing TableRepr enum, which indicates
the memory representation to use for the table.

wasm::ElemSegment uses wasm::ValType, which is a superset of the allowed
types of an elem segment. This commit switches it to use the proper
RefType class, and tweaks decoding.

Depends on D85063

Comment 15

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Add 'anyref' as universal super-type independent of 'externref'. r?lth

This commit adds back an 'anyref' type pref'ed to only the GC proposal.

Depends on D85064

Comment 16

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Add constant for abstracting reference-types values. r?lth

TypeCode::OptRef is currently used to represent any reference-typed value
for UnpackTypeCodeTypeAbstracted and IsReferenceType. I think it's actually
clearer to have this behind a named constant.

Depends on D85065

Comment 17

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Add non-nullable references. r?lth

This commit adds decoding, validation, and JS-API type-checking for
non-nullable references.

Depends on D85066

Comment 18

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Refine ref.null decoding to decode heaptype instead of reftype. r?lth

ref.null takes a heaptype immediate. This is backwards compatible for
reference-types, but is a breaking change for the GC proposal. This commit
implements decoding for this refinement.

Depends on D85068

Comment 19

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Report error when decoding a value-type and there are no bytes. r?lth

We need to report an error message here or else it will be reported as an OOM.

Depends on D85069

Comment 20

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1561521 - Clean up wasm::ToString and drop wasm::ToCString. r?lth

wasm::ToCString is losing relevance as we have more constructed types. This
commit removes ToCString and replaces all uses with wasm::ToString.

Depends on D85070

Comment 21

[Ryan Hunt [:rhunt] (on leave until early May)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cc7fc2fceb559517a86735dc44b42b29265e64ba

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/41e46f9041bd
Vendor the latest wat. r=lth
https://hg.mozilla.org/integration/autoland/rev/fe600b71015f
Tests: Automatically replace all uses of anyref with externref. r=lth
https://hg.mozilla.org/integration/autoland/rev/b8c0cb8771b6
Tests: Rename (ref opt T) to (ref null T). r=lth
https://hg.mozilla.org/integration/autoland/rev/a3abc2d14f78
Tests: Remove tests that assert one-level imports are invalid text. r=lth
https://hg.mozilla.org/integration/autoland/rev/bc80a4372ba0
Tests: Rewrite (ref.null $reftype) to (ref.null $heaptype). r=lth
https://hg.mozilla.org/integration/autoland/rev/a18f7f6320c4
Tests: Add wasm/ref-types/ folder, move non-gc specific tests there. r=lth
https://hg.mozilla.org/integration/autoland/rev/1cb2561bf28f
Tests: Rename ref-types/ tests that actually test externref. r=lth
https://hg.mozilla.org/integration/autoland/rev/c827ecead12d
Tests: Clean up disabled tests for gc/ref-types. r=lth
https://hg.mozilla.org/integration/autoland/rev/3d3e3d124767
Tests: Change tests that rely on old test messages. r=lth
https://hg.mozilla.org/integration/autoland/rev/538920d2d777
Rename TypeCode/ValType/RefType::Any to Extern. r=lth
https://hg.mozilla.org/integration/autoland/rev/a7ad7e2b02c1
Generalize table/element code for future reference-types. r=lth
https://hg.mozilla.org/integration/autoland/rev/cd847eba7f9c
Add constant for abstracting reference-types values. r=lth
https://hg.mozilla.org/integration/autoland/rev/00d2290648a7
Add non-nullable references. r=lth
https://hg.mozilla.org/integration/autoland/rev/02ce4be46831
Refine ref.null decoding to decode heaptype instead of reftype. r=lth
https://hg.mozilla.org/integration/autoland/rev/3d1a8283d2f8
Report error when decoding a value-type and there are no bytes. r=lth
https://hg.mozilla.org/integration/autoland/rev/a850ff24c2ae
Clean up wasm::ToString and drop wasm::ToCString. r=lth

Comment 23

[Dorel Luca [:dluca]]

Backed out 16 changesets (bug 1561521) for WASM build bustages

Log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=313289121&repo=autoland&lineNumber=32140

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=a850ff24c2aeaefac447d9e501f6e443f395f844

Backout:
https://hg.mozilla.org/integration/autoland/rev/edf436bd65285dab6d875bbfd583dc5995882867

Comment 24

[Ryan Hunt [:rhunt] (on leave until early May)]

Needed to rebase and fix compile error with new argument added to readValType.

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ea8dd38c1de0
Vendor the latest wat. r=lth
https://hg.mozilla.org/integration/autoland/rev/6f317b53f2d9
Tests: Automatically replace all uses of anyref with externref. r=lth
https://hg.mozilla.org/integration/autoland/rev/e131485d1705
Tests: Rename (ref opt T) to (ref null T). r=lth
https://hg.mozilla.org/integration/autoland/rev/c05cad14170b
Tests: Remove tests that assert one-level imports are invalid text. r=lth
https://hg.mozilla.org/integration/autoland/rev/1d18a543d88e
Tests: Rewrite (ref.null $reftype) to (ref.null $heaptype). r=lth
https://hg.mozilla.org/integration/autoland/rev/c725d08d250e
Tests: Add wasm/ref-types/ folder, move non-gc specific tests there. r=lth
https://hg.mozilla.org/integration/autoland/rev/42ff212c7a38
Tests: Rename ref-types/ tests that actually test externref. r=lth
https://hg.mozilla.org/integration/autoland/rev/30e6e85cc32c
Tests: Clean up disabled tests for gc/ref-types. r=lth
https://hg.mozilla.org/integration/autoland/rev/4fa52a0617de
Tests: Change tests that rely on old test messages. r=lth
https://hg.mozilla.org/integration/autoland/rev/7805ddfe2bfa
Rename TypeCode/ValType/RefType::Any to Extern. r=lth
https://hg.mozilla.org/integration/autoland/rev/8ce9bbcd26e3
Generalize table/element code for future reference-types. r=lth
https://hg.mozilla.org/integration/autoland/rev/e055945526a2
Add constant for abstracting reference-types values. r=lth
https://hg.mozilla.org/integration/autoland/rev/546d74acfb81
Add non-nullable references. r=lth
https://hg.mozilla.org/integration/autoland/rev/008f196ed172
Refine ref.null decoding to decode heaptype instead of reftype. r=lth
https://hg.mozilla.org/integration/autoland/rev/a72132acc190
Report error when decoding a value-type and there are no bytes. r=lth
https://hg.mozilla.org/integration/autoland/rev/048c2bf91290
Clean up wasm::ToString and drop wasm::ToCString. r=lth

Comment 26

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/ea8dd38c1de0
https://hg.mozilla.org/mozilla-central/rev/6f317b53f2d9
https://hg.mozilla.org/mozilla-central/rev/e131485d1705
https://hg.mozilla.org/mozilla-central/rev/c05cad14170b
https://hg.mozilla.org/mozilla-central/rev/1d18a543d88e
https://hg.mozilla.org/mozilla-central/rev/c725d08d250e
https://hg.mozilla.org/mozilla-central/rev/42ff212c7a38
https://hg.mozilla.org/mozilla-central/rev/30e6e85cc32c
https://hg.mozilla.org/mozilla-central/rev/4fa52a0617de
https://hg.mozilla.org/mozilla-central/rev/7805ddfe2bfa
https://hg.mozilla.org/mozilla-central/rev/8ce9bbcd26e3
https://hg.mozilla.org/mozilla-central/rev/e055945526a2
https://hg.mozilla.org/mozilla-central/rev/546d74acfb81
https://hg.mozilla.org/mozilla-central/rev/008f196ed172
https://hg.mozilla.org/mozilla-central/rev/a72132acc190
https://hg.mozilla.org/mozilla-central/rev/048c2bf91290