1532689 - Update to Cranelift 0.29

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, task)

Comment 1

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: Use a ModuleEnvironment pointer (instead of a reference) to work around a bindgen bug; r=sunfish

The structure layout was incorrectly computed because of the reference, meaning
that the data located after the reference was incorrect. In particular, it
means the minimal memory size wasn't correctly read. This works around it by
using a pointer, and rename a few function parameters to make their role
clearer.

Comment 2

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: output of mach vendor rust; r=sunfish

Comment 3

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: Bump Cranelift to 0.29 in Spidermonkey; r=sunfish

This adds new code to provide the module-relative initial function offset for
each function, and adds checks that the bytecode / code offsets are correct.

Comment 4

[Benjamin Bouvier [:bbouvier] (inactive)]

For what it's worth, I also experimented with conditional compilation, but:

• Cranelift will force the native target if none is selected at compile time.
• (with a modified version of Cranelift) forcing no targets implies dead code, which the Gecko build doesn't tolerate.

Comment 5

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: Use Cranelift features to include only architecture-specific support; r=froydnj

This introduces features in the jsrust crate, so we can enable/disable
compilation for a specific platform at compile-time. It also does only select
the architecture targeted by the JIT, which should result in slightly lower
compilation times on every platform, and lower binary sizes too.

Comment 6

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: Refactor baldrdash compile a bit; r?sunfish

• Uses constants for function namespaces;
• Puts the source location retrieval under a single function;

Comment 7

[Benjamin Bouvier [:bbouvier] (inactive)]

Attachment: Bug 1532689: Rename CraneliftMetadata fields to help distinguish them; r?sunfish

This renames:

• offset to codeOffset (which is an offset in the code generated by Cranelift,
  relative to the beginning of the function, which we can now infer from the
  name).
• srcLoc to moduleBytecodeOffset, since this is what it is (and much clearer
  than the previous name).

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/bf190c7c3de3
Use a ModuleEnvironment pointer (instead of a reference) to work around a bindgen bug; r=sunfish
https://hg.mozilla.org/mozilla-central/rev/2aa82e63f173
Bump Cranelift to 0.29 in Spidermonkey; r=sunfish

Comment 9

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/bf190c7c3de3
https://hg.mozilla.org/mozilla-central/rev/2aa82e63f173

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed3b55f9d326
Use Cranelift features to include only architecture-specific support; r=froydnj

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset ed3b55f9d326 (Bug 1532689) for causing build bustages

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=ed3b55f9d32607b2f60156551c627a5b88930373&selectedJob=232714747

Backout link: https://hg.mozilla.org/integration/autoland/rev/398dfc746ccd2f079d74198274020691b6299968

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=232714747&repo=autoland&lineNumber=4341

[task 2019-03-08T18:01:50.457Z] 18:01:50 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:50.457Z] 18:01:50 INFO - config/external/nspr/pr/darwin.o
[task 2019-03-08T18:01:50.457Z] 18:01:50 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk -std=gnu99 --target=x86_64-darwin11 -o darwin.o -c -fvisibility=hidden -fvisibility-inlines-hidden -DDEBUG=1 -D_NSPR_BUILD_ -DHAS_CONNECTX -DHAVE_BSD_FLOCK -DHAVE_POINTER_LOCALTIME_R -DDARWIN -DHAVE_SOCKLEN_T -DHAVE_CRT_EXTERNS_H -D_PR_PTHREADS -I/builds/worker/workspace/build/src/config/external/nspr/pr -I/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr -I/builds/worker/workspace/build/src/config/external/nspr -I/builds/worker/workspace/build/src/nsprpub/pr/include -I/builds/worker/workspace/build/src/nsprpub/pr/include/private -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fcrash-diagnostics-dir=/builds/worker/artifacts -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O3 -fno-omit-frame-pointer -funwind-tables -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -MD -MP -MF .deps/darwin.o.pp /builds/worker/workspace/build/src/nsprpub/pr/src/md/unix/darwin.c
[task 2019-03-08T18:01:50.457Z] 18:01:50 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:50.465Z] 18:01:50 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/modules/fdlibm/src'
[task 2019-03-08T18:01:50.465Z] 18:01:50 INFO - modules/fdlibm/src/e_atan2.o
[task 2019-03-08T18:01:50.465Z] 18:01:50 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/modules/fdlibm/src'
[task 2019-03-08T18:01:50.483Z] 18:01:50 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:50.487Z] 18:01:50 INFO - config/external/nspr/pr/praton.o
[task 2019-03-08T18:01:50.488Z] 18:01:50 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk -std=gnu99 --target=x86_64-darwin11 -o praton.o -c -fvisibility=hidden -fvisibility-inlines-hidden -DDEBUG=1 -D_NSPR_BUILD_ -DHAS_CONNECTX -DHAVE_BSD_FLOCK -DHAVE_POINTER_LOCALTIME_R -DDARWIN -DHAVE_SOCKLEN_T -DHAVE_CRT_EXTERNS_H -D_PR_PTHREADS -I/builds/worker/workspace/build/src/config/external/nspr/pr -I/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr -I/builds/worker/workspace/build/src/config/external/nspr -I/builds/worker/workspace/build/src/nsprpub/pr/include -I/builds/worker/workspace/build/src/nsprpub/pr/include/private -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fcrash-diagnostics-dir=/builds/worker/artifacts -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O3 -fno-omit-frame-pointer -funwind-tables -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -MD -MP -MF .deps/praton.o.pp /builds/worker/workspace/build/src/nsprpub/pr/src/misc/praton.c
[task 2019-03-08T18:01:50.489Z] 18:01:50 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:50.492Z] 18:01:50 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/toolkit/library/gtest/rust'
[task 2019-03-08T18:01:50.492Z] 18:01:50 INFO - toolkit/library/gtest/rust/force-cargo-library-build
[task 2019-03-08T18:01:50.493Z] 18:01:50 INFO - /builds/worker/workspace/build/src/rustc/bin/cargo rustc --frozen --manifest-path /builds/worker/workspace/build/src/toolkit/library/gtest/rust/Cargo.toml -vv --lib --target=x86_64-apple-darwin --features 'servo bindgen gecko_debug quantum_render simd-accel cubeb-remoting moz_memory spidermonkey_rust cranelift_x86 gecko_profiler' --
[task 2019-03-08T18:01:50.493Z] 18:01:50 INFO - error: Package gkrust-gtest v0.1.0 (/builds/worker/workspace/build/src/toolkit/library/gtest/rust) does not have these features: cranelift_x86
[task 2019-03-08T18:01:50.493Z] 18:01:50 INFO - /builds/worker/workspace/build/src/config/makefiles/rust.mk:213: recipe for target 'force-cargo-library-build' failed
[task 2019-03-08T18:01:50.493Z] 18:01:50 ERROR - make[4]: *** [force-cargo-library-build] Error 101
[task 2019-03-08T18:01:50.493Z] 18:01:50 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/toolkit/library/gtest/rust'
[task 2019-03-08T18:01:50.494Z] 18:01:50 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'toolkit/library/gtest/rust/target' failed
[task 2019-03-08T18:01:50.494Z] 18:01:50 ERROR - make[3]: *** [toolkit/library/gtest/rust/target] Error 2
[task 2019-03-08T18:01:50.494Z] 18:01:50 INFO - make[3]: *** Waiting for unfinished jobs....
[task 2019-03-08T18:01:50.497Z] 18:01:50 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/js/src/gc'
[task 2019-03-08T18:01:50.497Z] 18:01:50 INFO - js/src/gc/Unified_cpp_js_src_gc0.o
[task 2019-03-08T18:01:50.497Z] 18:01:50 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/js/src/gc'
[task 2019-03-08T18:01:51.022Z] 18:01:51 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:51.023Z] 18:01:51 INFO - config/external/nspr/pr/prcountr.o
[task 2019-03-08T18:01:51.023Z] 18:01:51 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk -std=gnu99 --target=x86_64-darwin11 -o prcountr.o -c -fvisibility=hidden -fvisibility-inlines-hidden -DDEBUG=1 -D_NSPR_BUILD_ -DHAS_CONNECTX -DHAVE_BSD_FLOCK -DHAVE_POINTER_LOCALTIME_R -DDARWIN -DHAVE_SOCKLEN_T -DHAVE_CRT_EXTERNS_H -D_PR_PTHREADS -I/builds/worker/workspace/build/src/config/external/nspr/pr -I/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr -I/builds/worker/workspace/build/src/config/external/nspr -I/builds/worker/workspace/build/src/nsprpub/pr/include -I/builds/worker/workspace/build/src/nsprpub/pr/include/private -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fcrash-diagnostics-dir=/builds/worker/artifacts -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O3 -fno-omit-frame-pointer -funwind-tables -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -MD -MP -MF .deps/prcountr.o.pp /builds/worker/workspace/build/src/nsprpub/pr/src/misc/prcountr.c
[task 2019-03-08T18:01:51.023Z] 18:01:51 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/config/external/nspr/pr'
[task 2019-03-08T18:01:51.026Z] 18:01:51 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mfbt'
[task 2019-03-08T18:01:51.030Z] 18:01:51 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk --target=x86_64-darwin11 -o Compression.o -c -fvisibility=hidden -fvisibility-inlines-hidden -DDEBUG=1 -DIMPL_MFBT -DLZ4LIB_VISIBILITY= -I/builds/worker/workspace/build/src/mfbt -I/builds/worker/workspace/build/src/obj-firefox/mfbt -I/builds/worker/workspace/build/src/mfbt/double-conversion -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++1z-compat -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fcrash-diagnostics-dir=/builds/worker/artifacts -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-exceptions -fno-strict-aliasing -stdlib=libc++ -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O3 -fno-omit-frame-pointer -funwind-tables -Werror -MD -MP -MF .deps/Compression.o.pp /builds/worker/workspace/build/src/mfbt/Compression.cpp
[task 2019-03-08T18:01:51.030Z] 18:01:51 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mfbt'
[task 2019-03-08T18:01:51.030Z] 18:01:51 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mfbt'
[task 2019-03-08T18:01:51.030Z] 18:01:51 INFO - mfbt/Decimal.o

Comment 12

[Benjamin Bouvier [:bbouvier] (inactive)]

Sorry. Forgot to include a few Cargo features in the gtest/rust directory.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=923c29d34549c7c2c89772990d3d4feda9b894ce

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dbd1b0222c6f
Use Cranelift features to include only architecture-specific support; r=froydnj

Comment 14

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/dbd1b0222c6f

Comment 15

[Benjamin Bouvier [:bbouvier] (inactive)]

Calling this fixed, will move the last two commits to another bug.

Comment 16

[Phabricator Automation]

Comment on attachment 9048806 [details]
Bug 1532689: Refactor baldrdash compile a bit; r?sunfish

Revision D22281 was moved to bug 1538099. Setting attachment 9048806 [details] to obsolete.

Comment 17

[Phabricator Automation]

Comment on attachment 9048807 [details]
Bug 1532689: Rename CraneliftMetadata fields to help distinguish them; r?sunfish

Revision D22282 was moved to bug 1538099. Setting attachment 9048807 [details] to obsolete.