1710358 - wasi support part 15: add CI build

Status: RESOLVED FIXED

(Core :: JavaScript Engine, enhancement, P2)

Description

[Dmitry Bezhetskov]

To test SM port on wasi we need to build it.
I build SM for wasi locally with following mozconfig:

CC=/opt/wasi-sdk/bin/clang
CXX=/opt/wasi-sdk/bin/clang++
AR=/opt/wasi-sdk/bin/ar
HOST_CC=gcc
HOST_CXX=g++

ac_add_options --enable-application=js
ac_add_options --target=wasm32-unknown-wasi

ac_add_options --disable-jit
ac_add_options --without-system-zlib
ac_add_options --disable-shared-js
ac_add_options --disable-shared-memory

./mach build

The final goal is to integrate this build to the mozilla's CI pipeline.

Comment 1

[Dmitry Bezhetskov]

The first task is to provide wasi-sdk 12.0 (https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-12) toolchain, this should be done via tooltool as I know.
Ben, could you consult me with this?

Comment 2

[Ted Campbell [:tcampbell]]

Researching more, it seems that tooltool is not the approach we want here, and instead should be following what was done in Bug 1582192 (and probably just extending the scripts.

I also wonder if our existing clang is a sufficient version for directly building, since the wasi-sdk readme says it just bundles upstream-LLVM.

Comment 3

[Ted Campbell [:tcampbell]]

+CC glandium, who is probably the expert on this sort of build/CI integration. I don't know if we have our specific questions yet, but any insights welcome here.

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Ted Campbell [:tcampbell] from comment #2)

Researching more, it seems that tooltool is not the approach we want here, and instead should be following what was done in Bug 1582192 (and probably just extending the scripts.

I also wonder if our existing clang is a sufficient version for directly building, since the wasi-sdk readme says it just bundles upstream-LLVM.

This is what I would suggest too. Tooltool is largely only used for things that we cannot publicly redistribute these days.

Comment 5

[Dmitry Bezhetskov]

So, I was able to at least build SM with WASI on CI with clang provided by mozilla's server and with built-in wasi-sysroot - https://treeherder.mozilla.org/jobs?repo=try&revision=4da80ea49c95b8c8ddef267ce858259275f51e12&selectedTaskRun=AEn4N4FET2iwX-5jKvm5kw.0.
Tests has failed but we don't want to run jit tests, at least now.

So, to do this we need:

1. upload this patch https://phabricator.services.mozilla.com/D116207 to properly configure clang with wasi sysroot
2. update wasi-sdk to the 12th version, because there is symbols conflict between wasi-libc and libjsrust. They are both required for the SM.
   The conflicting symbol is exit, see the fix in wasi-libc https://github.com/WebAssembly/wasi-libc/pull/201.
   Moreover, I tried to update wasi-sdk and it looks like nothing was crashed - https://treeherder.mozilla.org/jobs?repo=try&revision=95a8196e6709107f85c76a8ce86c9728811588ec.
3. disable nspr-build for WASI target because there is no support in nspr for WASI. In the build above I've just disabled it unconditionally
4. add wasm32-wasi as a rust toolchain to be able to compile rust components to Wasm
5. fix small warning for WASI in the code

After that steps we can use the following config to build SM with WASI:

ac_add_options --enable-application=js
ac_add_options --target=wasm32-unknown-wasi
ac_add_options --with-sysroot=/builds/worker/fetches/wasi-sysroot/share/wasi-sysroot/

ac_add_options --disable-gold
ac_add_options --disable-stdcxx-compat
ac_add_options --disable-clang-plugin
ac_add_options --without-system-zlib
ac_add_options --without-intl-api
ac_add_options --disable-jit
ac_add_options --disable-shared-js
ac_add_options --disable-shared-memory
ac_add_options --disable-tests
ac_add_options --disable-cranelift
ac_add_options --disable-jemalloc

Comment 6

[Dmitry Bezhetskov]

Attachment: Bug 1710358 - Update wasi-sdk to the version 12. r=jandem.

Update wasi-sdk from version 11 to the version 12.
This is primary needed to be able to build SM on wasi platform.

Comment 7

[Dmitry Bezhetskov]

Attachment: Bug 1710358 - Fix compilation warnings for WASI. r=jandem.

kNsPerUs is unused and fputwc return's type is wint_t.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cde0d09a0d91
Fix compilation warnings for WASI. r=jandem.

Comment 9

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/cde0d09a0d91

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d7851e4d8991
Update wasi-sdk to the version 12. r=tcampbell,glandium.

Comment 11

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/d7851e4d8991

Comment 12

[Dmitry Bezhetskov]

Attachment: Bug 1710358 - Add wasi SM CI build. r=jandem

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b964e59844e9
Add wasi SM CI build. r=jandem,sfink,glandium,jmaher

Comment 14

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/b964e59844e9

Comment 15

[Dmitry Bezhetskov]

Attachment: Bug 1710358 - Make wasi build tier 2. r=jandem

Since wasi build has been green for three weeks and since developers don't
want to break this build we give them
tier 2 to make it more visible.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ba55f401aea8
Make wasi build tier 2. r=jandem

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c7cb61bbff7
fix for wasi bustages. CLOSED TREE

Comment 18

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/ba55f401aea8
https://hg.mozilla.org/mozilla-central/rev/8c7cb61bbff7