1571559 - add support for a --with-wasi-sysroot option

Status: RESOLVED FIXED

(Firefox Build System :: General, task, P3)

Description

[Shravan Narayan]

This task tracks the progress of including the clang compiler toolchain to build wasm libraries in the Firefox build system. This task will enable allow the compilation of components to produce a wasm file compatible with wasi.

There are 2 approaches to do this

1. Include a prebuilt release of clang+wasm+wasi released as part of cranelift work here. I believe this clang defaults to wasm target and the wasi libc by default, so this clang can be used as normal, requiring no additional flags or arguments.
2. Build from source. The details to build are provided below. When built from source, we have to pass some extra flags (listed below) as part of the command line invocation of clang.

Building from source

1. Get source code for stock clang and llvm 8 (needed for wasm backend).
2. Get the llvm lld linker from here and include it in the tools folder of llvm. lld will now be built along with clang.
3. clang would have to be configured with the CMAKE flag LLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly to enable the wasm backend (as clang does not include this by default).
4. Build clang
5. Get the wasi sysroot - i.e. libc implementation that is wasm compatible from here
6. Build the wasi sysroot with the clang that was compiled in step 4 (more details available on the sysroot readme).

Files can now be compiled with

clang -B$(PATH_TO_LLD) --target=wasm32-wasi --sysroot $(PATH_TO_WASI_SYSROOT) <normal args to clang...>

Comment 1

[Nathan Froyd [:froydnj]]

(In reply to Shravan Narayan from comment #0)

Building from source

1. Get source code for stock clang and llvm 8 (needed for wasm backend).
2. Get the llvm lld linker from here and include it in the tools folder of llvm. lld will now be built along with clang.
3. clang would have to be configured with the CMAKE flag LLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly to enable the wasm backend (as clang does not include this by default).
4. Build clang

We have these scripts for building clang in automation:

https://searchfox.org/mozilla-central/source/build/build-clang

We don't set up wasm support, but that's pretty easy to add.

One thing that we would have to do is figure out how to test the selected clang during configure for wasm support.

5. Get the wasi sysroot - i.e. libc implementation that is wasm compatible from here

This is straightforward to do for automation.

6. Build the wasi sysroot with the clang that was compiled in step 4 (more details available on the sysroot readme).

Files can now be compiled with

clang -B$(PATH_TO_LLD) --target=wasm32-wasi --sysroot $(PATH_TO_WASI_SYSROOT) <normal args to clang...>

The built wasi sysroot can be provided as an artifact for automation builds and we can figure out how to make it downloadable for people running outside automation. We'll probably wind up specifying a configure option like --with-wasi-sysroot=SYSROOT and using that in automation.

The action items then look something like:

1. Build our clangs with wasm support (when available; I think the script builds clang versions that don't know anything about wasm).
2. Figure out how to test the clang we're building with for wasm support in Firefox configure.
3. Build the wasi sysroot in automation, and make any builds where we're enabling the sandboxing depend on that sysroot.
4. Provide a --with-wasi-sysroot configuration option.
5. Probably something else that I have failed to foresee, but will become obvious once I start implementing all of the above.

Comment 2

[Nathan Froyd [:froydnj]]

I think I'm going to repurpose this for adding configuration support for --with-wasi-sysroot; otherwise, I don't see how it's much different than bug 1579870.

Comment 3

[Nathan Froyd [:froydnj]]

Attachment: Bug 1571559 - add configuration support for --with-wasi-sysroot; r=#build

As part of compiling C/C++ to wasm, we're going to need a wasm-specific
sysroot to be provided. This option enables us to specify that sysroot
during configure.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a4e430e623a8
add configuration support for --with-wasi-sysroot; r=nalexander

Comment 5

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/a4e430e623a8