1584414 - Cranelift overhead on test application (jpeg re-encoder) over 100%

Status: RESOLVED INACTIVE

(Core :: JavaScript: WebAssembly, defect, P5)

Description

[Shravan Narayan]

Attachment: examples for native and wasm compiled application

This bug is in the context of use of Cranelift to generate wasm sandboxed libraries for use in Firefox - i.e. this is a non web embedding. Full details available here https://bugzilla.mozilla.org/show_bug.cgi?id=1562797

We are using lucet as our wasm compiler which uses Cranelift under the covers. The version of the Cranelift used in the below tests is on a commit from Sept 3rd (Id: 6850d8e60daf128633366b8c98461b0fb1972581)

To estimate overhead of Cranelift, I wrote a test application that measures overhead of a jpeg re-encoding for an image when compiled with a stock C compiler, versus when going to through Cranelift. SIMD is turned of on both cases. The application also does not make any hostcalls

When running this I see overheads of 100 to 150% which seems to possibly higher than expectations? I was told to file a bug in case this was something that needed further scrutiny. As a comparison point, I tested the overhead of a similar application in Native Client a while back - the overhead there was about 30 to 50%.

The repo containing the benchmark is located here. In the repo, the source of the benchmark application is here.

If you want to build the benchmark from the repo, use the wasm_test branch, update paths in the build_script and run the build_script.

Alternately, I have attached the zip file containing 2 binaries - the benchmark compiled natively, the benchmark compiled with clang-wasi+lucet.

Comment 1

[Lars T Hansen [:lth]]

Thank you for the report. Cranelift is currently generating fairly poor machine code, due in large part to a poor register allocator; significantly slower code than generated by our production JIT should be expected. Work is underway to fix this. This work will likely not be completed soon.

Comment 2

[Shravan Narayan]

Sounds good! For the moment, this level of performance difference is not blocking for our use cases. So, feel free to close the bug if this issue is already being tracked elsewhere.

(Also if it helps, feel free to use the above application as a perf benchmark for Cranelift).

Comment 3

[Lars T Hansen [:lth]]

(Cleaning up)

Please file Cranelift bugs over here: https://github.com/bytecodealliance/wasmtime, we don't track them here any longer.