1532279 - Investigate Fannkuch performance in js vs wasm

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, enhancement, P3)

Description

[Benjamin Bouvier [:bbouvier] (inactive)]

Alex gave us a benchmark where wasm is performing badly, in particular worse than JS. I haven't investigated it, but if anybody wanted to, it might be nice to do so (in particular, is there some low-hanging fruit that we could fix until we get The Ultimate Fix that will be Cranelift; or just understanding better what the issues are).

Comment 1

[Lars T Hansen [:lth]]

We have a C version of fannkuch in the embenchen suite, too. Of course it's anyone's guess if that's based on the same source, but at least the source is there to check. https://github.com/lars-t-hansen/embenchen

Comment 2

[Till Schneidereit [:till]]

Same in Chrome, but even more pronounced.

Firefox (current Nightly):

wasm fannkuch: 5735ms
js fannkuch: 3058ms

Chrome (current Canary):

wasm fannkuch: 10670.962646484375ms
js fannkuch:    2719.386962890625ms

Comment 3

[Lars T Hansen [:lth]]

I'm not so sure. I've isolated the js and wasm portions of this benchmark as best I can, see https://github.com/lars-t-hansen/rust-fannkuch-js-and-wasm. The program js-benchmark.js runs the JS benchmark, and wasm-benchmark.js runs the rust/wasm benchmark. Run these in a properly optimized JS shell built for release (use --wasm-compiler=X for wasm compiler selection). These programs both check that they get the right checksum.

I then find that JS is slowest, followed by wasm-cranelift, wasm-baseline, and wasm-ion:

JS        4541
Cranelift 4164
Rabaldr   3661
Baldr     2980

Judging from the running time without arguments for wasm-benchmark, the shell at least selects ion for its compiler, as expected (tiny program, no tiering).

Comment 4

[Lars T Hansen [:lth]]

Reran these in a current --disable-debug --enable-optimize --enable-release shell on my x64 Lenovo desktop. The times for JS are strongly bimodal: most of the time, I see about 4300ms, but some of the time I see about 2900ms. The faster time I've only observed so far if I compile on all cores, so it may be dependent on optimized ion code arriving at a "good" time. In contrast the wasm times are stable: Baldr around 3200ms and Rabaldr around 3700.

This is almost certainly the same system I tested as last time, and it appears Baldr times have gotten worse. This is not great.

Interesting fact: cranelift-compiled code runs in about 2900ms. The most important optimization to land since the last time I tested is probably Julian's redundant-reload-remover, so we may be seeing the effect of that, but we also landed ebb->bb conversion with propagation of diversions across block boundaries, and that may have had an effect too. Even so, that's a very significant speedup, and I'm a little worried about it.

Comment 5

[Lars T Hansen [:lth]]

The Wasm pipeline (Ion) is consistently fast on my system, mean 2600ms, median 2550ms, effectively neck-and-neck with node 16. JS is much slower (see linked bug) but even if it were fast as reported in comment 4 it would not beat wasm.