Enable SSE4.1 on wasm builds of marian for simd
Categories
(Firefox :: Translations, enhancement)
Tracking
()
People
(Reporter: anatal, Unassigned)
References
(Blocks 1 open bug)
Details
We should stop using intel sse instructions in bergamot-translator c++ code in favor of wasm simd instructions.
We are currently using intel sse3 instructions and this is seriously suboptimal probably, so we should move to intel sse4.1 instructions to perhaps get a better translation to wasm
|
||
Comment 1•5 years ago
|
||
Yes, the problem here is that the intel sse instructions are implemented in terms of the wasm simd instructions. Since there's not a one-to-one mapping (because the wasm simd instructions are cross-platform), the simulation is sometimes elaborate for no other reason than to make the simulation precise. By using wasm simd instructions directly, no such friction occurs. Yury did some investigations that suggested that the friction results in a performance loss.
The other problem that compounds the simulation problem is that, as Andre says, the current intel-specific code is compiled with flags that select sse3 (or ssse3, I forget which). This means that not only are the intel instructions simulated by a number of wasm instructions, but the intel instructions that are available to simulate are quite limited. In practice, allowing the use of sse4.1 instructions in the C++ code unlocks a richer instruction set that probably has a more direct mapping to wasm simd instructions.
Yury, do you have any data sitting around from when you last looked at this?
|
||
Comment 2•5 years ago
•
|
||
I did not see specific SSE4.1 optimizations or code/logic to generate SSE4.1-friendly instructions in intgemm. I guess the reason was that there are not so many such instructions and places that would make a difference. On the other hand, SpiderMonkey already generates SSE4.1 instructions for wasm simd code by default.
Yury, do you have any data sitting around from when you last looked at this?
I have only benchmark comparison between native and wasm+jit code, which just demonstrates that some loss of performance is happening when intel instructions are mapped to wasm simd and then back. If there is an interest to have comparison native vs wasm+jit machine code fragments, I can help with that.
|
|
Reporter | |
Updated•3 years ago
|
|
||
Comment 3•3 years ago
|
||
André: Is this bug still valid with the more recent work around this? Can we close it?
|
|
Reporter | |
Comment 4•3 years ago
|
||
I don't have the entire context if that's still the case after the port of intgemm to gemmology. I wasn't directly involved in the migration and am not aware of the specifics. Marco or Serge are better suited to addresses this question.
What I can say is that in terms of the web extension, we are still shipping the same artifact since we released it, but I don't have the info if the native version is using that or a newly compiled version.
|
||
Comment 5•3 years ago
|
||
The intrinsics are just for the matrix multiplication part of Bergamot, this bug is about the rest of the functionality. It is currently compiled with SSE3, which is suboptimal as it doesn't map well to WebAssembly SIMD.
|
|
||
Updated•3 years ago
|
|
|
||
Updated•3 years ago
|
As an owner of Intel Core2Duo CPU and a pre-GCN AMD GPU (r600 family, radeon driver) I'm strictly against of raising hardware requirements. Feel free to hide this comment as "advocacy" :)
|
||
Comment 7•2 years ago
|
||
(In reply to KOLANICH from comment #6)
As an owner of Intel Core2Duo CPU and a pre-GCN AMD GPU (r600 family, radeon driver) I'm strictly against of raising hardware requirements. Feel free to hide this comment as "advocacy" :)
If it's only about the build, you could take over the initiative and provide builds optimized for the older hardware or provide maintainable solution to enable both path (for old and new HW).
In place where I live finding a C2D CPU is pretty hard, but I believe there are lot of them running somewhere for sure and if the effort you put into it will gain community, why not.
|
|
||
Comment 8•2 years ago
|
||
I spoke with Yury on Slack, it isn't just a matter of enabling msse4.1 instead of mssse3 in https://github.com/browsermt/bergamot-translator/blob/321be8ae0486de3af67307c4cb2e005994593597/CMakeLists.txt#L127, we would actually have to modify the source code of the program to use Wasm SIMD instructions.
https://emscripten.org/docs/porting/simd.html has more details.
As a first step and for a potential quick win, we could try benchmarking the various options:
- only msimd128 and no mssse3;
- msimd128 + one of https://emscripten.org/docs/porting/simd.html#compiling-simd-code-targeting-x86-sse-instruction-sets (e.g. maybe mssse4.1 is not enough to have perfect Wasm SIMD, but it might be better than mssse3).
|
|
||
Comment 9•2 years ago
|
||
I couldn't see any difference between those options with a simple benchmark.
Description
•