1472987 - add wasm benchmark to raptor or shell benchmarks

Status: RESOLVED DUPLICATE of bug 1481708

(Testing :: Raptor, enhancement)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

the JS team would like a wasm benchmark to be run as we migrate away from AWFY.  There is WasmMisc which runs on AWFY (if you are logged into arewefastyet.com, you can view it on a windows browser config as Wasm), although I cannot find reference to the source or how it was created.

My understanding is we want to continue running this, but privately- against both firefox and chrome, ideally on windows.

we should be able to run firefox in different wasm modes here:
* regular (Ion)
* wasm baseline
* wasm tiering

and just chrome as chrome.

You can see the prefs to adjust firefox here:
https://github.com/mozilla/arewefastyet/blob/34d7366a4a479fcf88624ee75b13597095eb7821/slave/configs.py#L53

I would need to get confirmation of where this benchmark source is located at and if this is what we really want to run.  Also if this is run and reported on our public infrastructure is that OK?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:overholt, can you help me get more information about the wasm testing we want done.  I am trying to get things actionable so we can do them this month- this is one of the benchmarks I have not been able to figure out easily.

Comment 2

[Benjamin Bouvier [:bbouvier] (inactive)]

For what it's worth, I have access to the benchmark's code. Definitely happy to give it to you. Now, we would like to keep it protected / behind LDAP user/password access, because we (still?) don't want to start a benchmarketing war with other browser vendors. Especially since these micro-benchmarks are not very meaningful and might reflect some issues that are not relevant to implementations (e.g. issues in Emscripten, lack of ahead of time optimizations, etc.).

Also, it might be worth waiting for a bit, because people at PSPDFKit seem to be making a wasm benchmark out of a real-world use case. It is probably the kind of good benchmarks we could even show publicly, I think.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

ok, lets get everything else done first and revisit this.  Thanks for the information :bbouvier!