1933997 - Running Firefox's inference on multiple threads appears to be in WASM(Baseline) on worker threads.

Status: NEW

(Core :: JavaScript: WebAssembly, task, P3)

Description

[Mayank Bansal]

Attachment: inference settings.png

Full disclosure: I dont know what I am doing.

Profile (with lazy tiering enabled): https://share.firefox.dev/4g9SOf4
Profile (with lazy tiering disabled): https://share.firefox.dev/3B9b3m2

Profiler indicates that the threads appear to run in WASM(Baseline). Either it should run in the fastest tiers, or the profiler is showing incorrect information.

Comment 1

[Tarek Ziadé (:tarek)]

The ONNX runtime we use is in WASM and your profile looks right to me. What do you mean by fastest tiers?

Comment 2

[Tarek Ziadé (:tarek)]

oh you mean like ION tiers? Is it supposed to be in a different category in the profiler?

Have you tried to run the inference several times? maybe it was not compiled yet in ION

Comment 3

[Mayank Bansal]

(In reply to Tarek Ziadé (:tarek) from comment #2)

Have you tried to run the inference several times? maybe it was not compiled yet in ION

Profile with 2 runs: https://share.firefox.dev/3Vfv9Sr - again, the profiler shows that all the threads spend time in WASM(Baseline).

It could be that the default combination of models/quantization levels is not well supported in WASM. Which is why I filed bug in WASM component and cc'd you. Attached screenshot of the inference settings.

Comment 4

[Ryan Hunt [:rhunt]]

Ah, this is probably a bug in our integration with the profiler in the new lazy tiering system. We don't report the correct tier here [1].

I'll see if I can write a quick patch to fix this.

[1] https://searchfox.org/mozilla-central/rev/6597dd03bad82c891d084eed25cafd0c85fb333e/js/src/wasm/WasmCode.cpp#1384

Comment 5

[Mayank Bansal]

(In reply to Ryan Hunt [:rhunt] from comment #4)

Ah, this is probably a bug in our integration with the profiler in the new lazy tiering system. We don't report the correct tier here [1].

I'll see if I can write a quick patch to fix this.

[1] https://searchfox.org/mozilla-central/rev/6597dd03bad82c891d084eed25cafd0c85fb333e/js/src/wasm/WasmCode.cpp#1384

Note that I have updated comment 0 with profiles of lazy tiering disabled and enabled. In both cases, the profiler shows WASM(Baseline).
Additionally, in the lazy tier enabled profile, a lot of the time on the content-process mainthread is shown as WASM(Other).

Comment 6

[Ryan Hunt [:rhunt]]

(In reply to Mayank Bansal from comment #5)

(In reply to Ryan Hunt [:rhunt] from comment #4)

Ah, this is probably a bug in our integration with the profiler in the new lazy tiering system. We don't report the correct tier here [1].

I'll see if I can write a quick patch to fix this.

[1] https://searchfox.org/mozilla-central/rev/6597dd03bad82c891d084eed25cafd0c85fb333e/js/src/wasm/WasmCode.cpp#1384

Note that I have updated comment 0 with profiles of lazy tiering disabled and enabled. In both cases, the profiler shows WASM(Baseline).
Additionally, in the lazy tier enabled profile, a lot of the time on the content-process mainthread is shown as WASM(Other).

Looking at your lazy tiering disabled profile, I do see some Ion code running there. Click on the DOM worker that starts first and there's about 6% of the time spent in Ion. The ONNX module takes a while to compile with Ion, so tier up time can take a while.

Nearly all of the wasm(other) time appears to be wasm doing a wait operation on shared memory (e.g. blocking on another thread using a mutex).

Comment 7

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:rhunt, could you have a look please?

For more information, please visit BugBot documentation.

Comment 8

[Mayank Bansal]

This is what i get with latest Nightly (with bug 1934663 fixed)
Lazy Tiering disabled : https://share.firefox.dev/3WEHSPn (almost all the time is in Baseline)
Lazy tiering enabled: https://share.firefox.dev/3ElB6rB (major part in Baseline, with ion interspersed)
Lazy Tiering + 8 threads: https://share.firefox.dev/3CGNPV5 (All the DOMWorkers spend 100% of time in Baseline)