1713287 - Parallel wasm compilation makes assumptions about the number of helper threads

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, task, P3)

Description

[Jon Coppeard (:jonco)]

Parallel wasm compilation depends on being able to run a master task that generates work at the same times as least one other task to consume the work, otherwise we can deadlock. Currently we always create at least two threads when the helper thread system is in use, but when we move to using an external thread pool that will no longer be the case.

We should check the number of threads before started parallel compilation. The helper thread system will ensure that the number of threads reported is correct when using an external thread pool.

Comment 1

[Jon Coppeard (:jonco)]

Attachment: Bug 1713287 - Check helper thread count before starting parallel wasm compilation r?lth

This adds a check that there is more than one helper thread before starting
parallel compilation. It also changes the number of helper threads created so
there's only a single thread on unicore systems, which will be the case when
using the external thread pool.

Depends on D116220

Comment 2

[Jon Coppeard (:jonco)]

Attachment: Bug 1713287 - Relax the assumtion that there will be at least as many helper threads as cores r?lth

This won't necessarily be true when using an external thread pool.

The maximum number of Wasm tasks is now limited by the number of cores though.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ea0e06d3b037
Check helper thread count before starting parallel wasm compilation r=lth
https://hg.mozilla.org/integration/autoland/rev/99afd8d13d30
Relax the assumtion that there will be at least as many helper threads as cores r=lth

Comment 4

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/963df76dc655
Backed out 2 changesets for causing build bustages. CLOSED TREE

Comment 5

[Marian-Vasile Laza]

Backed out for causing build bustages

Push with failures

Failure Log

Backout link

Comment 6

[Jon Coppeard (:jonco)]

It seems that skipping parallel Wasm compliation when there's only a single thread is not enough to make this work. There are are still deadlocks when waiting for off-thread promise helper tasks with the previous patches in this bug.

Maybe a better approach is to make the XPCOM pool more like the current JS helper thread system and ensure that there are at least two threads at all times. This would also reduce the chance of performance regressions when switching over.

Comment 7

[Jon Coppeard (:jonco)]

Attachment: Bug 1713287 - Change the number of threads TaskController creates to match those required by the JS engine r?bas

Currently parallel Wasm compilation requires at least two threads for
architectural reasons. This patch updates the TaskController thread policy such
that there are always two threads available, even on single core systems.

Comment 8

[Jon Coppeard (:jonco)]

Attachment: Bug 1713287 - Add test for Wasm compilation with a single core r?lth

Depends on D116620

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ade4570ab563
Change the number of threads TaskController creates to match those required by the JS engine r=bas
https://hg.mozilla.org/integration/autoland/rev/9bc99e4fcbff
Relax the assumtion that there will be at least as many helper threads as cores r=lth
https://hg.mozilla.org/integration/autoland/rev/31567d7bd975
Add test for Wasm compilation with a single core r=lth

Comment 10

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/ade4570ab563
https://hg.mozilla.org/mozilla-central/rev/9bc99e4fcbff
https://hg.mozilla.org/mozilla-central/rev/31567d7bd975