1657850 - Add a pref to control how many background threads are used

Status: RESOLVED FIXED

(Core :: JavaScript: GC, task)

Description

[Jon Coppeard (:jonco)]

The GC uses background threads to parallelise work where possible. How many threads to use is an open question however. In general, using more threads helps but this is limited by memory bandwidth, resource contention and other CPU activity. Even when more threads do give an improvement there are likely to be diminishing returns, while potentially negatively affecting other parts of the system.

Currently we use a maximum of half as many threads as there are CPU cores.

We should add a pref to allow us to experiment with changing this ratio.

Comment 1

[Jon Coppeard (:jonco)]

Attachment: Bug 1657850 - Add prefs to control how many background threads are used for GC r?sfink

This adds a thread count for GC parallel tasks calculated from GC parameters
for a helper thread ratio and max helper thread count. It also adds one to
report the number of helper threads used for GC.

This is slightly complicated by the fact that the helper thread system is
per-process and there are potentially many JS runtimes in a process. I
disallowed setting these parameters from workers (i.e. child JS runtimes), but
there may be more than one non-worker JS runtime so this isn't perfect.

I had to swap the mutex order for the GC and helper thread locks. Whatever
reason they were the other way round seems to have gone and this order makes
more sense to me (I see the GC lock as being 'coarser' than the helper thread
lock).

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fb664f6d43ed
Add prefs to control how many background threads are used for GC r=sfink

Comment 3

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f46205a42fae
Backed out changeset fb664f6d43ed for failures on helper-thread-params.js. CLOSED TREE

Comment 4

[Natalia Csoregi [:nataliaCs]]

Backed out for failures on helper-thread-params.js

backout: https://hg.mozilla.org/integration/autoland/rev/f46205a42faecda0e5a73bdcde0a8e1caa0126d9

push: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedTaskRun=YcEyrsL_R0GwS_1ZNJRPUA.0&revision=fb664f6d43ed943abc6effa00214073365f34fa4

failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312945664&repo=autoland&lineNumber=25192

[task 2020-08-13T18:18:28.711Z] 18:18:28 INFO - TEST-PASS | tests/jit-test/jit-test/tests/gc/gczeal-range.js | Success (code 0, args "--blinterp-eager") [0.1 s]
[task 2020-08-13T18:18:28.729Z] 18:18:28 INFO - Exit code: -11
[task 2020-08-13T18:18:28.730Z] 18:18:28 INFO - FAIL - gc/helper-thread-params.js
[task 2020-08-13T18:18:28.730Z] 18:18:28 WARNING - TEST-UNEXPECTED-FAIL | tests/jit-test/jit-test/tests/gc/helper-thread-params.js | Unknown (code -11, args "") [0.1 s]
[task 2020-08-13T18:18:28.730Z] 18:18:28 INFO - INFO exit-status : -11
[task 2020-08-13T18:18:28.730Z] 18:18:28 INFO - INFO timed-out : False
[task 2020-08-13T18:18:28.734Z] 18:18:28 INFO - Exit code: -11
[task 2020-08-13T18:18:28.738Z] 18:18:28 INFO - FAIL - gc/helper-thread-params.js

Comment 5

[Jon Coppeard (:jonco)]

Attachment: Bug 1657850 - Add an uninitialized JSContext kind to simplify protected data checks r?jandem

Currently we hack around the protected data checks during helper thread
initialization in GlobalHelperThreadState::ensureContextListForThreadCount.
This removes the need to do that and allows us to simplify that method. I added
some more assertions in JSContext::set/clearHelperThread too.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b528e9b6bab0
Add an uninitialized JSContext kind to simplify protected data checks r=jandem
https://hg.mozilla.org/integration/autoland/rev/39e729cc2e67
Add prefs to control how many background threads are used for GC r=sfink

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/b528e9b6bab0
https://hg.mozilla.org/mozilla-central/rev/39e729cc2e67

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Backed out from Beta at Jon's request.
https://hg.mozilla.org/releases/mozilla-beta/rev/12fd47cd84fc