1686581 - [wpt-sync] Sync PR 27175 - [manifest] Explicitly context-manage the Pool

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P4)

Description

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Sync web-platform-tests PR 27175 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/27175
Details from upstream follow.

b'Stephen McGruer <[e-mail]>' wrote:

[manifest] Explicitly context-manage the Pool

Python requires that a Pool object be context managed
(https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool):

Warning: multiprocessing.pool objects have internal resources that need
to be properly managed (like any other resource) by using the pool as a
context manager or by calling close() and terminate() manually. Failure
to do this can lead to the process hanging on finalization.

Note that is not correct to rely on the garbage colletor to destroy the
pool as CPython does not assure that the finalizer of the pool will be
called (see object.__del__() for more information).

Chromium has been seeing hangs in 'wpt manifest' invocations since
moving to Python 3 for WPT, which sounds a lot like 'Failure to do this
can lead to the process hanging on finalization.' So let's
context-manage our Pool properly :)

Comment 1

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Pushed to try https://treeherder.mozilla.org/#/jobs?repo=try&revision=f1e02db128c82add6a4c74e45a59251fc2ca71ab

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c0161b95e29e
[wpt PR 27175] - [manifest] Explicitly terminate the multiprocessing.Pool, a=testonly

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Test result changes from PR not available.

Comment 4

[James Graham [:jgraham]]

Attachment: Bug 1686581 [wpt PR 27175] - [manifest] Explicitly terminate the multiprocessing.Pool, a=testonly

Automatic update from web-platform-tests
[manifest] Explicitly terminate the multiprocessing.Pool

Python requires that a Pool object be terminated before it is GC'd
(https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool):

Warning: multiprocessing.pool objects have internal resources that need
to be properly managed (like any other resource) by using the pool as a
context manager or by calling close() and terminate() manually. Failure
to do this can lead to the process hanging on finalization.

Note that is not correct to rely on the garbage colletor to destroy the
pool as CPython does not assure that the finalizer of the pool will be
called (see object.__del__() for more information).

Chromium has been seeing hangs in 'wpt manifest' invocations since
moving to Python 3 for WPT, which sounds a lot like 'Failure to do this
can lead to the process hanging on finalization.' So let's
terminate our Pool properly :)

--

wpt-commits: eb6aa97d61d689ee572ce0975315dcd36d9068e7
wpt-pr: 27175

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ada517987113
[wpt PR 27175] - [manifest] Explicitly terminate the multiprocessing.Pool, a=testonly

Comment 6

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Test result changes from PR not available.

Comment 7

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/ada517987113