1634720 - Calling HelperThreadTaskCallback causes timeout in xpcshell/selftest.py on linux

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Yoshi Cheng-Hao Huang [:allstars.chh][:allstarschh][:yoshi]]

See my sample patch in https://hg.mozilla.org/try/rev/437f097aeb4ea0d9d46de94025738dfeab714894

it's just calling HelperThreadTaskCallback, however, on linux platform it got timeout when running selftest.py

https://treeherder.mozilla.org/#/jobs?repo=try&revision=bb7493cb49abff71cdb97b57b7b12f17b92b24f3

running the test locally on a linux machine can also reproduce this problem
./mach python-test testing/xpcshell/selftest.py
(It took a while, because most of the tests got timeout)

Kris, do you know this problem when you landed the HelperThreadTaskCallback?
Thanks

Comment 1

[Kris Wright :KrisWright]

This is the same problem that causes assertions in bug 1632825 and it's related to the way thread pools shut down (bug 1633548). With the pool unable to service tasks if nsThreadManager isn't making new threads, the event queue gets stuck. Even after it asserts, the thread pool still can't shut down because the event queue is still stuck, so we end up with a hang.

Comment 2

[Yoshi Cheng-Hao Huang [:allstars.chh][:allstarschh][:yoshi]]

As it turned out this is not the same problem as in Bug 1632825. I've tried even not to dispatch task to HelperThreadPool after nsThreadManager shutdown, and the situtation is still there.

When nsThreadManager shuts down, it will shutdown all the threads,
including the main thread of HelperThreadPool.

However the main thread of HelperThreadPool has been waiting.
https://searchfox.org/mozilla-central/rev/446160560bf32ebf4cb7c4e25d7386ee22667255/xpcom/threads/nsThreadPool.cpp#277

and nsThreadManager will have to wait until HelperThreadPool wakes up to shutdown the main thread of HelperThreadPool.

I fix this by adding a HelperThreadPoolShutdownObserver to observe the xpcom-shutdown-threads notification, and shut down HelperThreadPool
accordingly.

Comment 3

[Yoshi Cheng-Hao Huang [:allstars.chh][:allstarschh][:yoshi]]

Attachment: Bug 1634720 : Add HelperThreadPoolShutdownObserver to shut down HelperThreadPool.

When nsThreadManager shuts down, it will shutdown all the threads,
including the main thread of HelperThreadPool.

However the main thread of HelperThreadPool has been waiting.
https://searchfox.org/mozilla-central/rev/446160560bf32ebf4cb7c4e25d7386ee22667255/xpcom/threads/nsThreadPool.cpp#277

and nsThreadManager will have to wait until HelperThreadPool wakes up to
shutdown the main thread of HelperThreadPool.

Adding a HelperThreadPoolShutdownObserver to observe the
xpcom-shutdown-threads notification, and shut down HelperThreadPool
accordingly.

Comment 4

[Yoshi Cheng-Hao Huang [:allstars.chh][:allstarschh][:yoshi]]

Attachment: Bug 1634720 - use UniquePtr::reset to release the pointer.

After fixing the timeout, I found there's a leak for the RunnableTask
dispatched to HelperThreadPool.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5a7274adecec
Add HelperThreadPoolShutdownObserver to shut down HelperThreadPool. r=KrisWright
https://hg.mozilla.org/integration/autoland/rev/d00b78d47707
use UniquePtr::reset to release the pointer. r=KrisWright

Comment 6

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/5a7274adecec
https://hg.mozilla.org/mozilla-central/rev/d00b78d47707