1526086 - Fennec can shutdown-hang trying to write the startup cache when running mochitest-plain

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, enhancement, P2)

Description

[:Gijs (he/him)]

After bug 1445739, we no longer invalidate the startup cache when installing add-ons. This is affecting mochitests on android, which hang on exit when the tests have completed, cf. https://treeherder.mozilla.org/#/jobs?repo=try&revision=1d216404bea7856e71ab9b9c75446babbce10e4c&selectedJob=226902286 .

Confusingly, this is happening on opt (and not on debug), and not 100% of the time, and not for all test runs. I've struggled to repro locally. It's possible this is a consequence of how long the browser has been up yet and whether it's had time to do any writing up till that point. Still, it probably shouldn't be taking 300+ seconds to write that cache (even on an emulator "SD card") and this seems buggy.

Because we're not actually changing breaking anything that isn't already broken in bug 1445739 (it just so happens that we install add-ons to run mochitests etc.), I'm likely to land bug 1445739 with a workaround that continues to invalidate the startup cache, but ideally this should be investigated and fixed "properly".

Comment 2

[Kris Maglione [:kmag]]

Attachment: Bug 1526086: Fix possible shutdown deadlock when writing preloader cache. r=erahm

There are currently some odd circumstances where we deadlock waiting for the
background save thread to finish while it is blocked on sync dispatch to the
main thread during shutdown.

There were existing workarounds to prevent this, which tried to synchronously
complete the main thread work required by the background thread at the start
of shutdown, and some fallback anti-deadlock assertions to catch any remaining
corner cases, but apparently Fennec has corner cases of its own that we didn't
anticipate.

This patch takes the more straightforward route of using an async shutdown
blocker, which allows the async shutdown service to safely spin the event loop
until the save completes, rather than an independent monitor loop, which does
not.

It also fixes a potential data race where the save thread could clear its
mSaveThread member before NS_NewNamedThread returned, running afoul of
nsCOMPtr sanity checks.

Comment 3

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ad8ea6ea53e9ec9df9ff49db568fa174c74daf36
Bug 1526086: Fix possible shutdown deadlock when writing preloader cache. r=erahm

Comment 4

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/ad8ea6ea53e9