1901294 - Increase extensions.background.idle.timeout in unit tests, at least for slow tests like asan / TV

Status: NEW

(WebExtensions :: General, task, P2)

Description

[Rob Wu [:robwu]]

I have recently noticed several test failures, ranging from unexpected messages to timeouts that all boil down to the issue of the test extension's background script (event page) being shut down unexpectedly in the middle of the test execution.

Examples:

• Bug 1894980 - test timeout happened because the test was expecting an event (permissions.onAdded) to fire in response to a permission change, but did not happen because the background script had shut down in the meantime, so the listening state was reset (https://hg.mozilla.org/integration/autoland/rev/bb0d32ef0c26#l1.20)
• Bug 1856265 - test failure happened because the background script sends an initialization message, and due the event page starting again, the message unexpectedly arrives twice: https://bugzilla.mozilla.org/show_bug.cgi?id=1856265#c7

To avoid this noise in intermittent failures, we should set extensions.background.idle.timeout to a high value, e.g. 300000 (5 minutes, up from the default 30 seconds). At least for builds that are known to be slow such as asan, tsan, MOZ_CHAOSMODE (TV aka --verify). Unless there is a good reason to make it build-specific, I recommend setting the pref unconditionally so that the test behavior is consistent across different jobs. Tests that expect event page termination should explicitly set a shorter timeout to exercise the behavior (this is already done, otherwise existing tests would wait for 30 seconds for no good reason).

Note: there is a separate issue (bug 1899772) in a test utility (terminateBackground): if the terminateBackground is called while an event was still being dispatched, then the termination request is silently ignored (ext-background.js terminateBackground implementation). This can already result in unexpectedly longer running tests (+30 seconds due to the default timeout), but with the change proposed here this problem could become even more visible. I recommend fixing the test utility to throw a hard error if the termination is ignored (with an opt-out in case a test explicitly wants to test the timeout behavior).

Comment 1

[Rob Wu [:robwu]]

Bug 1777018 has a spike in ASAN failures. We're going to try setting the pref as suggested here on ASAN only for a specific test, and if that improves the failure rates, we're going to expand that pref to all asan/tsan tests if possible.