1313187 - Increase default timeout for browser chrome tests when run on asan

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Geoff Brown [:gbrown]]

Bug 1292233 and bug 1297575 are examples of frequent intermittent timeouts in browser chrome tests which occur almost exclusively on linux64-asan. These tests run quickly (<5 seconds) on most platforms, but usually take nearly 45 seconds on asan; if they take > 45 seconds, they time out.

I don't think the tests are at fault: I see many tests running 10x as long on asan as on opt. We could avoid the timeouts with requestLongerTimeout(), but that seems inappropriate on all platforms except asan. We could skip the problematic tests on asan, but I'd prefer not to lose that coverage on asan.

Let's try increasing the 45 second default timeout on asan only.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: increase default timeout for browser mochitests to 90 seconds, on asan only

https://treeherder.mozilla.org/#/jobs?repo=try&revision=86e59b1f005a6cc6f191bdd18adece56d6442a6e looks good!

Comment 2

[Geoff Brown [:gbrown]]

Attachment: increase default timeout for browser mochitests to 90 seconds, on asan only

(Oops - now with less debug logging!)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8804893 [details] [diff] [review]
increase default timeout for browser mochitests to 90 seconds, on asan only

Review of attachment 8804893 [details] [diff] [review]:
-----------------------------------------------------------------

I am fine with your approach, also would be happy moving this to another location- please consider it and land what you feel is right

::: testing/mochitest/runtests.py
@@ +2180,5 @@
> +        # browser-chrome tests use a fairly short default timeout of 45 seconds;
> +        # this is sometimes too short on asan, where we expect reduced performance.
> +        if mozinfo.info["asan"] and options.flavor == 'browser' and options.timeout is None:
> +            self.log.info("Increasing default timeout to 90 seconds on ASAN")
> +            options.extraPrefs.append("testing.browserTestHarness.timeout=90")

could we move this closer to:
https://dxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#1622

possible if we are in ASAN and not options.timeout:
  options.timeout = 90

?

Comment 4

[Geoff Brown [:gbrown]]

OK, I'll move the code beside the other reference to testing.browserTestHarness.timeout - that seems quite sensible.

I had some trouble using options.timeout instead of testing.browserTestHarness.timeout. I'm not sure I understand, but I think that options.timeout = None results in a larger (>90s) timeout for the python harness. I'll just change testing.browserTestHarness.timeout (the js timeout) for now.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ef2f7a67a987
Increase default timeout on ASAN browser mochitests to 90 seconds; r=jmaher

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ef2f7a67a987