Closed Bug 1313187 Opened 3 years ago Closed 3 years ago

Increase default timeout for browser chrome tests when run on asan

Categories

(Testing :: Mochitest, defect)

Version 3
defect
Not set

Tracking

(firefox52 fixed)

RESOLVED FIXED
mozilla52
Tracking Status
firefox52 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(1 file, 1 obsolete file)

Bug 1292233 and bug 1297575 are examples of frequent intermittent timeouts in browser chrome tests which occur almost exclusively on linux64-asan. These tests run quickly (<5 seconds) on most platforms, but usually take nearly 45 seconds on asan; if they take > 45 seconds, they time out.

I don't think the tests are at fault: I see many tests running 10x as long on asan as on opt. We could avoid the timeouts with requestLongerTimeout(), but that seems inappropriate on all platforms except asan. We could skip the problematic tests on asan, but I'd prefer not to lose that coverage on asan.

Let's try increasing the 45 second default timeout on asan only.
(Oops - now with less debug logging!)
Attachment #8804892 - Attachment is obsolete: true
Attachment #8804892 - Flags: review?(jmaher)
Attachment #8804893 - Flags: review?(jmaher)
Comment on attachment 8804893 [details] [diff] [review]
increase default timeout for browser mochitests to 90 seconds, on asan only

Review of attachment 8804893 [details] [diff] [review]:
-----------------------------------------------------------------

I am fine with your approach, also would be happy moving this to another location- please consider it and land what you feel is right

::: testing/mochitest/runtests.py
@@ +2180,5 @@
> +        # browser-chrome tests use a fairly short default timeout of 45 seconds;
> +        # this is sometimes too short on asan, where we expect reduced performance.
> +        if mozinfo.info["asan"] and options.flavor == 'browser' and options.timeout is None:
> +            self.log.info("Increasing default timeout to 90 seconds on ASAN")
> +            options.extraPrefs.append("testing.browserTestHarness.timeout=90")

could we move this closer to:
https://dxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#1622

possible if we are in ASAN and not options.timeout:
  options.timeout = 90

?
Attachment #8804893 - Flags: review?(jmaher) → review+
OK, I'll move the code beside the other reference to testing.browserTestHarness.timeout - that seems quite sensible.

I had some trouble using options.timeout instead of testing.browserTestHarness.timeout. I'm not sure I understand, but I think that options.timeout = None results in a larger (>90s) timeout for the python harness. I'll just change testing.browserTestHarness.timeout (the js timeout) for now.
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ef2f7a67a987
Increase default timeout on ASAN browser mochitests to 90 seconds; r=jmaher
https://hg.mozilla.org/mozilla-central/rev/ef2f7a67a987
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
Duplicate of this bug: 1297575
Duplicate of this bug: 1292233
You need to log in before you can comment on or make changes to this bug.