1848121 - Timeout multiplier for DEBUG and TSAN builds are too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As noticed while working on bug 1847578 the timeout multiplier for TSAN builds is equal to the one for ASAN builds. But at least on Linux the tests are running around twice that long.

To prove that I fetched the timings for all the tests and added them to this spreadsheet:

https://docs.google.com/spreadsheets/d/1d14yxniODL-BpJPf07hOzoHTRucb4pw-XT0ivp9ksWs/edit#gid=1980929651

As such I would propose to increase TSAN multiplier by the factor of 2x for its current value of 4. This is also in the range of 5-15% slower rate as mentioned in the llvm docs.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d8efed7abad1
[wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8. r=jgraham

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/41425 for changes under testing/web-platform/tests

Comment 4

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/d8efed7abad1

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The timeout multiplier for DEBUG builds has been increased as well to be 4 instead of 3 in all the cases.

Comment 6

[Intermittent Failures Robot]

2 failures in 3341 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform and build breakdown:

• linux1804-64-tsan-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1848121&startday=2023-08-07&endday=2023-08-13&tree=all

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 9348378 [details]
Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: Fixes a perma wdspec failure on the esr-115 branch.
• User impact if declined: None
• Fix Landed on Version: 118
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Just changes the wptrunner's multiplier value to allow TSAN jobs to run longer.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9348378 [details]
Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.

Technically test-only and not needing approval, but a=me for good measure.

Comment 10

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/15b12fe4522e