Closed Bug 1848121 Opened 4 months ago Closed 4 months ago

Timeout multiplier for DEBUG and TSAN builds are too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures

Categories

(Testing :: web-platform-tests, defect, P1)

Product:
Testing
Component:
web-platform-tests
Version:
Default
Type:
defect
Points:
2

Tracking

(firefox-esr115 fixed, firefox118 fixed)

Status:
RESOLVED FIXED
Milestone:
118 Branch
Tracking Flags:
Tracking Status
firefox-esr115 --- fixed
firefox118 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

(Blocks 2 open bugs)

Details

(Whiteboard: [webdriver:m8], [wptsync upstream])

Attachments

(1 file)

Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.
48 bytes, text/x-phabricator-request
RyanVM
: approval-mozilla-esr115+
Details | Review

As noticed while working on bug 1847578 the timeout multiplier for TSAN builds is equal to the one for ASAN builds. But at least on Linux the tests are running around twice that long.

To prove that I fetched the timings for all the tests and added them to this spreadsheet:

https://docs.google.com/spreadsheets/d/1d14yxniODL-BpJPf07hOzoHTRucb4pw-XT0ivp9ksWs/edit#gid=1980929651

As such I would propose to increase TSAN multiplier by the factor of 2x for its current value of 4. This is also in the range of 5-15% slower rate as mentioned in the llvm docs.

Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Points: --- → 2
Priority: -- → P1
Summary: Timeout multiplier for TSAN builds too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures → Timeout multiplier for TSAN builds is too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures
Whiteboard: [webdriver:m8]
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d8efed7abad1
[wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8. r=jgraham
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/41425 for changes under testing/web-platform/tests
Whiteboard: [webdriver:m8] → [webdriver:m8], [wptsync upstream]
Blocks: 1630162

https://hg.mozilla.org/mozilla-central/rev/d8efed7abad1

Status: ASSIGNED → RESOLVED
Closed: 4 months ago
status-firefox118: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 118 Branch

The timeout multiplier for DEBUG builds has been increased as well to be 4 instead of 3 in all the cases.

Summary: Timeout multiplier for TSAN builds is too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures → Timeout multiplier for DEBUG and TSAN builds are too low and causing a lot of TEST-UNEXPECTED-TIMEOUT failures
Blocks: 1783217
Upstream PR merged by moz-wptsync-bot
Severity: -- → S3
Blocks: 1866578

Comment on attachment 9348378 [details]
Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.

ESR Uplift Approval Request

  • If this is not a sec:{high,crit} bug, please state case for ESR consideration: Fixes a perma wdspec failure on the esr-115 branch.
  • User impact if declined: None
  • Fix Landed on Version: 118
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Just changes the wptrunner's multiplier value to allow TSAN jobs to run longer.
Attachment #9348378 - Flags: approval-mozilla-esr115?

Comment on attachment 9348378 [details]
Bug 1848121 - [wptrunner] Increase timeout multiplier for TSAN builds from 4 to 8.

Technically test-only and not needing approval, but a=me for good measure.

Attachment #9348378 - Flags: approval-mozilla-esr115? → approval-mozilla-esr115+
You need to log in before you can comment on or make changes to this bug.