Open Bug 1956935 Opened 8 months ago Updated 5 days ago

[meta] Intermittent bugs with WPT TEST-UNEXPECTED-TIMEOUT and "Got timeout in harness", for some random test that has no reason to have timed out

Categories

(Testing :: web-platform-tests, task)

Product:
Testing
Component:
web-platform-tests
Type:
task

Tracking

(Not tracked)

People

(Reporter: dholbert, Unassigned)

References

(Depends on 18 open bugs)

Details

(Keywords: meta)

I've noticed an uptick in new bugs being filed in layout components for intermittent WPT test timeouts, often with "Got timeout in harness" and/or "TestRunner hit external timeout (this may indicate a hang)". We've had at least one such bug filed per day for the past few days (with a handful all coming in on 3/24 for whatever reason).

They're usually for a test that's not doing anything particularly slow/interesting, which makes me suspect a bug in the test harness or in the configuration (or some action-at-a-distance where e.g. the web server fell over, or something along those lines).

Examples:
bug 1956915 (filed 3/27)
bug 1956471 (filed 3/26)
bug 1956214 (filed 3/25)
bug 1956167 (filed 3/24)
bug 1956166 (filed 3/24)
bug 1956142 (filed 3/24)
bug 1956076 (filed 3/24)
bug 1955949 (filed 3/24)
bug 1955823 (filed 3/23)

Not sure what's going on, but it seems like a pattern that's worth investigating...

Note that a lot of these are for a ccov (code coverage) build, which is inherently slow. (Sometimes but not always these are labeled in the bug title as [tier 2]). So I initially thought that this might be "just" the regular code-coverage slowness, but it's not that... As one example: in the most recent bug that I listed, bug 1956915, the log shows that the previous test took less than 1 second to execute; and then we loaded the test text-align-inline-end-crash.html (which is trivial, and doesn't actually do anything dynamic or interesting, and loads instantly on Nighlty) and it timed out after 30 seconds of sitting. I strongly doubt that the timeout was from us just struggling to load the test in 30 seconds with the regular CCOV overhead, so I wonder what's really timing out and breaking here...

Summary: [meta] Intermittent bugs with "Got timeout in harness" in some random test that has no reason to have timed out → [meta] Intermittent bugs with WPT TEST-UNEXPECTED-TIMEOUT and "Got timeout in harness", for some random test that has no reason to have timed out
Depends on: 1958098

In bug 1958098, this issue happened (in a ccov build) for two tests back-to-back.

I was thinking today that maybe this was due to some background task that gets kicked off in the parent process, after some delay, and that background task happens to block the parent process's main thread (or some other critical thread) until it's complete -- and it happens to take long enough when run under on a CCOV build that it times out whatever test is running... But I think that's not actually the right explanation, because it seems that the CPU is idle for the whole time that the we're sitting waiting for these tests to timeout as noted at the end of bug 1958098 comment 1 (see the resource-usage profiler link there).

Depends on: 1958184
Depends on: 1958355
Depends on: 1958455, 1958459

There's been a recent spike on Android 14 for wpts that are timing out
Seems to have started since this pushlog as you can see here

Depends on: 1988337, 1988336, 1988335, 1988334
See Also: → 1988591
You need to log in before you can comment on or make changes to this bug.