Perma [tier 2] Headless TEST-UNEXPECTED-OK | /webdriver/tests/interop/beforeunload_prompt.py | expected TIMEOUT
Categories
(Remote Protocol :: Marionette, defect, P5)
Tracking
(firefox-esr128 unaffected, firefox-esr140 unaffected, firefox142 unaffected, firefox143 unaffected, firefox144 disabled, firefox147 disabled, firefox148 disabled, firefox149 fixed)
| Tracking | Status | |
|---|---|---|
| firefox-esr128 | --- | unaffected |
| firefox-esr140 | --- | unaffected |
| firefox142 | --- | unaffected |
| firefox143 | --- | unaffected |
| firefox144 | --- | disabled |
| firefox147 | --- | disabled |
| firefox148 | --- | disabled |
| firefox149 | --- | fixed |
People
(Reporter: intermittent-bug-filer, Assigned: Sasha)
References
(Regression)
Details
(Keywords: intermittent-failure, regression, test-disabled)
Attachments
(1 external file)
Filed by: agoloman [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=523381241&repo=mozilla-central&task=J3YywzWiThGd0_XrAgUqVQ.0
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/J3YywzWiThGd0_XrAgUqVQ/runs/0/artifacts/public/logs/live_backing.log
[task 2025-08-21T11:36:07.840+00:00] 11:36:07 INFO - STDOUT: =================================== FAILURES ===================================
[task 2025-08-21T11:36:07.840+00:00] 11:36:07 INFO - STDOUT: _________________________________ test_accept __________________________________
[task 2025-08-21T11:36:07.841+00:00] 11:36:07 INFO - STDOUT: fut = <Future cancelled>, timeout = 8.0
[task 2025-08-21T11:36:07.842+00:00] 11:36:07 INFO - STDOUT: async def wait_for(fut, timeout):
[task 2025-08-21T11:36:07.843+00:00] 11:36:07 INFO - STDOUT: """Wait for the single Future or coroutine to complete, with timeout.
[task 2025-08-21T11:36:07.843+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.844+00:00] 11:36:07 INFO - STDOUT: Coroutine will be wrapped in Task.
[task 2025-08-21T11:36:07.844+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.845+00:00] 11:36:07 INFO - STDOUT: Returns result of the Future or coroutine. When a timeout occurs,
[task 2025-08-21T11:36:07.846+00:00] 11:36:07 INFO - STDOUT: it cancels the task and raises TimeoutError. To avoid the task
[task 2025-08-21T11:36:07.846+00:00] 11:36:07 INFO - STDOUT: cancellation, wrap it in shield().
[task 2025-08-21T11:36:07.847+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.848+00:00] 11:36:07 INFO - STDOUT: If the wait is cancelled, the task is also cancelled.
[task 2025-08-21T11:36:07.849+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.849+00:00] 11:36:07 INFO - STDOUT: If the task suppresses the cancellation and returns a value instead,
[task 2025-08-21T11:36:07.850+00:00] 11:36:07 INFO - STDOUT: that value is returned.
[task 2025-08-21T11:36:07.851+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.851+00:00] 11:36:07 INFO - STDOUT: This function is a coroutine.
[task 2025-08-21T11:36:07.852+00:00] 11:36:07 INFO - STDOUT: """
[task 2025-08-21T11:36:07.852+00:00] 11:36:07 INFO - STDOUT: # The special case for timeout <= 0 is for the following case:
[task 2025-08-21T11:36:07.853+00:00] 11:36:07 INFO - STDOUT: #
[task 2025-08-21T11:36:07.854+00:00] 11:36:07 INFO - STDOUT: # async def test_waitfor():
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: # func_started = False
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: #
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: # async def func():
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: # nonlocal func_started
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: # func_started = True
[task 2025-08-21T11:36:07.857+00:00] 11:36:07 INFO - STDOUT: #
[task 2025-08-21T11:36:07.858+00:00] 11:36:07 INFO - STDOUT: # try:
[task 2025-08-21T11:36:07.858+00:00] 11:36:07 INFO - STDOUT: # await asyncio.wait_for(func(), 0)
[task 2025-08-21T11:36:07.859+00:00] 11:36:07 INFO - STDOUT: # except asyncio.TimeoutError:
[task 2025-08-21T11:36:07.859+00:00] 11:36:07 INFO - STDOUT: # assert not func_started
[task 2025-08-21T11:36:07.860+00:00] 11:36:07 INFO - STDOUT: # else:
[task 2025-08-21T11:36:07.861+00:00] 11:36:07 INFO - STDOUT: # assert False
[task 2025-08-21T11:36:07.861+00:00] 11:36:07 INFO - STDOUT: #
[task 2025-08-21T11:36:07.862+00:00] 11:36:07 INFO - STDOUT: # asyncio.run(test_waitfor())
[task 2025-08-21T11:36:07.862+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.863+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.864+00:00] 11:36:07 INFO - STDOUT: if timeout is not None and timeout <= 0:
[task 2025-08-21T11:36:07.864+00:00] 11:36:07 INFO - STDOUT: fut = ensure_future(fut)
[task 2025-08-21T11:36:07.865+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.865+00:00] 11:36:07 INFO - STDOUT: if fut.done():
[task 2025-08-21T11:36:07.866+00:00] 11:36:07 INFO - STDOUT: return fut.result()
[task 2025-08-21T11:36:07.866+00:00] 11:36:07 INFO - STDOUT:
[task 2025-08-21T11:36:07.867+00:00] 11:36:07 INFO - STDOUT: await _cancel_and_wait(fut)
[task 2025-08-21T11:36:07.868+00:00] 11:36:07 INFO - STDOUT: try:
[task 2025-08-21T11:36:08.025+00:00] 11:36:08 INFO - TEST-UNEXPECTED-OK | /webdriver/tests/interop/beforeunload_prompt.py | expected TIMEOUT
[task 2025-08-21T11:36:08.025+00:00] 11:36:08 INFO - TEST-INFO expected TIMEOUT | took 97474ms
[task 2025-08-21T11:36:17.994+00:00] 11:36:17 WARNING - Forcibly terminating runner process
Updated•1 year ago
|
Comment 1•1 year ago
|
||
Interesting fact here is that this test only passes (and not timeout) on mozilla-central but not on autoland and also only in headless mode. This is maybe due to some other test running before which in this constellation hasn't been run on autoland yet and some headless mode related behavior. I'll dig deeper.
Comment 2•1 year ago
|
||
In case of headless we have the following tests run, which is all what I would expect:
https://treeherder.mozilla.org/logviewer?job_id=523549797&repo=mozilla-central&task=Ip5EsfEqTC25UT47a0YUVQ.0
[task 2025-08-22T05:21:56.993+00:00] 05:21:56 INFO - TEST-START | /webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:21:57.232+00:00] 05:21:57 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept
[task 2025-08-22T05:22:10.356+00:00] 05:22:10 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept_and_notify
[task 2025-08-22T05:22:30.538+00:00] 05:22:30 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss
[task 2025-08-22T05:22:50.662+00:00] 05:22:50 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss_and_notify
[task 2025-08-22T05:23:10.560+00:00] 05:23:10 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_ignore
[task 2025-08-22T05:23:36.877+00:00] 05:23:36 INFO - TEST-UNEXPECTED-OK | /webdriver/tests/interop/beforeunload_prompt.py | expected TIMEOUT
But in case of the timeout I see some suspicious logging:
https://treeherder.mozilla.org/logviewer?job_id=523549710&repo=mozilla-central&task=BV8Cvf8pSr6vVzX7k__2BA.0
[task 2025-08-22T05:21:32.398+00:00] 05:21:32 INFO - TEST-START | /webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:21:32.615+00:00] 05:21:32 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept
[task 2025-08-22T05:21:46.771+00:00] 05:21:46 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept_and_notify
[task 2025-08-22T05:22:06.852+00:00] 05:22:06 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss
[task 2025-08-22T05:22:26.669+00:00] 05:22:26 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss_and_notify
[task 2025-08-22T05:22:46.469+00:00] 05:22:46 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_ignore
[task 2025-08-22T05:23:12.136+00:00] 05:23:12 INFO - TEST-OK | /webdriver/tests/interop/beforeunload_prompt.py | took 99685ms
[task 2025-08-22T05:23:12.208+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.210+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.340+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.343+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.492+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.494+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.641+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.643+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.796+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.798+00:00] 05:23:12 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py
[task 2025-08-22T05:23:12.849+00:00] 05:23:12 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept - webdriver.error.TimeoutException: timeout (Future did not resolve within the given timeout)
[task 2025-08-22T05:23:12.850+00:00] 05:23:12 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_accept_and_notify - webdriver.error.TimeoutException: timeout (Future did not resolve within the given timeout)
[task 2025-08-22T05:23:12.850+00:00] 05:23:12 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss - webdriver.error.TimeoutException: timeout (Future did not resolve within the given timeout)
[task 2025-08-22T05:23:12.851+00:00] 05:23:12 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_dismiss_and_notify - webdriver.error.TimeoutException: timeout (Future did not resolve within the given timeout)
[task 2025-08-22T05:23:12.852+00:00] 05:23:12 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/interop/beforeunload_prompt.py::test_ignore - webdriver.error.TimeoutException: timeout (Future did not resolve within the given timeout)
The timeout that we are seeing is coming from:
# Using WebDriver classic's navigation command to navigate away from
# the page can hang if the prompt is not accepted. As such, wrap the
# command to fail on timeout gracefully.
def sync_navigate():
current_session.url = page_target
await wait_for_future_safe(
asyncio.create_task(execute_as_async(sync_navigate)))
By default wait_for_future_safe() has a timeout of just 2s, which in case of a navigation is clearly too short. We should change that to at least use the default navigation timeout which is 3s, which gives us 12s for ASAN builds instead of just 8s due to the timeout multiplier of 4. Maybe these extra 4s will help - but still not sure why it is passing in headless mode. Maybe some widget related behavior with the beforeunload prompt.
Here a try build:
https://treeherder.mozilla.org/jobs?repo=try&revision=84d67379c9303b8f127356998260b39a1321c29f
Comment 3•1 year ago
|
||
When extending the timeout of the navigation we surprisingly now as well timeout for headless. Because we restart the browser a couple of times I would suggest that we extend the timeout similar to what we do for other user prompt tests. That should then make the test file to pass.
Lets see if that works:
https://treeherder.mozilla.org/jobs?repo=try&revision=8804f5a01f4f506a6584dd038a5c7aceefb426d5
| Comment hidden (Intermittent Failures Robot) |
Comment 5•1 year ago
|
||
Ok, so the problem here is actually that we timeout for the navigation that opens the beforeunload prompt. But instead of a shorter page load timeout of 3s * multiplier we run the full 300s now:
[task 2025-08-22T20:01:31.756+00:00] 20:01:31 INFO - PID 61880 | 1755892891754 RemoteAgent TRACE [15] No navigation found to commit for url: chrome://global/content/commonDialog.xhtml
[task 2025-08-22T20:01:32.149+00:00] 20:01:32 INFO - PID 61880 | 1755892892142 RemoteAgent TRACE Received observer notification common-dialog-loaded
[task 2025-08-22T20:01:32.149+00:00] 20:01:32 INFO - PID 61880 | 1755892892144 RemoteAgent TRACE Received observer notification common-dialog-loaded
[task 2025-08-22T20:01:32.149+00:00] 20:01:32 INFO - PID 61880 | 1755892892144 RemoteAgent TRACE [98ad9977-126e-4947-815d-36f0c7dccd5a] Navigation started for url: (56854542-6dd6-417e-abb2-b98fb2986be5)
[task 2025-08-22T20:01:32.149+00:00] 20:01:32 INFO - PID 61880 | 1755892892144 RemoteAgent TRACE Received observer notification common-dialog-loaded
[task 2025-08-22T20:01:32.153+00:00] 20:01:32 INFO - PID 61880 | 1755892892148 RemoteAgent DEBUG WebDriverBiDiConnection d0f49a3e-4cbc-463f-996a-ed0e9e78d089 <- {"type":"event","method":"browsingContext.userPromptOpened","params":{"context":"98ad9977-126e-4947-815d-36f0c7dccd5a","handler":"accept","message":"This page is asking you to confirm that you want to leave — information you’ve entered may not be saved.","type":"beforeunload"}}
[task 2025-08-22T20:01:32.946+00:00] 20:01:32 INFO - PID 61880 | -----------------------------------------------------
[task 2025-08-22T20:01:32.947+00:00] 20:01:32 INFO - PID 61880 | Suppressions used:
[task 2025-08-22T20:01:32.948+00:00] 20:01:32 INFO - PID 61880 | count bytes template
[task 2025-08-22T20:01:32.949+00:00] 20:01:32 INFO - PID 61880 | 31 16288 nsComponentManagerImpl
[task 2025-08-22T20:01:32.950+00:00] 20:01:32 INFO - PID 61880 | 3 320 libfontconfig.so
[task 2025-08-22T20:01:32.951+00:00] 20:01:32 INFO - PID 61880 | 1 9496 style::sharing::SHARING_CACHE_KEY
[task 2025-08-22T20:01:32.951+00:00] 20:01:32 INFO - PID 61880 | 1 4104 style::bloom::BLOOM_KEY
[task 2025-08-22T20:01:32.951+00:00] 20:01:32 INFO - PID 61880 | -----------------------------------------------------
[taskcluster 2025-08-22T20:04:31.978Z] [taskcluster-proxy] Successfully refreshed taskcluster-proxy credentials: task-client/SiB3Nj2ySKGfBbJNpGPfkA/0/on/us-central1-c/4977546067492281329/until/1755894271.94
[task 2025-08-22T20:06:33.135+00:00] 20:06:33 INFO - PID 61880 | 1755893193134 Marionette DEBUG 0 <- [1,6,{"error":"timeout","message":"Navigation timed out after 300000 ms","stacktrace":"RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Er ... tError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:847:5\nbail@chrome://remote/content/shared/Sync.sys.mjs:440:19\n"},null]
Because my patch uses current_session.timeouts.page_load we do not seem to setup correctly the custom page load timeout.
Comment 6•1 year ago
|
||
Because all the tests in this file wont work anyway until we have bug 1905086 implemented we potentially should disable the full test file.
Comment 7•1 year ago
|
||
Updated•1 year ago
|
Comment 11•1 year ago
|
||
| bugherder | ||
Updated•1 year ago
|
Comment 12•1 year ago
|
||
Set release status flags based on info from the regressing bug 1965465
| Comment hidden (Intermittent Failures Robot) |
Comment 15•10 months ago
|
||
This is now fixed by bug 1905086 and the test was re-enabled for headless mode.
Updated•10 months ago
|
Description
•