1847732 - wptrunner doesn't kill browser instance for wdspec tests when a hang occurs and the test times out

Status: NEW

(Testing :: web-platform-tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This failure can be seen in the following GitHub CI check:
https://github.com/web-platform-tests/wpt/pull/41381/checks?check_run_id=15703721492

The patch that this check is running for requires a newer version of Firefox to not hang. But in this case the Nightly build wasn't available yet. As such the Python WebDriver BiDi client was failing in processing the response message and that causes a hang. With the test timed out we do not force kill the browser but continue with the next test. Because the browser hasn't been killed the socket ports as used are still blocked and cannot be used by the next instance of Firefox.

In case of a failure closing the browser via the driver the wptrunner should make sure to actually kill the spawned Firefox process. Note that as long as we are able to successfully create a new session we have the process id at least in the capabilities. Otherwise I'm not sure how we can force close it given that geckodriver manages the browser instance.

This can be reproduced with the following steps:

1. Get a Firefox Nightly build from before August 7th, 2023 like that one
2. Apply the patch from https://github.com/web-platform-tests/wpt/pull/41381
3. Run the stability job like the following:

python ./wpt run --channel=nightly --verify --verify-no-chaos-mode --verify-repeat-loop=0 --verify-repeat-restart=10 --github-checks-text-file=/home/test/artifacts/checkrun.md --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --no-headless --verify-log-full firefox webdriver/tests/bidi/browsing_context/dom_content_loaded/dom_content_loaded.py --webdriver-arg=-vv --binary=%path_to_binary%

James, any idea how to make this work?

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

When we have an active WebDriver (classic and BiDi) session as started by geckodriver it will know about the process id of Firefox. If deleting the session at the end of the test module doesn't work we could force-kill Firefox via its pid. If there is no active session we probably cannot do anything for now given that even using SIGTERM will not allow geckodriver to also kill the Firefox process (see bug 1430064).

Lets check again which Marionette port is actually in use and if we are going to re-use the same port in such a situation like the one from above. Maybe this bug could also fix bug 1571426. I'll have a look throughout this week.