Closed Bug 1571426 Opened 5 years ago Closed 1 year ago

Intermittent Wd | <random> - setup error: error: [Errno 111] Connection refused

Categories

(Remote Protocol :: Marionette, defect, P5)

Version 3
defect

Tracking

(firefox-esr115 wontfix, firefox118 wontfix, firefox119 wontfix, firefox120 fixed)

RESOLVED FIXED
120 Branch
Tracking Status
firefox-esr115 --- wontfix
firefox118 --- wontfix
firefox119 --- wontfix
firefox120 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: jgraham)

References

(Depends on 2 open bugs, Blocks 1 open bug)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [webdriver:m9][webdriver:external][wptsync upstream])

Attachments

(2 files, 1 obsolete file)

Filed by: opoprus [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=259904343&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/WNCJ-YWOTSCCbPb9a9Fqzw/runs/0/artifacts/public/logs/live_backing.log


[task 2019-08-05T11:17:41.229Z] 11:17:41 INFO - TEST-START | /webdriver/tests/perform_actions/key.py
[task 2019-08-05T11:17:41.236Z] 11:17:41 INFO - PID 19431 | /builds/worker/workspace/build/tests/bin/geckodriver: error: Address already in use (os error 98)
[task 2019-08-05T11:17:41.236Z] 11:17:41 INFO - PID 19431 | geckodriver 0.24.0 (d681969e4480 2019-08-05 07:02 +0000)
[task 2019-08-05T11:17:41.236Z] 11:17:41 INFO - PID 19431 | WebDriver implementation for Firefox
[task 2019-08-05T11:17:41.236Z] 11:17:41 INFO - PID 19431 |
[task 2019-08-05T11:17:41.239Z] 11:17:41 INFO - PID 19431 | USAGE:
[task 2019-08-05T11:17:41.239Z] 11:17:41 INFO - PID 19431 | geckodriver [FLAGS] [OPTIONS]
[task 2019-08-05T11:17:41.239Z] 11:17:41 INFO - PID 19431 |
[task 2019-08-05T11:17:41.240Z] 11:17:41 INFO - PID 19431 | FLAGS:
[task 2019-08-05T11:17:41.240Z] 11:17:41 INFO - PID 19431 | --connect-existing Connect to an existing Firefox instance
[task 2019-08-05T11:17:41.240Z] 11:17:41 INFO - PID 19431 | -h, --help Prints this message
[task 2019-08-05T11:17:41.241Z] 11:17:41 INFO - PID 19431 | --jsdebugger Attach browser toolbox debugger for Firefox
[task 2019-08-05T11:17:41.241Z] 11:17:41 INFO - PID 19431 | -v Log level verbosity (-v for debug and -vv for trace level)
[task 2019-08-05T11:17:41.241Z] 11:17:41 INFO - PID 19431 | -V, --version Prints version and copying information
[task 2019-08-05T11:17:41.249Z] 11:17:41 INFO - PID 19431 |
[task 2019-08-05T11:17:41.250Z] 11:17:41 INFO - PID 19431 | OPTIONS:
[task 2019-08-05T11:17:41.258Z] 11:17:41 INFO - PID 19431 | -b, --binary <BINARY> Path to the Firefox binary
[task 2019-08-05T11:17:41.258Z] 11:17:41 INFO - PID 19431 | --log <LEVEL> Set Gecko log level [possible values: fatal, error, warn, info, config, debug,
[task 2019-08-05T11:17:41.269Z] 11:17:41 INFO - PID 19431 | trace]
[task 2019-08-05T11:17:41.269Z] 11:17:41 INFO - PID 19431 | --marionette-host <HOST> Host to use to connect to Gecko [default: 127.0.0.1]
[task 2019-08-05T11:17:41.284Z] 11:17:41 INFO - PID 19431 | --marionette-port <PORT> Port to use to connect to Gecko [default: system-allocated port]
[task 2019-08-05T11:17:41.285Z] 11:17:41 INFO - PID 19431 | --host <HOST> Host IP to use for WebDriver server [default: 127.0.0.1]
[task 2019-08-05T11:17:41.288Z] 11:17:41 INFO - PID 19431 | -p, --port <PORT> Port to use for WebDriver server [default: 4444]
[task 2019-08-05T11:17:41.465Z] 11:17:41 INFO - STDOUT: ============================= test session starts ==============================
[task 2019-08-05T11:17:41.465Z] 11:17:41 INFO - STDOUT: platform linux2 -- Python 2.7.12, pytest-unknown, py-1.5.2, pluggy-0.5.3.dev -- /builds/worker/workspace/build/venv/bin/python
[task 2019-08-05T11:17:41.466Z] 11:17:41 INFO - STDOUT: rootdir: /builds/worker/workspace/build, inifile:
[task 2019-08-05T11:17:41.466Z] 11:17:41 INFO - STDOUT: collecting ...
[task 2019-08-05T11:17:41.486Z] 11:17:41 INFO - STDOUT: collected 4 items
[task 2019-08-05T11:17:41.486Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/perform_actions/key.py::test_null_response_value
[task 2019-08-05T11:17:41.609Z] 11:17:41 INFO - STDOUT: ERROR
[task 2019-08-05T11:17:41.609Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/perform_actions/key.py::test_no_browsing_context
[task 2019-08-05T11:17:41.653Z] 11:17:41 INFO - STDOUT: ERROR
[task 2019-08-05T11:17:41.653Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/perform_actions/key.py::test_element_not_focused
[task 2019-08-05T11:17:41.701Z] 11:17:41 INFO - STDOUT: ERROR
[task 2019-08-05T11:17:41.702Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/perform_actions/key.py::test_backspace_erases_keys
[task 2019-08-05T11:17:41.742Z] 11:17:41 INFO - STDOUT: ERROR
[task 2019-08-05T11:17:41.743Z] 11:17:41 INFO - STDOUT: ==================================== ERRORS ====================================
[task 2019-08-05T11:17:41.744Z] 11:17:41 INFO - STDOUT: __________________ ERROR at setup of test_null_response_value __________________
[task 2019-08-05T11:17:41.746Z] 11:17:41 INFO - STDOUT: capabilities = {}
[task 2019-08-05T11:17:41.750Z] 11:17:41 INFO - STDOUT: configuration = {'capabilities': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'pref...rm.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}, 'host': '127.0.0.1', 'port': 56320}
[task 2019-08-05T11:17:41.751Z] 11:17:41 INFO - STDOUT: request = <SubRequest 'session' for <Function 'test_null_response_value'>>
[task 2019-08-05T11:17:41.751Z] 11:17:41 INFO - STDOUT: @pytest.fixture(scope="function")
[task 2019-08-05T11:17:41.762Z] 11:17:41 INFO - STDOUT: def session(capabilities, configuration, request):
[task 2019-08-05T11:17:41.763Z] 11:17:41 INFO - STDOUT: """Create and start a session for a test that does not itself test session creation.
[task 2019-08-05T11:17:41.764Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.765Z] 11:17:41 INFO - STDOUT: By default the session will stay open after each test, but we always try to start a
[task 2019-08-05T11:17:41.765Z] 11:17:41 INFO - STDOUT: new one and assume that if that fails there is already a valid session. This makes it
[task 2019-08-05T11:17:41.766Z] 11:17:41 INFO - STDOUT: possible to recover from some errors that might leave the session in a bad state, but
[task 2019-08-05T11:17:41.767Z] 11:17:41 INFO - STDOUT: does not demand that we start a new session per test."""
[task 2019-08-05T11:17:41.768Z] 11:17:41 INFO - STDOUT: global _current_session
[task 2019-08-05T11:17:41.769Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.770Z] 11:17:41 INFO - STDOUT: # Update configuration capabilities with custom ones from the
[task 2019-08-05T11:17:41.770Z] 11:17:41 INFO - STDOUT: # capabilities fixture, which can be set by tests
[task 2019-08-05T11:17:41.771Z] 11:17:41 INFO - STDOUT: caps = copy.deepcopy(configuration["capabilities"])
[task 2019-08-05T11:17:41.772Z] 11:17:41 INFO - STDOUT: caps.update(capabilities)
[task 2019-08-05T11:17:41.772Z] 11:17:41 INFO - STDOUT: caps = {"alwaysMatch": caps}
[task 2019-08-05T11:17:41.773Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.773Z] 11:17:41 INFO - STDOUT: # If there is a session with different capabilities active, end it now
[task 2019-08-05T11:17:41.778Z] 11:17:41 INFO - STDOUT: if _current_session is not None and (
[task 2019-08-05T11:17:41.779Z] 11:17:41 INFO - STDOUT: caps != _current_session.requested_capabilities):
[task 2019-08-05T11:17:41.779Z] 11:17:41 INFO - STDOUT: _current_session.end()
[task 2019-08-05T11:17:41.780Z] 11:17:41 INFO - STDOUT: _current_session = None
[task 2019-08-05T11:17:41.781Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.782Z] 11:17:41 INFO - STDOUT: if _current_session is None:
[task 2019-08-05T11:17:41.782Z] 11:17:41 INFO - STDOUT: _current_session = webdriver.Session(
[task 2019-08-05T11:17:41.783Z] 11:17:41 INFO - STDOUT: configuration["host"],
[task 2019-08-05T11:17:41.784Z] 11:17:41 INFO - STDOUT: configuration["port"],
[task 2019-08-05T11:17:41.785Z] 11:17:41 INFO - STDOUT: capabilities=caps)
[task 2019-08-05T11:17:41.785Z] 11:17:41 INFO - STDOUT: try:
[task 2019-08-05T11:17:41.786Z] 11:17:41 INFO - STDOUT: > _current_session.start()
[task 2019-08-05T11:17:41.787Z] 11:17:41 INFO - STDOUT: capabilities = {}
[task 2019-08-05T11:17:41.787Z] 11:17:41 INFO - STDOUT: caps = {'alwaysMatch': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'prefs...w.xn--n8j6ds53lwwkrqhv28a.web-platform.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}}
[task 2019-08-05T11:17:41.788Z] 11:17:41 INFO - STDOUT: configuration = {'capabilities': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'pref...rm.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}, 'host': '127.0.0.1', 'port': 56320}
[task 2019-08-05T11:17:41.789Z] 11:17:41 INFO - STDOUT: request = <SubRequest 'session' for <Function 'test_null_response_value'>>
[task 2019-08-05T11:17:41.790Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/support/fixtures.py
[task 2019-08-05T11:17:41.790Z] 11:17:41 INFO - STDOUT: :150:
[task 2019-08-05T11:17:41.791Z] 11:17:41 INFO - STDOUT: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-08-05T11:17:41.792Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/tools/webdriver/webdriver/client.py
[task 2019-08-05T11:17:41.793Z] 11:17:41 INFO - STDOUT: :429: in start
[task 2019-08-05T11:17:41.793Z] 11:17:41 INFO - STDOUT: value = self.send_command("POST", "session", body=body)
[task 2019-08-05T11:17:41.794Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/tools/webdriver/webdriver/client.py
[task 2019-08-05T11:17:41.795Z] 11:17:41 INFO - STDOUT: :471: in send_command
[task 2019-08-05T11:17:41.796Z] 11:17:41 INFO - STDOUT: session=self)
[task 2019-08-05T11:17:41.796Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/tools/webdriver/webdriver/transport.py
[task 2019-08-05T11:17:41.797Z] 11:17:41 INFO - STDOUT: :175: in send
[task 2019-08-05T11:17:41.798Z] 11:17:41 INFO - STDOUT: response = self._request(method, uri, payload, headers)
[task 2019-08-05T11:17:41.799Z] 11:17:41 INFO - STDOUT: tests/web-platform/tests/tools/webdriver/webdriver/transport.py
[task 2019-08-05T11:17:41.799Z] 11:17:41 INFO - STDOUT: :190: in _request
[task 2019-08-05T11:17:41.800Z] 11:17:41 INFO - STDOUT: self.connection.request(method, url, payload, headers)
[task 2019-08-05T11:17:41.801Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.802Z] 11:17:41 INFO - STDOUT: :1057: in request
[task 2019-08-05T11:17:41.802Z] 11:17:41 INFO - STDOUT: self._send_request(method, url, body, headers)
[task 2019-08-05T11:17:41.803Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.804Z] 11:17:41 INFO - STDOUT: :1097: in _send_request
[task 2019-08-05T11:17:41.804Z] 11:17:41 INFO - STDOUT: self.endheaders(body)
[task 2019-08-05T11:17:41.805Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.806Z] 11:17:41 INFO - STDOUT: :1053: in endheaders
[task 2019-08-05T11:17:41.807Z] 11:17:41 INFO - STDOUT: self._send_output(message_body)
[task 2019-08-05T11:17:41.807Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.808Z] 11:17:41 INFO - STDOUT: :897: in _send_output
[task 2019-08-05T11:17:41.809Z] 11:17:41 INFO - STDOUT: self.send(msg)
[task 2019-08-05T11:17:41.809Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.810Z] 11:17:41 INFO - STDOUT: :859: in send
[task 2019-08-05T11:17:41.810Z] 11:17:41 INFO - STDOUT: self.connect()
[task 2019-08-05T11:17:41.812Z] 11:17:41 INFO - STDOUT: /usr/lib/python2.7/httplib.py
[task 2019-08-05T11:17:41.812Z] 11:17:41 INFO - STDOUT: :836: in connect
[task 2019-08-05T11:17:41.813Z] 11:17:41 INFO - STDOUT: self.timeout, self.source_address)
[task 2019-08-05T11:17:41.813Z] 11:17:41 INFO - STDOUT: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-08-05T11:17:41.814Z] 11:17:41 INFO - STDOUT: address = ('127.0.0.1', 56320), timeout = <object object at 0x7f77191710d0>
[task 2019-08-05T11:17:41.815Z] 11:17:41 INFO - STDOUT: source_address = None
[task 2019-08-05T11:17:41.816Z] 11:17:41 INFO - STDOUT: def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
[task 2019-08-05T11:17:41.817Z] 11:17:41 INFO - STDOUT: source_address=None):
[task 2019-08-05T11:17:41.817Z] 11:17:41 INFO - STDOUT: """Connect to address and return the socket object.
[task 2019-08-05T11:17:41.819Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.819Z] 11:17:41 INFO - STDOUT: Convenience function. Connect to address (a 2-tuple (host, [task 2019-08-05T11:17:41.820Z] 11:17:41 INFO - STDOUT: port)) and return the socket object. Passing the optional
[task 2019-08-05T11:17:41.821Z] 11:17:41 INFO - STDOUT: timeout parameter will set the timeout on the socket instance
[task 2019-08-05T11:17:41.821Z] 11:17:41 INFO - STDOUT: before attempting to connect. If no timeout is supplied, the
[task 2019-08-05T11:17:41.822Z] 11:17:41 INFO - STDOUT: global default timeout setting returned by :func:getdefaulttimeout
[task 2019-08-05T11:17:41.823Z] 11:17:41 INFO - STDOUT: is used. If source_address is set it must be a tuple of (host, port)
[task 2019-08-05T11:17:41.823Z] 11:17:41 INFO - STDOUT: for the socket to bind as a source address before making the connection.
[task 2019-08-05T11:17:41.824Z] 11:17:41 INFO - STDOUT: A host of '' or port 0 tells the OS to use the default.
[task 2019-08-05T11:17:41.824Z] 11:17:41 INFO - STDOUT: """
[task 2019-08-05T11:17:41.825Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.827Z] 11:17:41 INFO - STDOUT: host, port = address
[task 2019-08-05T11:17:41.827Z] 11:17:41 INFO - STDOUT: err = None
[task 2019-08-05T11:17:41.829Z] 11:17:41 INFO - STDOUT: for res in getaddrinfo(host, port, 0, SOCK_STREAM):
[task 2019-08-05T11:17:41.829Z] 11:17:41 INFO - STDOUT: af, socktype, proto, canonname, sa = res
[task 2019-08-05T11:17:41.830Z] 11:17:41 INFO - STDOUT: sock = None
[task 2019-08-05T11:17:41.830Z] 11:17:41 INFO - STDOUT: try:
[task 2019-08-05T11:17:41.832Z] 11:17:41 INFO - STDOUT: sock = socket(af, socktype, proto)
[task 2019-08-05T11:17:41.832Z] 11:17:41 INFO - STDOUT: if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
[task 2019-08-05T11:17:41.833Z] 11:17:41 INFO - STDOUT: sock.settimeout(timeout)
[task 2019-08-05T11:17:41.833Z] 11:17:41 INFO - STDOUT: if source_address:
[task 2019-08-05T11:17:41.835Z] 11:17:41 INFO - STDOUT: sock.bind(source_address)
[task 2019-08-05T11:17:41.836Z] 11:17:41 INFO - STDOUT: sock.connect(sa)
[task 2019-08-05T11:17:41.836Z] 11:17:41 INFO - STDOUT: return sock
[task 2019-08-05T11:17:41.837Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.837Z] 11:17:41 INFO - STDOUT: except error as _:
[task 2019-08-05T11:17:41.839Z] 11:17:41 INFO - STDOUT: err = _
[task 2019-08-05T11:17:41.840Z] 11:17:41 INFO - STDOUT: if sock is not None:
[task 2019-08-05T11:17:41.840Z] 11:17:41 INFO - STDOUT: sock.close()
[task 2019-08-05T11:17:41.841Z] 11:17:41 INFO - STDOUT:
[task 2019-08-05T11:17:41.842Z] 11:17:41 INFO - STDOUT: if err is not None:
[task 2019-08-05T11:17:41.843Z] 11:17:41 INFO - STDOUT: > raise err
[task 2019-08-05T11:17:41.843Z] 11:17:41 INFO - STDOUT: E error: [Errno 111] Connection refused
[task 2019-08-05T11:17:41.844Z] 11:17:41 INFO - STDOUT: _ = error(111, 'Connection refused')
[task 2019-08-05T11:17:41.848Z] 11:17:41 INFO - STDOUT: address = ('127.0.0.1', 56320)
[task 2019-08-05T11:17:41.849Z] 11:17:41 INFO - STDOUT: af = 2
[task 2019-08-05T11:17:41.850Z] 11:17:41 INFO - STDOUT: canonname = ''
[task 2019-08-05T11:17:41.851Z] 11:17:41 INFO - STDOUT: err = error(111, 'Connection refused')
[task 2019-08-05T11:17:41.855Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.855Z] 11:17:41 INFO - TEST-UNEXPECTED-ERROR | /webdriver/tests/perform_actions/key.py | test_null_response_value - setup error
[task 2019-08-05T11:17:41.855Z] 11:17:41 INFO - capabilities = {}
[task 2019-08-05T11:17:41.855Z] 11:17:41 INFO - configuration = {'capabilities': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'pref...rm.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}, 'host': '127.0.0.1', 'port': 56320}
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - request = <SubRequest 'session' for <Function 'test_null_response_value'>>
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - @pytest.fixture(scope="function")
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - def session(capabilities, configuration, request):
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - """Create and start a session for a test that does not itself test session creation.
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - By default the session will stay open after each test, but we always try to start a
[task 2019-08-05T11:17:41.856Z] 11:17:41 INFO - new one and assume that if that fails there is already a valid session. This makes it
[task 2019-08-05T11:17:41.857Z] 11:17:41 INFO - possible to recover from some errors that might leave the session in a bad state, but
[task 2019-08-05T11:17:41.857Z] 11:17:41 INFO - does not demand that we start a new session per test."""
[task 2019-08-05T11:17:41.857Z] 11:17:41 INFO - global _current_session
[task 2019-08-05T11:17:41.857Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.857Z] 11:17:41 INFO - # Update configuration capabilities with custom ones from the
[task 2019-08-05T11:17:41.858Z] 11:17:41 INFO - # capabilities fixture, which can be set by tests
[task 2019-08-05T11:17:41.858Z] 11:17:41 INFO - caps = copy.deepcopy(configuration["capabilities"])
[task 2019-08-05T11:17:41.858Z] 11:17:41 INFO - caps.update(capabilities)
[task 2019-08-05T11:17:41.859Z] 11:17:41 INFO - caps = {"alwaysMatch": caps}
[task 2019-08-05T11:17:41.859Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.859Z] 11:17:41 INFO - # If there is a session with different capabilities active, end it now
[task 2019-08-05T11:17:41.860Z] 11:17:41 INFO - if _current_session is not None and (
[task 2019-08-05T11:17:41.861Z] 11:17:41 INFO - caps != _current_session.requested_capabilities):
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO - _current_session.end()
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO - _current_session = None
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO - if _current_session is None:
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO - _current_session = webdriver.Session(
[task 2019-08-05T11:17:41.862Z] 11:17:41 INFO - configuration["host"],
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - configuration["port"],
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - capabilities=caps)
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - try:
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - > _current_session.start()
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - capabilities = {}
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - caps = {'alwaysMatch': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'prefs...w.xn--n8j6ds53lwwkrqhv28a.web-platform.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}}
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - configuration = {'capabilities': {'moz:firefoxOptions': {'binary': '/builds/worker/workspace/build/application/firefox/firefox', 'pref...rm.test,xn--lve-6lad.www2.not-web-platform.test,www2.www.not-web-platform.test'}}}, 'host': '127.0.0.1', 'port': 56320}
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - request = <SubRequest 'session' for <Function 'test_null_response_value'>>
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - tests/web-platform/tests/webdriver/tests/support/fixtures.py:150:
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - tests/web-platform/tests/tools/webdriver/webdriver/client.py:429: in start
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - value = self.send_command("POST", "session", body=body)
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - tests/web-platform/tests/tools/webdriver/webdriver/client.py:471: in send_command
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - session=self)
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - tests/web-platform/tests/tools/webdriver/webdriver/transport.py:175: in send
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - response = self._request(method, uri, payload, headers)
[task 2019-08-05T11:17:41.863Z] 11:17:41 INFO - tests/web-platform/tests/tools/webdriver/webdriver/transport.py:190: in _request
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - self.connection.request(method, url, payload, headers)
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:1057: in request
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - self._send_request(method, url, body, headers)
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:1097: in _send_request
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - self.endheaders(body)
[task 2019-08-05T11:17:41.864Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:1053: in endheaders
[task 2019-08-05T11:17:41.865Z] 11:17:41 INFO - self._send_output(message_body)
[task 2019-08-05T11:17:41.865Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:897: in _send_output
[task 2019-08-05T11:17:41.865Z] 11:17:41 INFO - self.send(msg)
[task 2019-08-05T11:17:41.865Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:859: in send
[task 2019-08-05T11:17:41.865Z] 11:17:41 INFO - self.connect()
[task 2019-08-05T11:17:41.866Z] 11:17:41 INFO - /usr/lib/python2.7/httplib.py:836: in connect
[task 2019-08-05T11:17:41.866Z] 11:17:41 INFO - self.timeout, self.source_address)
[task 2019-08-05T11:17:41.866Z] 11:17:41 INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-08-05T11:17:41.866Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - address = ('127.0.0.1', 56320), timeout = <object object at 0x7f77191710d0>
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - source_address = None
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - source_address=None):
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - """Connect to address and return the socket object.
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.867Z] 11:17:41 INFO - Convenience function. Connect to address (a 2-tuple (host, [task 2019-08-05T11:17:41.868Z] 11:17:41 INFO - port)) and return the socket object. Passing the optional
[task 2019-08-05T11:17:41.868Z] 11:17:41 INFO - timeout parameter will set the timeout on the socket instance
[task 2019-08-05T11:17:41.868Z] 11:17:41 INFO - before attempting to connect. If no timeout is supplied, the
[task 2019-08-05T11:17:41.869Z] 11:17:41 INFO - global default timeout setting returned by :func:getdefaulttimeout
[task 2019-08-05T11:17:41.869Z] 11:17:41 INFO - is used. If source_address is set it must be a tuple of (host, port)
[task 2019-08-05T11:17:41.869Z] 11:17:41 INFO - for the socket to bind as a source address before making the connection.
[task 2019-08-05T11:17:41.870Z] 11:17:41 INFO - A host of '' or port 0 tells the OS to use the default.
[task 2019-08-05T11:17:41.870Z] 11:17:41 INFO - """
[task 2019-08-05T11:17:41.870Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - host, port = address
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - err = None
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - for res in getaddrinfo(host, port, 0, SOCK_STREAM):
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - af, socktype, proto, canonname, sa = res
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - sock = None
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - try:
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - sock = socket(af, socktype, proto)
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
[task 2019-08-05T11:17:41.871Z] 11:17:41 INFO - sock.settimeout(timeout)
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - if source_address:
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - sock.bind(source_address)
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - sock.connect(sa)
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - return sock
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - except error as _:
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - err = _
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - if sock is not None:
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - sock.close()
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - if err is not None:
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - > raise err
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - E error: [Errno 111] Connection refused
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - _ = error(111, 'Connection refused')
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - address = ('127.0.0.1', 56320)
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - af = 2
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - canonname = ''
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - err = error(111, 'Connection refused')
[task 2019-08-05T11:17:41.872Z] 11:17:41 INFO - host = '127.0.0.1'
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - port = 56320
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - proto = 6
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - res = (2, 1, 6, '', ('127.0.0.1', 56320))
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - sa = ('127.0.0.1', 56320)
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - sock = <socket._socketobject object at 0x7f770fa96c20>
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - socktype = 1
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - source_address = None
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - timeout = <object object at 0x7f77191710d0>
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - /usr/lib/python2.7/socket.py:575: error
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO -
[task 2019-08-05T11:17:41.873Z] 11:17:41 INFO - TEST-UNEXPECTED-ERROR | /webdriver/tests/perform_actions/key.py | test_no_browsing_context - setup error

Summary: Intermittent /webdriver/tests/perform_actions/key.py | test_null_response_value - setup error → Intermittent /webdriver/tests/perform_actions/key.py | test_null_response_value - setup error [due to "CannotSendRequest"]
Whiteboard: [test isolation]
Summary: Intermittent /webdriver/tests/perform_actions/key.py | test_null_response_value - setup error [due to "CannotSendRequest"] → Intermittent /webdriver/tests/<random> | <random> - setup error [due to "CannotSendRequest"]

(In reply to Andreas Tolfsen 「:ato」 from bug 1574075 comment #1)

Perhaps we should consider binding to 0 to have the system
atomically allocate the port?

Are we doing that with wdspec tests and geckodriver? I thought always uses a different port, or is that only Selenium which is doing that?

Flags: needinfo?(ato)

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #3)

(In reply to Andreas Tolfsen 「:ato」 from bug 1574075 comment #1)

Perhaps we should consider binding to 0 to have the system
atomically allocate the port?

Are we doing that with wdspec tests and geckodriver? I thought
always uses a different port, or is that only Selenium which is
doing that?

To my recollection, Selenium clients randomise the port ahead of
starting the process which can intermittently cause race conditions.
As can be seen in this log we don’t do that with wdspec.

I’ve filed bug 1574097 about binding to port 0.

Depends on: 1574097
Flags: needinfo?(ato)
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INCOMPLETE
Whiteboard: [test isolation] → [test isolation-]
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Summary: Intermittent /webdriver/tests/<random> | <random> - setup error [due to "CannotSendRequest"] → Intermittent Wd | <random> - setup error [due to "CannotSendRequest"]
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → INCOMPLETE
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → INCOMPLETE
See Also: → 1643775
Summary: Intermittent Wd | <random> - setup error [due to "CannotSendRequest"] → Intermittent Wd | <random> - setup error: error: [Errno 111] Connection refused
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

(In reply to Treeherder Bug Filer from comment #18)

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=369997870&repo=mozilla-esr91

This is bug 1720748.

Status: REOPENED → RESOLVED
Closed: 5 years ago3 years ago
Resolution: --- → INCOMPLETE
Duplicate of this bug: 1800428
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Severity: normal → S4
Status: REOPENED → RESOLVED
Closed: 3 years ago2 years ago
Resolution: --- → INCOMPLETE
Moving bug to Remote Protocol::Marionette component per bug 1815831.
Component: geckodriver → Marionette
Product: Testing → Remote Protocol
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Duplicate of this bug: 1831117
Duplicate of this bug: 1827423
Duplicate of this bug: 1834929
Duplicate of this bug: 1836902
Duplicate of this bug: 1836984

The number of failures increased over the last nearly two months. They all have a pattern like:

[task 2023-06-21T03:18:04.412Z] 03:18:04     INFO - STDOUT: Running command: /builds/worker/fetches/geckodriver --port 60503 -vv --binary /builds/worker/workspace/build/application/firefox/firefox
[task 2023-06-21T03:18:04.413Z] 03:18:04     INFO - Test harness output was not a valid structured log message
[task 2023-06-21T03:18:04.413Z] 03:18:04     INFO - 1687317484413	geckodriver	INFO	Listening on 127.0.0.1:60503
[..]
task 2023-06-21T03:18:08.864Z] 03:18:08     INFO - 1687317488863	geckodriver::browser	DEBUG	Browser process stopped: exit status: 0
[task 2023-06-21T03:18:08.864Z] 03:18:08     INFO - 1687317488863	webdriver::server	DEBUG	<- 200 OK {"value":null}
[task 2023-06-21T03:18:08.868Z] 03:18:08     INFO - STDOUT: Running command: /builds/worker/fetches/geckodriver --port 42196 -vv --binary /builds/worker/workspace/build/application/firefox/firefox
[task 2023-06-21T03:18:08.869Z] 03:18:08     INFO - Test harness output was not a valid structured log message
[task 2023-06-21T03:18:08.869Z] 03:18:08     INFO - /builds/worker/fetches/geckodriver: error: Address in use (os error 98)

Firefox is normally quit but due to some reason we error out with Address in use even through geckodriver is run on a different port. And that port has not been used previously in this particular job. So there must be some other external factor.

The same can also be seen on Android and that's tracked on bug 1791216, and for the test_root.py test as tracked on bug 1785893.

James, given that the wptrunner tries to find such a free port before starting geckodriver I assume we somewhat have a problem that other processes running on this machine claim the same port? Do you have an idea what we could do to find out more why it's not working? Maybe we could extend the error message in geckodriver and include eg. at least the process id or even name which has been taken that port?

Flags: needinfo?(james)
See Also: → 1791216, 1785893

We could try running lsof -i :{port} if the job fails like this, although it seems that on Android things are more complicated.

But also, looking at some of the logs, it seems like probably we are reusing the port? https://treeherder.mozilla.org/logviewer?job_id=420098181&repo=mozilla-central&lineNumber=267431 for example the error is:

[task 2023-06-21T04:57:46.662Z] 04:57:46     INFO - TEST-UNEXPECTED-FAIL | /_mozilla/webdriver/classic/protocol/allow_origins.py | test_allow_hosts[allow_origins4-http://web-platform.test-500] - ConnectionRefusedError: [Errno 111] Connection refused
[task 2023-06-21T04:57:46.662Z] 04:57:46     INFO - configuration = {'browser': {'args': ['--profile', '/tmp/tmp5b5thnk2.mozrunner'], 'binary': '/builds/worker/workspace/build/applicatio...{'moz:firefoxOptions': {'args': ['--profile', '/tmp/tmp5b5thnk2.mozrunner']}}, 'host': '127.0.0.1', 'port': 59211, ...}

But searching for that port number in the logs produces hits both before and after the error. So I suspect what's actually going on here is a race condition with tests that stop and start geckodriver. We could consider having something in the shutdown code that tries to connect to the port in a loop and blocks until it's no longer possible.

Flags: needinfo?(james)

Hm, that's interesting. So we should not conflate the port that is passed via the capabilities and the actual port used in those tests. The port 59211 above is the one set by wptrunner and which is used for the normal wdspec tests. But for the Mozilla specific classic tests we use the geckodriver fixture and that gets its own port by calling the get_free_port helper.

But there is one interesting case. The port 37438 where we fail to get geckodriver started is here:
https://treeherder.mozilla.org/logviewer?job_id=420098181&repo=mozilla-central&lineNumber=266928-266932

[task 2023-06-21T04:57:33.885Z] 04:57:33     INFO - STDOUT: Running command: /builds/worker/fetches/geckodriver --port 37438 -vv --binary /builds/worker/workspace/build/application/firefox/firefox --allow-origins http://web-platform.test:8000
[task 2023-06-21T04:57:33.890Z] 04:57:33     INFO - Test harness output was not a valid structured log message
[task 2023-06-21T04:57:33.890Z] 04:57:33     INFO - /builds/worker/fetches/geckodriver: error: Address in use (os error 98)
[task 2023-06-21T04:57:33.890Z] 04:57:33     INFO - WebDriver implementation for Firefox
[task 2023-06-21T04:57:33.890Z] 04:57:33     INFO - Usage:

That port is not used anywhere else by geckodriver! But there was a WebSocket port running for the RemoteAgent formerly:
https://treeherder.mozilla.org/logviewer?job_id=420098181&repo=mozilla-central&lineNumber=39322

[task 2023-06-21T04:29:35.153Z] 04:29:35     INFO - STDOUT: tests/web-platform/tests/webdriver/tests/bidi/input/perform_actions/key_events.py::test_key_special_key_sends_keydown[F10-expected15] 
[task 2023-06-21T04:29:35.156Z] 04:29:35     INFO - PID 9254 | 1687321775156	RemoteAgent	INFO	Perform WebSocket upgrade for incoming connection from 127.0.0.1:37438

This is happening roughly 30 minutes before. First I thought that we might leak some ports but after checking other failing tests this was just an exception. No other test here had an overlap.

I pushed a try build with some additional debug logging. Lets see if there might be some insight:
https://treeherder.mozilla.org/jobs?repo=try&revision=4f6179ce7b4f53e7b1ce140f9eb9d8421d3e3f6b

Duplicate of this bug: 1785890
Duplicate of this bug: 1785894

I finally reproduced the failure once in that try build:
https://treeherder.mozilla.org/logviewer?job_id=428833150&repo=try&lineNumber=252538-252542

[task 2023-09-12T12:08:44.730Z] 12:08:44     INFO - STDOUT: ** attempt 1 to get a free port
[task 2023-09-12T12:08:44.731Z] 12:08:44     INFO - STDOUT: ** found free port 51652
[task 2023-09-12T12:08:44.732Z] 12:08:44     INFO - STDOUT: Running command: /builds/worker/fetches/geckodriver --port 51652 -vv --binary /builds/worker/workspace/build/application/firefox/firefox --marionette-port 0
[task 2023-09-12T12:08:44.733Z] 12:08:44     INFO - Test harness output was not a valid structured log message
[task 2023-09-12T12:08:44.733Z] 12:08:44     INFO - /builds/worker/fetches/geckodriver: error: Address in use (os error 98)

It clearly shows that we found a free port but geckodriver isn't able to use it. Maybe some other process on that machine already claimed it? Maybe I can add some more details to geckodriver itself so we could see which other process is actually using this port.

Here is a new try build where geckodriver should print all the processes on the system in case it cannot connect to a given port:
https://treeherder.mozilla.org/jobs?repo=try&revision=553fd0c49bf7e9304a924efca5355dda74d6bb3c

Previously this bound to a free port, but we weren't allowing
connections because we checked the Host header against the specified
port (i.e. 0) rather than the bound port.

Assignee: nobody → james

Instead of passing in a port selected in the fixture, pass in 0 as the
port, allowing geckodriver to select the port, and then read the port
number from the utput in stdout.

This should prevent us trying to reuse an already bound port in
a race-free way.

Instead of using get_free_port(), pass in 0 as the port number and
allow geckodriver to select a free port. Then read the port from the
geckodriver output at startup. This should prevent rare race
conditions where we happen to select a port that later turns out to be
used.

Comment on attachment 9356360 [details]
Bug 1571426 - Make passing --port=0 select a free port,

Revision D189925 was moved to bug 1857193. Setting attachment 9356360 [details] to obsolete.

Attachment #9356360 - Attachment is obsolete: true
Duplicate of this bug: 1856153
Pushed by james@hoppipolla.co.uk: https://hg.mozilla.org/integration/autoland/rev/4e85807cccd7 Read geckodriver port number in fixture, r=webdriver-reviewers,jdescottes https://hg.mozilla.org/integration/autoland/rev/334a64fb61ff Read geckodriver port on startup for all wdspec tests, r=webdriver-reviewers,jdescottes
Failed to create upstream wpt PR due to merge conflicts. This requires fixup from a wpt sync admin.
Flags: needinfo?(james)
Flags: needinfo?(aborovova)
Whiteboard: [test isolation-] → [test isolation-], [wptsync upstream error]
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/42389 for changes under testing/web-platform/tests
Whiteboard: [test isolation-], [wptsync upstream error] → [test isolation-], [wptsync upstream]
Status: REOPENED → RESOLVED
Closed: 2 years ago1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 120 Branch

James the upstream PR cannot be merged because of the following failure:

>   ???
E     File "/Users/runner/work/1/s/tools/wptrunner/wptrunner/browsers/firefox.py", line 609
E       PORT_RE = re.compile(b".*Listening on [^ :]*:(\d+)")
E                           ^
E   SyntaxError: invalid escape sequence \d

Can you please get this fixed? Thanks.

As such reopening the bug for now.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Upstream PR merged by moz-wptsync-bot

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #77)

Can you please get this fixed? Thanks.

This got directly fixed in the upstream PR.

Status: REOPENED → RESOLVED
Closed: 1 year ago1 year ago
Flags: needinfo?(james)
Flags: needinfo?(aborovova)
Resolution: --- → FIXED

The patch landed in nightly and beta is affected.
:jgraham, is this bug important enough to require an uplift?

  • If yes, please nominate the patch for beta approval.
  • If no, please set status-firefox119 to wontfix.

For more information, please visit BugBot documentation.

Flags: needinfo?(james)

The amount of failures on esr115 is pretty low so lets wait with a potential uplift of this patch.

Flags: needinfo?(james)
Whiteboard: [test isolation-], [wptsync upstream] → [webdriver:m9][webdriver:external][wptsync upstream]
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e3fca1fe6b52 [wpt PR 42389] - [Gecko Bug 1571426] Read geckodriver port on startup for all wdspec tests, a=testonly
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: