Closed Bug 1721012 Opened 2 months ago Closed 2 months ago

Permanent /webdriver/tests/bidi/new_session/connect.py | test_websocket_url_connect[capabilities0] - KeyError: 'webSocketUrl' - when Gecko 92 merges to Beta on 2021-08-09

Categories

(Testing :: Marionette, defect, P2)

Default
defect
Points:
2

Tracking

(firefox-esr78 unaffected, firefox90 unaffected, firefox91 unaffected, firefox92+ verified)

VERIFIED FIXED
92 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox90 --- unaffected
firefox91 --- unaffected
firefox92 + verified

People

(Reporter: smolnar, Assigned: whimboo)

References

(Regression)

Details

(Keywords: regression, Whiteboard: [bidi-m1-mvp])

Attachments

(1 file)

[task 2021-07-17T12:09:37.733Z] 12:09:37     INFO - STDOUT: session = <Session 10d905d4-3d9c-49ee-910e-5ef8864100a4>
[task 2021-07-17T12:09:37.733Z] 12:09:37     INFO - STDOUT:     @pytest.mark.asyncio
[task 2021-07-17T12:09:37.734Z] 12:09:37     INFO - STDOUT:     @pytest.mark.capabilities({"webSocketUrl": True})
[task 2021-07-17T12:09:37.735Z] 12:09:37     INFO - STDOUT:     async def test_websocket_url_connect(session):
[task 2021-07-17T12:09:37.736Z] 12:09:37     INFO - STDOUT: >       websocket_url = session.capabilities["webSocketUrl"]
[task 2021-07-17T12:09:37.736Z] 12:09:37     INFO - STDOUT: E       KeyError: 'webSocketUrl'
[task 2021-07-17T12:09:37.737Z] 12:09:37     INFO - STDOUT: session    = <Session 10d905d4-3d9c-49ee-910e-5ef8864100a4>
[task 2021-07-17T12:09:37.738Z] 12:09:37     INFO - STDOUT: tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py
[task 2021-07-17T12:09:37.738Z] 12:09:37     INFO - STDOUT: :11: KeyError
[task 2021-07-17T12:09:37.739Z] 12:09:37     INFO - STDOUT: =========================== short test summary info ============================
[task 2021-07-17T12:09:37.740Z] 12:09:37     INFO - 
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/bidi/new_session/connect.py | test_websocket_url_connect[capabilities0] - KeyError: 'webSocketUrl'
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO - session = <Session 10d905d4-3d9c-49ee-910e-5ef8864100a4>
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO - 
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO -     @pytest.mark.asyncio
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO -     @pytest.mark.capabilities({"webSocketUrl": True})
[task 2021-07-17T12:09:37.741Z] 12:09:37     INFO -     async def test_websocket_url_connect(session):
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - >       websocket_url = session.capabilities["webSocketUrl"]
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - E       KeyError: 'webSocketUrl'
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - 
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - session    = <Session 10d905d4-3d9c-49ee-910e-5ef8864100a4>
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - 
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - tests/web-platform/tests/webdriver/tests/bidi/new_session/connect.py:11: KeyError
[task 2021-07-17T12:09:37.742Z] 12:09:37     INFO - TEST-INFO | expected ERROR
[task 2021-07-17T12:09:37.755Z] 12:09:37     INFO - ...
[task 2021-07-17T12:09:37.755Z] 12:09:37     INFO - TEST-OK | /webdriver/tests/bidi/new_session/connect.py | took 21763ms```
See Also: 1721011

The failure here happens when accessing session.capabilities["webSocketUrl"]. Given that on beta the Remote Agent is disabled, Marionette doesn't return this capability to the client. The current manifest of the file looks like:

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/webdriver/tests/bidi/new_session/connect.py.ini

Problem here is most likely that we expect a FAIL instead of an ERROR. I'll have a look soon.

Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Points: --- → 2
Priority: -- → P2
Whiteboard: [bidi-m1-mvp]

Indeed the ERROR state is only reached when something goes wrong during setup or teardown. In this case the above mentioned failure happens within the test, and as such it's expecting to fail.

Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2df3d7d91c43
[wdspec] Expect 'bidi/new_session/connect.py | test_websocket_url_connect" to fail when BiDi is disabled. r=webdriver-reviewers,jdescottes DONTBUILD
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 92 Branch
You need to log in before you can comment on or make changes to this bug.