1686841 - [wpt-sync] Sync PR 27195 - WebDriver Bidi tests with new Py3 WebSockets dependencies

Status: RESOLVED FIXED

(Testing :: web-platform-tests, task, P4)

Description

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Sync web-platform-tests PR 27195 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/27195
Details from upstream follow.

b'Shengfa Lin <[e-mail]>' wrote:

WebDriver Bidi tests with new Py3 WebSockets dependencies

Created a prototype involving WebDriver Bidi client according to [1].

The actual dependencies for the WebSockets lib[2] and pytest-asyncio[3] are not included in this PR. They are installed locally using pip install.

However, later versions of pytest[4] and pluggy[5] are needed for pytest-asyncio and they are included here.

The change includes:

A simple WebDriver Bidi capabilities test to demonstrate enabling WebSockets according to the protocol.
Upgraded pytest and pluggy third party dependencies(with a work around for an version error in pytest).
Added an async style test that enables WebSocket by directly specifying the corresponding capability.
Add BidiSession class in WebDriver client for use in fixture to conveniently create a Bidi Session and tests to demonstrate behavior between tests.
[1] #26015 (comment)
[2] https://github.com/aaugustin/websockets
[3] https://github.com/pytest-dev/pytest-asyncio
[4] https://github.com/pytest-dev/pytest
[5] https://github.com/pytest-dev/pluggy

Comment 1

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Pushed to try (stability) https://treeherder.mozilla.org/#/jobs?repo=try&revision=7f39352f9ff3e1e80bf68fea77bc84a3e8360ece

Comment 2

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Attachment: Notable wpt changes

# CI Results

Ran 10 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 171 tests

## Status Summary

### Firefox
OK     : 166
PASS   : 2165
FAIL   : 137
TIMEOUT: 167[Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt, Gecko-windows7-32-debug, Gecko-windows7-32-opt] 168[Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt]
ERROR  : 9

### Chrome
OK     : 166
PASS   : 2164
FAIL   : 142
ERROR  : 5

### Safari
OK     : 164
PASS   : 1757
FAIL   : 479
TIMEOUT: 2
ERROR  : 39

## Links
[Gecko CI (Treeherder)](https://treeherder.mozilla.org/#/jobs?repo=try&revision=7f39352f9ff3e1e80bf68fea77bc84a3e8360ece)
[GitHub PR Head](https://wpt.fyi/results/?sha=8ee4f2c93a1fc460884e2841572185e49d681975&label=pr_head)
[GitHub PR Base](https://wpt.fyi/results/?sha=8ee4f2c93a1fc460884e2841572185e49d681975&label=pr_base)

## Details

### Firefox-only Failures
/webdriver/tests/element_clear/clear.py
  test_contenteditable: FAIL
/webdriver/tests/element_send_keys/events.py
  test_file_upload: FAIL
/webdriver/tests/execute_async_script/promise.py
  test_await_promise_resolve: FAIL
/webdriver/tests/execute_script/promise.py
  test_await_promise_resolve: FAIL
/webdriver/tests/get_element_property/get.py
  test_primitives["foobar"-foobar]: FAIL
  test_primitives[42-42]: FAIL
  test_primitives[js_primitive2-py_primitive2]: FAIL
  test_primitives[js_primitive3-py_primitive3]: FAIL
  test_primitives_set_by_execute_script["foobar"-foobar]: FAIL
  test_primitives_set_by_execute_script[42-42]: FAIL
  test_primitives_set_by_execute_script[js_primitive2-py_primitive2]: FAIL
  test_primitives_set_by_execute_script[js_primitive3-py_primitive3]: FAIL
/webdriver/tests/get_window_rect/user_prompts.py
  test_accept_and_notify[capabilities0-alert-None]: FAIL
/webdriver/tests/maximize_window/maximize.py
  test_maximize_when_resized_to_max_size: FAIL
/webdriver/tests/maximize_window/user_prompts.py
  test_dismiss_and_notify[capabilities0-alert-None]: FAIL
/webdriver/tests/perform_actions/key_events.py
  test_modifier_key_sends_correct_events[\ue03d-META]: FAIL
  test_special_key_sends_keydown[ADD-expected0]: FAIL
  test_special_key_sends_keydown[META-expected30]: FAIL
  test_special_key_sends_keydown[NUMPAD4-expected37]: FAIL
  test_special_key_sends_keydown[NUMPAD5-expected38]: FAIL
  test_special_key_sends_keydown[PAGE_DOWN-expected43]: FAIL
  test_special_key_sends_keydown[PAGE_UP-expected44]: FAIL
  test_special_key_sends_keydown[SUBTRACT-expected66]: FAIL
/webdriver/tests/perform_actions/validity.py
  test_pause_positive_integer[none]: FAIL
  test_pause_positive_integer[key]: FAIL
  test_pause_positive_integer[pointer]: FAIL
/webdriver/tests/perform_actions/wheel.py
  test_no_top_browsing_context: FAIL
  test_no_browsing_context: FAIL
/webdriver/tests/set_window_rect/user_prompts.py
  test_dismiss_and_notify[capabilities0-alert-None]: FAIL
  test_default[alert-None]: FAIL

(See attachment for full changes)

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1a8d0061858b
[wpt PR 27195] - WebDriver Bidi tests with new Py3 WebSockets dependencies, a=testonly
https://hg.mozilla.org/integration/autoland/rev/1f206b66ad5c
[wpt PR 27195] - Update wpt metadata, a=testonly

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4f4bd5c12504
[wpt PR 27195] - WebDriver Bidi tests with new Py3 WebSockets dependencies, a=testonly
https://hg.mozilla.org/integration/autoland/rev/894b3b951ef7
[wpt PR 27195] - Update wpt metadata, a=testonly

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/718cdacc1eef
[wpt PR 27195] - WebDriver Bidi tests with new Py3 WebSockets dependencies, a=testonly
https://hg.mozilla.org/integration/autoland/rev/e84d7ae90173
[wpt PR 27195] - Update wpt metadata, a=testonly

Comment 6

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/718cdacc1eef
https://hg.mozilla.org/mozilla-central/rev/e84d7ae90173