1932311 - Investigate all Playwright requirements for WebDriver BiDi

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, task, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We need to thoroughly investigate Playwright's requirements for WebDriver BiDi. While we’re already aware of the critical missing features (see the attached document on bug 1917540), it’s also important to identify which other CDP APIs are in use. To achieve this, we should run the entire Playwright test suite with Chrome and enable CDP protocol logging. By analyzing the logged API usage, we can prioritize the most commonly used APIs, at least as seen in the unit and integration tests.

Additionally, as we transition to using JIRA for project management, we’ll need to organize the Playwright-related work into several buckets in our planning document for better tracking and prioritization.

All this work is targeted for the current milestone 14.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Checking latest test results for Chrome and Firefox show the following stats for the overall 4068 tests as run for BiDi:

Chrome:

 1479 skipped
 2485 passed (29.3m)

Firefox:

  1318 skipped
  2695 passed (32.2m)

This means we are even better than Chrome these days when the WebDriver BiDi protocol is used. Comparing it to end of September when Playwright added the CI jobs we nearly doubled the passing tests 1385 / 3877 passed (30min) .

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I was not able to finish the task yet. But here some updated numbers:

2718 / 3676 passed (51 min) - 227 skipped

I'm currently still investigating tests that timeout and cause a longer runtime of CI jobs. Right now those tests are marked as skipped and it would be good to get them identified and fixed if possible. I'm working with Yury to get the current status improved. See the attached spreadsheet.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

All the test failures for Playwright have been investigated and comments explain what's missing to get them to pass. Bugs have been filed accordingly for issues in Firefox, and issues for WebDriver BiDi.

As such there is no more work to do for this particular bug and we can close it finally.