1963526 - For several Playwright tests the targeted test page never finishes loading unless you open the DevTools panel

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We're currently experiencing issues where several Playwright tests hang and eventually time out because the test page never completes loading. In the status bar, I can see the message Transferring data from localhost..., indicating that a connection is established, but the page never reaches either the interactive or load document states. It appears that something is preventing the page from fully loading, even though the connection remains open.

Earlier today, I encountered the same behavior in the test page/page-click.spec.ts :: should issue clicks in parallel in page and popup. Notably, this test does not involve any network interception, so it’s unclear why no data is being transferred.

To investigate further, I opened the DevTools network panel during the test using Cmd+Option+I. As soon as I did, the test immediately resumed and completed successfully. This suggests that opening DevTools somehow influences browser behavior in a way that unblocks the page load.

I was able to reduce the issue to the following minimal reproducible test case:

await page.goto(server.PREFIX + '/empty.html');
const [popup] = await Promise.all([
  page.waitForEvent('popup'),
  page.evaluate(() => window.open('/counter.html')),
]);
await popup.locator('button').click();

Interestingly, the issue does not occur if the click() call on the popup button is removed.

Julian, do you have an idea why opening DevTools would affect this behavior? It would also be good to know what might be causing the page to hang in the first place.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I was able to capture a Gecko profile for this case with a slightly modified test to make it clear which page load for localhost actually stalls:

https://share.firefox.dev/4cVt6uD

The process of the newly opened tab is foreground but there is no activity for about 9s until I manually opened devtools, which caused the test to proceed.

Comment 2

[Julian Descottes [:jdescottes]]

It's not only related to DevTools. Simply clicking on the background tab will resume the test.

Comment 3

[Julian Descottes [:jdescottes]]

Interestingly, the issue does not occur if the click() call on the popup button is removed.

Testing locally, it makes the reduced test case pass, but the page still doesn't load. You can wait for 5 seconds, you'll see the HTML is never loaded.

Comment 4

[Julian Descottes [:jdescottes]]

Regression window: https://hg-edge.mozilla.org/integration/autoland/pushloghtml?fromchange=8b055aa60807b71b880ea47770eb40e519609443&tochange=0b9e529a5541927220ce52a9577c4fb41782e0ae

• Emilio Cobos Álvarez — Bug 1958942 - Refactor how we schedule and suppress rendering phases. r=smaug
• Ryan VanderMeulen — Bug 1958041 - Pass the ZSTD_DISABLE_ASM flag more consistently to Windows builds. r=jesup

Comment 5

[Julian Descottes [:jdescottes]]

Hi Emilio it looks like Bug 1958942 regressed some Playwright tests, where tab calling window.open never seems to render the new tab. The network request corresponding to the HTML page is done properly, the response is received, but the tab keeps loading and is blank.

Opening devtools or focusing back and forth the new tab fixes it. Any idea?

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1958942

Comment 7

[Emilio Cobos Álvarez (:emilio)]

Can you give me a way of reproducing this? That'd probably be the faster way of fixing this.

It seems the pres shell is not initialized, so we don't render the document. DevTools seems to kick it off by flushing layout via Element.hasVisibleScrollbars.

So the question is why StartLayout() wasn't called by nsContentSink before.

Comment 8

[Julian Descottes [:jdescottes]]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #7)

Can you give me a way of reproducing this? That'd probably be the faster way of fixing this.

It seems the pres shell is not initialized, so we don't render the document. DevTools seems to kick it off by flushing layout via Element.hasVisibleScrollbars.

So the question is why StartLayout() wasn't called by nsContentSink before.

(I didn't manage to repro outside of playwright, not sure what I'm missing)

• git clone https://github.com/microsoft/playwright/
• cd playwright && npm i
• in a test file eg tests/page/page-click.spec.ts, add

it.only('should be able to use window open', async ({ page, server }) => {
  await page.goto(server.PREFIX + '/empty.html');
  await page.bringToFront();
  const [popup] = await Promise.all([
    page.waitForEvent('popup'),
    page.evaluate(() => window.open('/counter.html')),
  ]);
  await popup.locator('button').click();
});

• DEBUG=pw:browser BIDI_FFPATH='PATH_TO_YOUR_LOCAL_BUILD' npm run biditest -- --project='moz-firefox-page' --headed

Comment 9

[Emilio Cobos Álvarez (:emilio)]

Ok, let me try that.

Comment 10

[Emilio Cobos Álvarez (:emilio)]

Do I need a specific version of node or something?

$ DEBUG=pw:browser BIDI_FFPATH='~/src/moz/firefox/obj-debug/dist/bin/firefox' npm run biditest -- --project='moz-firefox-page' --headed

> playwright-internal@1.53.0-next biditest
> playwright test --config=tests/bidi/playwright.config.ts --project=moz-firefox-page --headed

node:internal/modules/cjs/loader:668
      throw e;
      ^

Error: Cannot find module '/home/emilio/src/moz/playwright/node_modules/@playwright/experimental-ct-core/lib/program.js'
    at createEsmNotFoundErr (node:internal/modules/cjs/loader:1441:15)
    at finalizeEsmResolution (node:internal/modules/cjs/loader:1430:15)
    at resolveExports (node:internal/modules/cjs/loader:661:14)
    at Function._findPath (node:internal/modules/cjs/loader:753:31)
    at Function._resolveFilename (node:internal/modules/cjs/loader:1391:27)
    at defaultResolveImpl (node:internal/modules/cjs/loader:1061:19)
    at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1066:22)
    at Function._load (node:internal/modules/cjs/loader:1215:37)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:235:24) {
  code: 'MODULE_NOT_FOUND',
  path: '/home/emilio/src/moz/playwright/node_modules/@playwright/experimental-ct-core'
}

Node.js v23.9.0

I also tried to follow the instructions in https://github.com/microsoft/playwright/blob/main/CONTRIBUTING.md, which seemed to get further along, but then npx install playwright or your command fails with:

npx playwright install
node:internal/modules/cjs/loader:1408
  throw err;
  ^

Error: Cannot find module './utilsBundleImpl'
Require stack:
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/utilsBundle.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/server/utils/debugLogger.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/server/utils/socksProxy.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/remote/playwrightConnection.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/remote/playwrightServer.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/androidServerImpl.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/inProcessFactory.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/inprocess.js
- /home/emilio/src/moz/playwright/packages/playwright-core/index.js
- /home/emilio/src/moz/playwright/packages/playwright-core/lib/cli/program.js
- /home/emilio/src/moz/playwright/packages/playwright/lib/program.js
- /home/emilio/src/moz/playwright/packages/playwright-ct-core/lib/program.js
- /home/emilio/src/moz/playwright/packages/playwright-ct-react/cli.js
    at Function._resolveFilename (node:internal/modules/cjs/loader:1405:15)
    at defaultResolveImpl (node:internal/modules/cjs/loader:1061:19)
    at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1066:22)
    at Function._load (node:internal/modules/cjs/loader:1215:37)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:235:24)
    at Module.require (node:internal/modules/cjs/loader:1491:12)
    at require (node:internal/modules/helpers:135:16)
    at Object.<anonymous> (/home/emilio/src/moz/playwright/packages/playwright-core/lib/utilsBundle.js:44:16)
    at Module._compile (node:internal/modules/cjs/loader:1734:14) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/utilsBundle.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/server/utils/debugLogger.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/server/utils/socksProxy.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/remote/playwrightConnection.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/remote/playwrightServer.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/androidServerImpl.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/inProcessFactory.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/inprocess.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/index.js',
    '/home/emilio/src/moz/playwright/packages/playwright-core/lib/cli/program.js',
    '/home/emilio/src/moz/playwright/packages/playwright/lib/program.js',
    '/home/emilio/src/moz/playwright/packages/playwright-ct-core/lib/program.js',
    '/home/emilio/src/moz/playwright/packages/playwright-ct-react/cli.js'
  ]
}

Node.js v23.9.0

Comment 11

[Julian Descottes [:jdescottes]]

Not that I know of, I'm on v22.3.0

Comment 12

[Emilio Cobos Álvarez (:emilio)]

With that version, same steps, I hit the error at the bottom :(

Comment 13

[Julian Descottes [:jdescottes]]

OK let me try to move it to a wdspec test

Comment 14

[Emilio Cobos Álvarez (:emilio)]

On mac it works (after running npm run watch). Let me try to debug it ther first :)

Comment 15

[Emilio Cobos Álvarez (:emilio)]

Ok, so I debugged this and this seems all kind of expected given the current code I believe? But I understand the regression. Here's the Pernosco session with notes.

The main issue is that WebDriver is blocking the parser (and thus the load event, but more importantly StartLayout() and everything else) from here.

That in turn waits for the viewport to be set via here and then awaits for the window dimensions to be set here, which in turn waits for an animation frame and resize events here.

So the main issue is that we're waiting for resize events and animation frames that will never arrive, because the parsing is blocked and rendering is suppressed.

This just so happened to work before my patch because this line didn't check for pres shell initialization. But really it was mostly by chance, we would've gotten similarly deadlocked if we didn't happen to create the PresShell early.

In terms of how to fix this... I guess one option could be to explicitly initialize rendering, but that's somewhat lame. It's pretty much what was happening tho, I guess, because of the .innerWidth / .innerHeight right below the animation frame callback... So maybe that's a good enough short term workaround?

We could also unblock parsing with a timeout, but that's also not great... We could also not block rendering when Document::BlockParsing() is being used... Which I guess it's fair given there's basically arbitrary script blocking our initialization... Thoughts Olli / Henrik / Julian?

Comment 16

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1963526 - Explicitly initialize layout from document-element-inserted if needed. r=whimboo,jdescottes

This is a somewhat low-risk-fix-to-previous behavior. But, I'd note, not
great.

Comment 17

[Julian Descottes [:jdescottes]]

Thanks for the detailed explanation!

We could also unblock parsing with a timeout, but that's also not great

We actually have another issue related to an AnimationFramePromise which never resolves. There is a patch to fix this at Bug 1947402 using a timeout and it also happens to "fix" this playwright issue, although as you said not great, and it stills slows down the test by 1500ms ...

The BiDi spec makes it clear that for new contexts we should update the viewport before running preload scripts, which should run before any content script. As a temporary workaround, force rendering if we attempt to update the viewport sounds fine, but if not blocking rendering during that phase is an option, it would probably be better.

Comment 18

[Emilio Cobos Álvarez (:emilio)]

I'd prefer avoiding web observable behavior changes based on a privileged api, but also I guess in this particular case it'd be very hard to observe.

Comment 19

[Julian Descottes [:jdescottes]]

Attachment: Bug 1963526 - [wdspec] Fix type hints for various BiDi wdspec helpers

Comment 20

[Julian Descottes [:jdescottes]]

Attachment: Bug 1963526 - [wdspec] Add test for setViewport for user context and window.open()

Depends on D247764

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b66f3182de7f
Explicitly initialize layout from document-element-inserted if needed. r=jdescottes,webdriver-reviewers
https://hg.mozilla.org/integration/autoland/rev/ef874abc66b9
[wdspec] Fix type hints for various BiDi wdspec helpers r=webdriver-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/f097102ffdf0
[wdspec] Add test for setViewport for user context and window.open() r=webdriver-reviewers,whimboo

Comment 22

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/52371 for changes under testing/web-platform/tests

Comment 23

[Cristina Horotan [:chorotan]]

Backed out for causing multiple failures

Backout link

Push with failures

Failure log Wd
Failure log lint

Comment 24

[Julian Descottes [:jdescottes]]

mistake in the test, will fix

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fcdb3ed4a111
Explicitly initialize layout from document-element-inserted if needed. r=jdescottes,webdriver-reviewers
https://hg.mozilla.org/integration/autoland/rev/d7071f1be055
[wdspec] Fix type hints for various BiDi wdspec helpers r=webdriver-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/c18424fd2f84
[wdspec] Add test for setViewport for user context and window.open() r=webdriver-reviewers,whimboo

Comment 26

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/b66f3182de7f
https://hg.mozilla.org/mozilla-central/rev/ef874abc66b9
https://hg.mozilla.org/mozilla-central/rev/f097102ffdf0
https://hg.mozilla.org/mozilla-central/rev/fcdb3ed4a111
https://hg.mozilla.org/mozilla-central/rev/d7071f1be055
https://hg.mozilla.org/mozilla-central/rev/c18424fd2f84

Comment 27

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 28

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 29

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 30

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:emilio, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
  • See https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift for documentation on how to request an uplift.
• If no, please set status-firefox139 to wontfix.

For more information, please visit BugBot documentation.

Comment 31

[Emilio Cobos Álvarez (:emilio)]

301 Julian :)

Comment 32

[Julian Descottes [:jdescottes]]

Attachment: Bug 1963526 - Explicitly initialize layout from document-element-inserted if needed. r=whimboo,jdescottes

This is a somewhat low-risk-fix-to-previous behavior. But, I'd note, not
great.

Original Revision: https://phabricator.services.mozilla.com/D247662

Comment 33

[Julian Descottes [:jdescottes]]

Attachment: Bug 1963526 - [wdspec] Fix type hints for various BiDi wdspec helpers

Depends on D247662

Original Revision: https://phabricator.services.mozilla.com/D247764

Comment 34

[Julian Descottes [:jdescottes]]

Attachment: Bug 1963526 - [wdspec] Add test for setViewport for user context and window.open()

Original Revision: https://phabricator.services.mozilla.com/D247765

Comment 35

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined: using window.open in automation will hang indefinitely if a viewport configuration is defined
• Code covered by automated testing: yes
• Fix verified in Nightly: yes
• Needs manual QE test: no
• Steps to reproduce for manual QE testing: N/A
• Risk associated with taking this patch: low
• Explanation of risk level: Simple JS fix to force rendering, webdriver bidi only
• String changes made/needed: none
• Is Android affected?: no

Comment 36

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/bc84093d898c
https://hg.mozilla.org/releases/mozilla-beta/rev/08ac6702e35d
https://hg.mozilla.org/releases/mozilla-beta/rev/b9cb05ae8ea2

Comment 37

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We fixed the regression in the same version of Firefox. So nothing we actually have to announce to our users.