1942792 - Trying to load the HTTP-only site "http://people.com.cn" with HTTP-First enabled never receives a response

Status: RESOLVED INVALID

(Core :: DOM: Security, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As it looks like there are cases when loading http://people.com.cn that a HTTP request is sent but no response is received when running in Firefox Nightly or DevEdition which has HTTP-First mode enabled.

I saw that when I tried to run the following Playwright tests with our WebDriver BiDi implementation:
https://github.com/microsoft/playwright/blob/main/tests/library/firefox/launcher.spec.ts

Both tests work when I run the test in a beta or release build of Firefox (where HTTP-First is turned off).

After a discussion with Simon we decided that I just file this bug for further investigation given that when running manual we can see different results - while the test in Playwright always produces it.

Here the steps to run the test:

1. Clone the Playwright repository (shallow clone is enough): https://github.com/microsoft/playwright
2. Run npm install and npm run build
3. Open the test file and change it( to it.only( (so that only this test is run)
4. Optionally add the remote.log.level preference to the list of preferences with a value of Trace set (enables trace logs of WebDriver BiDi)
5. Run the test: PWTEST_USE_BIDI_EXPECTATIONS=1 DEBUG="pw:browser" BIDIPATH=%path_to_firefox% npm run biditest -- --project='bidi-firefox-nightly*' --retries=0 --workers=1 --headed

Comment 1

[Malte Jürgens [:maltejur]]

Trying to reproduce this, I get the following error at step 5:

PWTEST_USE_BIDI_EXPECTATIONS=1 DEBUG="pw:browser" BIDIPATH=/tmp/tmpkh2mmub4/firefox/firefox npm run biditest -- --project='bidi-firefox-nightly*' --retries=0 --workers=1 --headed              13:39:23

> playwright-internal@1.51.0-next biditest
> playwright test --config=tests/bidi/playwright.config.ts --project=bidi-firefox-nightly* --retries=0 --workers=1 --headed

node:internal/modules/cjs/loader:647
      throw e;
      ^

Error: Cannot find module '/tmp/playwright/node_modules/@playwright/experimental-ct-core/lib/program.js'
    at createEsmNotFoundErr (node:internal/modules/cjs/loader:1277:15)
    at finalizeEsmResolution (node:internal/modules/cjs/loader:1266:15)
    at resolveExports (node:internal/modules/cjs/loader:640:14)
    at Function._findPath (node:internal/modules/cjs/loader:739:31)
    at Function._resolveFilename (node:internal/modules/cjs/loader:1227:27)
    at Function._load (node:internal/modules/cjs/loader:1066:27)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:220:24)
    at Module.require (node:internal/modules/cjs/loader:1327:12)
    at require (node:internal/modules/helpers:136:16) {
  code: 'MODULE_NOT_FOUND',
  path: '/tmp/playwright/node_modules/@playwright/experimental-ct-core'
}

Node.js v23.4.0

do you have any fixes for that?

I also just want to note that entering people.com.cn in a fresh Nightly 2025-01-18 profile starts loading successfully over HTTP after 3s, which would be expected with the HTTPS-First timer.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh wait. I missed to add a step! You will have to run npm run build.

Comment 3

[Malte Jürgens [:maltejur]]

I can reproduce it now, thanks. From what I can tell, this is totally expected behavior. The only thing HTTPS-First does is it upgrades the connection from http://example.com to https://example.com. As the proxy is only configured for HTTP connections, this means it won't be used anymore for the upgraded connection, so we also don't get any NS_ERROR_PROXY_CONNECTION_REFUSED. So the easiest fix I think would be to just also configure the same proxy for HTTPS connections. The following change makes the test work again for me:

--- a/tests/library/firefox/launcher.spec.ts
+++ b/tests/library/firefox/launcher.spec.ts
@@ -23,6 +23,8 @@
       'network.proxy.type': 1,
       'network.proxy.http': '127.0.0.1',
       'network.proxy.http_port': 3333,
+      'network.proxy.ssl': '127.0.0.1',
+      'network.proxy.ssl_port': 3333,
     }
   });
   const page = await browser.newPage();
@@ -38,6 +40,8 @@
       'network.proxy.type': 1,
       'network.proxy.http': '127.0.0.1',
       'network.proxy.http_port': 3333,
+      'network.proxy.ssl': '127.0.0.1',
+      'network.proxy.ssl_port': 3333,
     }
   });
   const error = await page.goto('http://example.com').catch(e => e);

And as for people.com.cn, that seems largely unrelated to the playwright failure, but we can still discuss it here. Do you have more precise steps to reproduce for that? As I said, the site works just fine for me.

Comment 4

[Simon Friedberger [:simonf]]

I saw some odd behavior for that as well, when investigating with Henrik earlier, but I cannot reproduce it.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, I missed one for step. Update the URL that the Playwright test uses to people.com.cn because we really need a HTTP-only site here which is not causing an upgrade to HTTPS.

Comment 6

[Malte Jürgens [:maltejur]]

Ah, I see. In that case, I think the following is happening:

• HTTPS-First is upgrading http://people.com.cn to https://people.com.cn
• The connection on https://people.com.cn is timing out, but not erroring out because this connection doesn't use the configured HTTP proxy
• After 3s, Firefox does an additional background request to http://people.com.cn to check if it can downgrade the request again
• This request fails with NS_ERROR_PROXY_CONNECTION_REFUSED because of the configured HTTP proxy, meaning no downgrade is performed

I would say everything behaves as expected here. For fixing the test, you can now either configure an HTTPS proxy as well as I suggested, or switch to a different method of detecting if prefs successfully apply, like you already do in this pr. Both sounds good to me.