1812360 - Investigate impact of 103 Early Hints (Link header) on network-related extension APIs

Status: NEW

(WebExtensions :: Request Handling, task, P3)

Description

[Rob Wu [:robwu]]

HTTP 103 Early Hints (MDN, Early Hints Chrome blog) is a feature to instruct the user agent to preconnect/preload ahead of receiving the full response. Firefox only implements this for top-level navigations (bug 1409290, source code of EarlyHintsService::EarlyHint) and is rolling this out on Nightly 111 (bug 1810543).

There are several network-related extension APIs, some of which used for privacy purposes. The webRequest API is commonly used to block requests from happening at all, at webRequest.onBeforeRequest. The proxy.onRequest API can be used to provide a custom proxy for network requests (but there are cases where there is no meaningful way to block it, see bug 1799411 and related bugs).

We should confirm whether the the webRequest and declarativeNetRequest behave as desired for requests triggered via 103 Early Hints. In particular, we should verify whether the request is associated with the expected principal and frame, and that the event details are meaningful. Even if the implementation works reasonable well, the task here is to write unit tests to ensure that this continues to be the case.

Comment 1

[Rob Wu [:robwu]]

Early hints is also going to be rolled out to Beta 111+ (bug 1813035).

Manuel, since you've enabled this on Nightly (bug 1810543) and Beta (bug 1813035), do you know whether it may potentially affect extensions? What are the differences with regular http(s) requests? E.g. are these requests correctly associated with the BrowsingContext and principals that are triggering the request?

Comment 2

[Manuel Bucher [:manuel]]

So, for the principal, I'm fairly sure that we are using the correct one. https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/netwerk/protocol/http/EarlyHintsService.cpp#57-75

Early Hint preloads behave the same as Link-Preloads from the Link-Header. The difference is that the main document can still have same origin redirects before the preload gets used.

It is not associated directly with any browsing context. It shares the cookieJarSettings of the main document load: https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/netwerk/protocol/http/EarlyHintsService.cpp#80,95

How can I test whether extensions are affected? (e.g. is creating a block rule for an image in uBlock and then making sure the image isn't loaded via Early Hints enough?). Is it easy/necessary to create an automated test case?

Comment 3

[Rob Wu [:robwu]]

(In reply to Manuel Bucher [:manuel] from comment #2)

So, for the principal, I'm fairly sure that we are using the correct one. https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/netwerk/protocol/http/EarlyHintsService.cpp#57-75

Early Hint preloads behave the same as Link-Preloads from the Link-Header. The difference is that the main document can still have same origin redirects before the preload gets used.

It is not associated directly with any browsing context. It shares the cookieJarSettings of the main document load: https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/netwerk/protocol/http/EarlyHintsService.cpp#80,95

I see, the current implementation only processes responses from the top-level document ("main document" in your comment/source), and not subframes.

How can I test whether extensions are affected? (e.g. is creating a block rule for an image in uBlock and then making sure the image isn't loaded via Early Hints enough?).

That would be a quick manual test indeed. Maybe even keep Wireshark running alongside to verify that there are no unwanted network requests.

A more detailed version (but then you may as well head to the unit test directly), is to manually inspect the reported webRequest event details, and compare it with regular requests.

You can do it manually by visiting about:debugging, clicking This Nightly, and then clicking on the "manifest URL" link from any of the listed extensions that use the webRequest API. uBlock Origin would be an example. I'm suggesting to click on the "manifest URL" link because that opens a new tab, and the listener will automatically be unregistered after closing the tab. An alternative is to click "Inspect" to inspect the existing background page, but then the listener will be around until the extension or browser is restarted.

The following snippet reports all initial request info for all http(s) URLs. If you'd like, you can make the "urls" pattern more specific if you know which requests you're interested in (or put an if-condition before the log statement).

browser.webRequest.onBeforeRequest.addListener(details => {
  console.log(details);
}, {
  urls: ["*://*/*"],
});

Is it easy/necessary to create an automated test case?

If we want the use case to continue working, yes.
Here is an example that shows how to register the listener, how to open a document and subscribe/assert events. https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/toolkit/components/extensions/test/xpcshell/test_ext_webRequest_incognito.js.

Here is how the test is run the supported configuration for desktop (mainly --tag=remote-webextensions, otherwise you'll run the test twice with different test configurations):

./mach test toolkit/components/extensions/test/xpcshell/test_ext_webRequest_incognito.js --verbose --log-mach-verbose --tag=remote-webextensions --sequential

If the feature may have browser-specific dependencies (browser UI/config) that are not seen in xpcshell environments (I doubt FYI), then you could also consider a mochitest instead, which runs in an actual browser. Here is an example of that: https://searchfox.org/mozilla-central/rev/861fb9abfcaff123aab45f6ac56a0106b116dc14/toolkit/components/extensions/test/mochitest/test_ext_embeddedimg_iframe_frameAncestors.html

Here is general dev documentation on authoring extension tests: https://wiki.mozilla.org/WebExtensions/Contribution_Onramp#Testing

Comment 4

[Manuel Bucher [:manuel]]

Attachment: Screenshot from 2023-02-01 16-48-37.png

By testing with uBlock Origin I've found that currently:

• No network requests are made for the blocked resource (image in this case). I've also looked at the Wireshark log to verify that.
• but an empty frame is displayed

Edit:
the custom rule looks like this:

! 2023-02-01 http://localhost:2001
||localhost:2001/Aerial_view_of_Somapura_Mahavihara.jpg.webp$image

Comment 5

[Manuel Bucher [:manuel]]

Unassigning myself from Early Hint bugs to reflect reality of me not working on them anymore. Feel free to reach out for Early Hints related questions. If urgent I can still take bugs.