1621935 - webRequest listeners are not called for subresources loaded by file:// pages

Status: RESOLVED FIXED

(WebExtensions :: Request Handling, defect, P1)

Description

[Giorgio Maone [:ma1]]

Attachment: WebRequestFromFile.zip

This is quite urgent because it affects badly any content blocker webextension (most obviously NoScript / uBlock), which are unable to "see", let alone block, any 3rd party request from locally opened files, including scripts.
According to user reports, this is a 74 regression.

Steps to reproduce:

1. unzip the attached WebRequestFromFile.zip in a local directory and install the manifest.json it contains as a temporary addon and click the "Inspect"
2. Move the test.html file from the aforementioned directory to somewhere else outside (not a subdirectory, see "Fun fact" below), and open it in Firefox
3. Check the console in the Toolbox - Extension tab

Actual results:

No event is logged in the console.

Expected results:

The console should show two log lines, one for onBeforeRequest and the other for onHeadersReceived.

Fun fact that left me scraping my head for a while:

If I opened test.html from

file:///my/original/working/path/WebRequestFromFile/test.html

or from a subdirectory, the bug was NOT reproducible (both events get logged).

Wherever else I placed the file (either by moving, copying or symlinking it), other than there or in a subdirectory within, the bug was reliably reproducible. I've attempted any kind of variation (path length, path components, even just renaming the same directory with just one character change), but the mystery persisted.

Then the enlightenment: I tried to move the temporary add-on path, rather than the file, and could confirm that the bug is not reproducible if the HTML file is in the same path as the extension or in a subpath :D (bonus points to whomever can explain why).

Comment 1

[Rob Wu [:robwu]]

This is caused by a change from bug 1420296, specifically https://hg.mozilla.org/mozilla-central/rev/a32b39b12097

WebExtensionPolicy::CanAccessURI specifically excludes file:, to prevent extensions from silently loading local files.
The webRequest API only allows extensions to see requests if the extension can access the document that requested the resource. This is important, because the ability to modify script load requests effectively allows an extension to run code in whoever requested the site.

We do currently allow extensions to run content scripts on local files when they have the file:-permission, so there is no point in blocking subresource requests.

(In reply to Giorgio Maone [:mao] from comment #0)

I tried to move the temporary add-on path, rather than the file, and could confirm that the bug is not reproducible if the HTML file is in the same path as the extension or in a subpath :D (bonus points to whomever can explain why).

Unpacked extensions have a file://-baseURI (not sure if intentional, but it's what it is). Because of that, the implementation considers the request to be same-origin, and the CanAccessURI check is skipped: https://searchfox.org/mozilla-central/rev/7d0c94a0e9a9fe1f83553f49b10128567d21709d/toolkit/components/extensions/webrequest/ChannelWrapper.cpp#588-591

PS. Please do not set the Priority flag on bugs. In this case you cc'd me and that's why I saw the bug, but otherwise our normal bug triage process involves looking at bugs without priority. When a bug is reported with an assigned priority, odds are that nobody sees the bug.

Comment 2

[Tomislav Jovanovic :zombie]

(In reply to Rob Wu [:robwu] from comment #1)

PS. Please do not set the Priority flag on bugs. In this case you cc'd me and that's why I saw the bug, but otherwise our normal bug triage process involves looking at bugs without priority. When a bug is reported with an assigned priority, odds are that nobody sees the bug.

Agree with Rob, and in addition, if you have an urgent issue in the future, please feel free to contact or ni? one of us directly.

Comment 3

[Giorgio Maone [:ma1]]

(In reply to Tomislav Jovanovic :zombie from comment #2)

(In reply to Rob Wu [:robwu] from comment #1)

PS. Please do not set the Priority flag on bugs.

Sorry for that: I didn't actually set it, but rather overlooked resetting it, as I cloned another recent webRequest bug to be sure component and CCed people were correct (BTW I couldn't find the bugmail address of the uBlock guy, probably he'd like to be added).
It won't happen again.

Agree with Rob, and in addition, if you have an urgent issue in the future, please feel free to contact or ni? one of us directly.

Thank you.

Comment 4

[Julien Cristau [:jcristau]]

Rob, is this something you're working on for 75?

Comment 5

[Rob Wu [:robwu]]

Yes. It's a small fix and easy to fit in my other work.

Comment 6

[Rob Wu [:robwu]]

Attachment: Bug 1621935 - Allow webRequest to see subresource requests in local files

And remove the explicit "baseURL" origin check. This check was meant to
ensure that extensions can always intercept requests that it generated,
but changed in https://hg.mozilla.org/mozilla-central/rev/cd219dd096 by
accident to allowing access to the real jar:/file:-URL that backs
the moz-extension:-protocol handler.

That mistake did not break functionality, because the check was
redundant: the moz-extension:-origin is already explicitly added to
the internal set of host permissions of an extension. This scenario is
covered by the existing test_ext_webRequest_from_extension_page.js test.

Comment 7

[Rob Wu [:robwu]]

The test_ext_webRequest_from_extension_page.js test mentioned in comment 6 was added in bug 1624062.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fec7d30f8811
Allow webRequest to see subresource requests in local files r=mixedpuppy

Comment 9

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/fec7d30f8811

Comment 10

[Rob Wu [:robwu]]

Comment on attachment 9134879 [details]
Bug 1621935 - Allow webRequest to see subresource requests in local files

Beta/Release Uplift Approval Request

• User impact if declined: Extensions (e.g. NoScript and adblocker type extensions) with the proper permissions are not able to observe web requests from local files.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: bug 1624062 (mainly for extra test coverage; not required for the fix)
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Small change to revert the webRequest API's behavior to Firefox 73 and earlier (pre bug 1420296). Potentially relevant scenarios are fully covered by unit tests.
• String changes made/needed:

Comment 11

[Julien Cristau [:jcristau]]

Comment on attachment 9134879 [details]
Bug 1621935 - Allow webRequest to see subresource requests in local files

webext regression fix, approved for 75.0b10

Comment 12

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/releases/mozilla-beta/rev/13656732b5d4