1552715 - webRequest.onBeforeRequest redirect to data:text/html gets blocked

Status: NEW

(WebExtensions :: Request Handling, enhancement, P3)

Description

[Marcin Rataj]

Attachment: data-url-redirect-issue.zip

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

Used webRequest.onBeforeRequest to redirect HTTP request to data URL of an HTML page.

Actual results:

Request got blocked due to "security.data_uri.block_toplevel_data_uri_navigations" protection.

I attach simple browser extension that demonstrates the problem by registering redirect to data URL and then opening matching URL in a new tab to trigger a redirect in onBeforeRequest.

Error should be visible in Browser Console.
Additional screenshots can be found at: https://github.com/ipfs-shipyard/ipfs-companion/issues/723

Expected results:

I imagine "security.data_uri.block_toplevel_data_uri_navigations" protection should not block redirects done via WebExtension APIs, as extension code is already screened as a part of AMO review.

Comment 1

[Alex Cornestean]

The issue can be reproduced on the latest Release version (67.0 – 64 bit, in addition to the 66.05 - 64 bit Release version), Beta (68.0b3 – 64 bit) and Nightly (69.0a1 – 64 bit) by following the above STR under ‘Ubuntu16.04 LTS’, ‘Windows 10 Pro 64-bit’ and ‘macOS High Sierra 10.13.6’.

Comment 2

[Shane Caraveo (:mixedpuppy)]

My understanding from some cursory reading is that we purposely do not allow top level redirects like this, but specifically allowed subrequest redirects.

Bug 1434357 added a flag to allow data urls in a webRequest redirect, but it only tests a subrequest.
Bug 1380959 adds the pref to prevent top level redirects, probably for good reason, but lets get input from Christoph.

Comment 3

[Shane Caraveo (:mixedpuppy)]

Could we get a good use case?

Comment 4

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Shane Caraveo (:mixedpuppy) from comment #2)

My understanding from some cursory reading is that we purposely do not allow top level redirects like this, but specifically allowed subrequest redirects.

Bug 1434357 added a flag to allow data urls in a webRequest redirect, but it only tests a subrequest.
Bug 1380959 adds the pref to prevent top level redirects, probably for good reason, but lets get input from Christoph.

Sorry for the lag. As far as the problem goes, we explicitly added a carveout for extensions which allows insecure redirects to a data: URI within Bug 1434357. I am wondering what uBlock is doing differently so they are hitting that codepath [1] which greenlights the redirect. Probably we could/should update that codepath as well and set the allowInsecureRedirectToDataURI for that codepath as well.

[1] https://hg.mozilla.org/integration/mozilla-inbound/rev/416adbc7c8e7#l13.11

Comment 5

[Marcin Rataj]

(In reply to Shane Caraveo (:mixedpuppy) from comment #3)

Could we get a good use case?

• Generic
  • Improve UX by redirecting requests known to take long time to ephemeral "loading" screen that works the same across all vendors
  • Enable offline preview for HTML authoring in WebExtension Context that does not require 3rd party service
• Specific to ipfs-companion extension:
  • display a loading screen when content discovery takes too long (DHT warmup etc)
  • return HTML pages fetched from IPFS without the need for running HTTP Gateway as a separate localhost app

Comment 6

[Shane Caraveo (:mixedpuppy)]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #4)

(In reply to Shane Caraveo (:mixedpuppy) from comment #2)

My understanding from some cursory reading is that we purposely do not allow top level redirects like this, but specifically allowed subrequest redirects.

Bug 1434357 added a flag to allow data urls in a webRequest redirect, but it only tests a subrequest.
Bug 1380959 adds the pref to prevent top level redirects, probably for good reason, but lets get input from Christoph.

Sorry for the lag. As far as the problem goes, we explicitly added a carveout for extensions which allows insecure redirects to a data: URI within Bug 1434357. I am wondering what uBlock is doing differently so they are hitting that codepath [1] which greenlights the redirect. Probably we could/should update that codepath as well and set the allowInsecureRedirectToDataURI for that codepath as well.

Christoph, are you suggesting we check the allow flag here?

https://searchfox.org/mozilla-central/rev/f8b11433159cbc9cc80500b3e579d767473fa539/dom/security/nsContentSecurityManager.cpp#48

Comment 7

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Shane Caraveo (:mixedpuppy) from comment #6)

Christoph, are you suggesting we check the allow flag here?

https://searchfox.org/mozilla-central/rev/f8b11433159cbc9cc80500b3e579d767473fa539/dom/security/nsContentSecurityManager.cpp#48

Yes, basically do:
// Web Extensions are exempt from that restriction and are allowed to redirect
// a channel to a data: URI. When a web extension redirects a channel, we set
// a flag on the loadInfo which allows us to identify such redirects here.
if (loadInfo->GetAllowInsecureRedirectToDataURI()) {
return true;
}

which hopefully does the trick, but I would imagine the redirect goes through:
https://searchfox.org/mozilla-central/source/toolkit/components/extensions/webrequest/WebRequest.jsm#846

which would set the flag.

Comment 8

[[e-mail]]

@Marcin Rataj,

Have you entertained just opening an extension page instead of redirecting outright?

That is, block the network request and instead load your extension page using browser.tabs.update(...) with whatever query parameters needed to
configure the rendering of the page.

An example of such technique is used in uBlock Origin to warn a user before loading a navigated-to web page.

Comment 9

[Shane Caraveo (:mixedpuppy)]

Given that there are other mechanisms that can be used to achieve the same, a) I see no reason we shouldn't do this, and b) it's not a P1 but it's simple so we should get it done sooner than later. P2 it is.

Comment 10

[Marcin Rataj]

(In reply to [e-mail] from comment #8)

@Marcin Rataj,

Have you entertained just opening an extension page instead of redirecting outright? [..]

Good idea, should be good enough for our loading screen. Thanks!

(In reply to Shane Caraveo (:mixedpuppy) from comment #9)

Given that there are other mechanisms that can be used to achieve the same[..]

There are some interesting properties of data:text/html, that make them useful, eg. when we don't want to run third party JS in the context of extension's background page. dataURL will have window.location.origin equal "null", removing some risks.
But I agree, its a niche use, so low priority.

Comment 11

[Rob Wu [:robwu]]

Would be nice to fix, but not a high priority.

Comment 12

[Joaquín Serna]

Wasn't this already fixed on bug 1434357 (see https://searchfox.org/mozilla-central/source/dom/security/nsContentSecurityManager.cpp#147-152)
I still can't redirect to data URIs using webRequest.onBeforeRequest.

Comment 13

[Rob Wu [:robwu]]

We don't have test coverage for redirects to data:-URLs, except for scripts at https://searchfox.org/mozilla-central/rev/27932d4e6ebd2f4b8519865dad864c72176e4e3b/toolkit/components/extensions/test/xpcshell/test_ext_file_access.js#157-170

Consequently, there is unfortunately no guarantee that it works, or continues to work.
A new unit test is added to test redirects to data:-URLs for font files in bug 1645683, but that's still not everything.