1421824 - Add API to filter speculative connections

Status: NEW

(WebExtensions :: Request Handling, enhancement, P5)

Description

[S.P.]

User Agent: Mozilla/5.0 (Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0
Build ID: 20170914024831

Steps to reproduce:

1. Configure request-blocking addon (legacy or web extension) to block third-party requests
2. Visit a page that loads resources from those third-party hosts with Firefox (47 to 58 beta)
3. Observe network connections made by Firefox with network monitoring software




Actual results:

Initial request for a web page behaves as expected. Network requests for third-party resources that are blocked by addons are not initiated by Firefox.

But after the initial page load, reloading the page (most of the time) or clicking on a link within the page (every time) will trigger DNS requests and establish TCP connections to blocked third-party resources. Disabling DNS prefetching does not change this behavior.

The blocked resources do not affect the loaded page. Network monitor in the Firefox development toolbar does not show the connections made to the thirds-party hosts.

Tested with Firefox (several versions between 47 and 58 beta inclusive) legacy and web extensions on Windows 7-8.


Expected results:

Network requests that are blocked by addons should not go through. Network connections and DNS requests should not be initiated.

Comment 1

[:Gijs (he/him)]

Does turning off speculative connect fix this? (I *think* you can do this by setting network.http.speculative-parallel-limit to 0)

Comment 2

[Kris Maglione [:kmag]]

Can you provide a specific testcase add-on and specific steps to reproduce?

Comment 3

[S.P.]

Gijs:
I have network.http.speculative-parallel-limit set to 0 already.

Kris:
I have initially discovered this issue on Firefox 58 beta (and DevEd) using a custom web extension. I have repeated the same test on version 57 and 56 with the same results and with the same and other extensions.

The quickest and easiest test I've just completed is to use latest legacy version of Adblock Plus (2.8.2) with Firefox 56 or earlier. Legacy ABP allows you to quickly manipulate filters manually. I hope you are familiar with ABP as it was the most popular extension on addons site.

1. To block all third-party requests ABP you'll need to add only one blocking filter: "$third-party" (without quotes). You don't have to remove or modify anything else.
2. Browse to "https://www.mozilla.org". This page contains link to "https://www.googletagmanager.com".
3. After the page is loaded for the first time, the blocking filter does its job and there are no DNS or TCP requests to made for googletagmanager.
4. I reloaded the page with the same result - in some cases I had other web sites let the blocked connections through already.
5. Click on the "Firefox" link at the top left. You'll navigate to "https://www.mozilla.org/en-US/firefox/".
6. You should see DNS requests for "www.googletagmanager.com" and "www-googletagmanager.l.google.com" going through.
7. TLS (https) connection is made to the IP address associated with googletagmanager.
8. After short hansshake Firefox drops the connection. I have not confirmed what data is transferred (it's encrypted), but it might be that Firefox only goes through initial TLS hanshake.
9. Network monitor in developer toolbox does not display any connections to google. This should be the confirmation that addon does its part of the job.

I have repeated similar/exact test with several versions of Firefox just to try to see if I can roughly determine if there is earlier version of Firefox that doesn't have this issue. I've given up at 47.

Comment 4

[S.P.]

Attachment: Test for bug 1421824

Simple test for bug 1421824. Should block all requests to "secure.gravatar.com" on "https://bugzilla.mozilla.org/show_bug.cgi?id=1421824". And after initial page load, it does. But if you reload a page once or twice and check your network protocol analyzer logs, the request was made to DNS server and multiple connections were initiated to "secure.gravatar.com" IP address.

Network monitor in FF development toolbar does not show these requests, and avatars do not render on page.

Comment 5

[S.P.]

The attached extension will only install on "https://www.mozilla.org/en-US/firefox/developer/", as it's unsigned. I've tested it on DevEdition 58 and 56. They behave as described in comment 4.

Comment 6

[Kris Maglione [:kmag]]

Setting the network.predictor.enabled and network.dns.disablePrefetch preferences to false solves this for me.

I suppose it's worth considering whether we want to add an API that allows filtering speculative connections.

Comment 7

[Honza Bambas (:mayhemer)]

Predictor is Nick's area (CCing)

Comment 8

[S.P.]

Kris:

Thanks. Setting network.predictor.enabled to false did the trick.

What would be the best up-to-date place to find out about settings like these? I already had other settings applied that I expected to eliminate this kind of behavior.

Additional API to filter speculative connections is a great idea. I would suspect that many users are unaware of the privacy issues associated with performance features like the network predictor. As new add-ons are unable to modify about:config settings like these, add-on creators are unable to provide assurances to their users that the browser will not create unwanted requests if these settings are not considered.

It would also be beneficial if the network monitor in developer toolbar could display speculative connections. I would find it very useful in this case.

Comment 9

[Mike Conca [:mconca]]

I'm going to gently redirect this bug into a request for WebExtensions API that filter speculative connections. Additional API around security and privacy are always good ideas and align well with the Firefox and Mozilla mission.

Are these the (only) prefs involved in this?
  network.http.speculative-parallel-limit
  network.predictor.enabled
  network.dns.disablePrefetch

I'll refer this to the design decision needed meeting, but any suggestions on how to structure API(s) around those are welcome.

Comment 10

[Kris Maglione [:kmag]]

I don't think we want extensions to change those preferences. That would have a severe performance impact, and as a rule, add-ons hosted on AMO are not allowed to change network preferences anyway.

What we want is a way for extensions to block particular speculative connections if they are going to block those requests. Honestly, we should ideally just give them a way to add entries to the URL classifier, which would be much more efficient than callback-based filtering. But it might make sense to give them something more flexible, too.

Comment 11

[Andy McKay]

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/privacy/network was implemented in bug 1312802. It sets all these preferences I believe.

Comment 12

[Bob Silverberg [:bsilverberg]]

(In reply to Andy McKay [:andym] from comment #11)
> https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/privacy/
> network was implemented in bug 1312802. It sets all these preferences I
> believe.

That is correct. The setting is network.networkPredictionEnabled [1], which is also available in Chrome. To set it one would call browser.privacy.network.networkPredictionEnabled.set({value: false}).

Calling it sets all of the following preferences:

network.predictor.enabled
network.prefetch-next
network.http.speculative-parallel-limit
network.dns.disablePrefetch


[1] https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/privacy/network

Comment 13

[Bob Silverberg [:bsilverberg]]

Having said that, perhaps we want to also implement Kris' suggestion from comment 10, as a more performant solution than turning network prediction off globally.

Comment 14

[Mike Conca [:mconca]]

As a point of reference, 61 of the ~7,300 extensions on AMO use the privacy permission, so it's not used a lot. I do not know what percentage of ADI those 61 extensions represent. Making this more performant is a good idea, but probably not a high priority right now.

Comment 15

[Shane Caraveo (:mixedpuppy)]

This seems like something we should just approve per comment 9 and comment 10.

Comment 16

[Mike Conca [:mconca]]

(In reply to Shane Caraveo (:mixedpuppy) from comment #15)
> This seems like something we should just approve per comment 9 and comment
> 10.

Agreed. Approved.