507578 - disable DNS prefetching when PAC or WPAD is used

Status: RESOLVED FIXED

(Core :: Networking: DNS, defect)

Description

[harald hermann]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12
Build Identifier: Mozilla/5.0(Windows; U; Windows NT 5.1; de; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1

Problem with prefetching of DNS and automatic configuration file in Firefox 3.5.1.
Eyerthing works fine with our autoconfiguration except when it comes to prefetching. 
Firefox 3.5.1 doesn't use the DNS Server as given in the autoproxy.file
but the DNs given by windows system and so our internal Nameservers are swapt with senseless dns requests.




Reproducible: Always

Comment 1

[Christian :Biesinger (don't email me, ping me on IRC)]

fwiw, the autoproxy configuration file doesn't specify any DNS servers.

we should probably disable prefetching not only when a manual proxy is configured (bug 488162) but also when using WPAD or an explicit PAC file.

Comment 2

[Patrick McManus [:mcmanus]]

I'm not actively working on ff at the moment, so assigning back to default

Comment 3

[Jo Hermans]

The fix needs to be done in nsDNSService::Init (nsDNSService2.cpp line 394) :

// Disable prefetching either by explicit preference or if a proxy is configured (manual, PAC, WPAD)
mDisablePrefetch = disablePrefetch
   || (proxyType == nsProtocolProxyService::eProxyConfig_Manual)
   || (proxyType == nsProtocolProxyService::eProxyConfig_PAC)
   || (proxyType == nsProtocolProxyService::eProxyConfig_WPAD);

On the other hand, some PAC files (and thus WPAD too) use DNS information to determine which proxy server to use. In that case, it's actually an advantage to have the DNS prefetch.

Counter-argument: such PAC files are slow anyway, and the advice is normally NOT to use DNS information in the PAC files, so that a request can be routed by the hostname alone.

Comment 4

[Timo Ruiter]

That may be good advice, but you cannot rely on people to heed it.
When you say that PAC/WPAD files need the DNS (prefetch) are you contradicting yourself?

Comment 5

[Jo Hermans]

The problem is that every PAC files is different - some might benefit from DNS prefetch (if they use DNS lookups themselves), some don't.

Comment 6

[Marek Gleska]

DNS prefetching should be disabled when PAC/WPAD file is active.
Usually rules in PAC file return DIRECT only for limited set of local servers, when local DNS server is close and response time is short.
For other servers (in URL) PAC return PROXY, so only proxy server should make a DNS query.

Comment 7

[Lee Carré]

RFE:
Unless “network.dns.disablePrefetch(FromHTTPS);false” would already cover this; ideally, there should be a way to enable DNS pre-fetching for networks in which there are no undesired consequences from pre-fetching while using a proxy via WPAD, and DNS pre-fetching is an enhancement.

Comment 8

[biffidus.rex]

It is worth noting here that with NetBIOS over IP enabled, Windows may broadcast the prefetch requests to your entire subnetwork. Please disable prefetching when using a proxy.

This behaviour was noticed on Firefox 14.

Comment 9

[Jo Hermans]

NetBIOS has nothing to do with a proxy.

Comment 10

[biffidus.rex]

Correct. But with NetBIOS enabled, all DNS lookups are broadcast over the local network. This includes the browser's prefetch lookups. When using a proxy, the DNS lookups are handled by the proxy itself so there is no reason for the browser to be performing dns prefetching.

In a corporate environment, with NetBIOS installed and a proxy in use, dns prefetching results in unnecessary requests being spammed all over the network.

Comment 11

[Jo Hermans]

(In reply to biffidus.rex from comment #10)
> When using a
> proxy, the DNS lookups are handled by the proxy itself so there is no reason
> for the browser to be performing dns prefetching.

Note that the PAC-file might send some request directly, and some over a proxy. Prefetches might still be useful in the former, but not in the latter.

Also note that NetBIOS is only used for simple hostnames without a domainname, and without a dot in their name, which aren't often run through a proxyserver. And most websites use FQDN in the URLs anyway, so they don't generate request over NetBIOS.

> In a corporate environment, with NetBIOS installed and a proxy in use, dns
> prefetching results in unnecessary requests being spammed all over the
> network.

Is NetBIOS still widely used ? All networks that I know have it disabled, or at least limited to small working groups (not company wide).

Comment 12

[Patrick McManus [:mcmanus]]

netbios is the first argument I've heard that makes me interested in this - the  broadcast nature of it is pretty horrendous and I can see how dns prefetch would impact that.

I propose to address the issue by disabling dns prefetch for the remainder of the dns service uptime whenever we do a proxyservice resolution that actually decides to use a proxy (either http or socks).

That means pac configurations and use-system-settings configurations that actually end up going direct all the time will continue to use prefetch. proxied and mixed environments will stop using it extremely early on.

You can make a couple minor criticisms of this, but I think they are of the perfect is the enemy of the good school and don't impact correctness in any way - so I don't feel addressing them has a good ROI at this time:
  * There is no automated mechanism for turning prefetch back on other than restarting..
  * at least 1 prefetch is likely to happen before the first proxy resolution

Comment 14

[Patrick McManus [:mcmanus]]

Attachment: patch 0

Jason, this is dep'd on 769764 simply because the patch in that bug gets rid of the nsProtoocolProxyService::Resolve() path (making everything use asyncResolve) - so its one less path for this code to insert itself into. You certainly don't need to learn 769764 to look at this.

we've got a try run over here
https://tbpl.mozilla.org/?tree=Try&rev=029fbebd6c14

and I've hand tested via tcpdump observation that prefetch still works in normal config, with a system config that does not use a proxy, and with a PAC that returns DIRECT along of course with it being disabled if a proxy is used via any mechanism.

Comment 15

[Jason Duell]

Comment on attachment 660419 [details] [diff] [review]
patch 0

Review of attachment 660419 [details] [diff] [review]:
-----------------------------------------------------------------

::: netwerk/base/src/nsProtocolProxyService.cpp
@@ +1626,5 @@
> +    nsCOMPtr<nsPIDNSService> pdns = do_QueryInterface(dns);
> +    if (!pdns)
> +        return;
> +
> +    pdns->SetPrefetchEnabled(false);

stick comment here (or wherever you feel is best) noting that once we turn off prefetch we don't re-enable it for remainder of DNS service uptime (do we want a followup to re-enable if the user changes their proxy settings, or there's some other event that might make re-enabling sensible?)

::: netwerk/dns/nsPIDNSService.idl
@@ +29,5 @@
> +
> +    /**
> +     * Whether or not DNS prefetching (aka RESOLVE_SPECULATE) is enabled
> +     */
> +    attribute boolean prefetchEnabled;

I assume we change the uuid even of explicitly private IDLs when we add new members...

Comment 16

[Patrick McManus [:mcmanus]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/2a30593cca79

Comment 17

[Ed Morley [:emorley]]

Push backed out for m-oth permaorange in browser_463205.js:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e59b9b887c25

Comment 18

[Patrick McManus [:mcmanus]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/75a3c47a3fe7

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/75a3c47a3fe7

Comment 20

[Ed Morley [:emorley]]

I've had to back this and two other changesets in the same push out, for extremely frequent OS X 10.8 mochitest-other leaks of the form found in bug 773255:
https://tbpl.mozilla.org/php/getParsedLog.php?id=15272317&tree=Mozilla-Inbound

http://brasstacks.mozilla.com/orangefactor/?display=Bug&tree=trunk&startday=2012-09-01&endday=2012-09-17&bugid=773255

https://hg.mozilla.org/integration/mozilla-inbound/rev/161847c0ac46

Comment 21

[Ed Morley [:emorley]]

Merge of backout:
https://hg.mozilla.org/mozilla-central/rev/161847c0ac46

Comment 22

[Patrick McManus [:mcmanus]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a41f9fff4cd1

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/a41f9fff4cd1