94079 - PAC: too many DNS lookups

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Austin Donnelly]

Mozilla version 0.9.3 (Build ID: 2001080104)

If preferences Advanced/Proxies is set to "Automatic proxy configuration URL"
then when loading http://news.bbc.co.uk/ I see around 34 extraneous DNS lookups
happen. This happens within a 3 second period.

My guess is that mozilla doing a DNS lookup for each subcomponent of the page it
has just loaded, despite the fact mozilla should leave that to the proxy.

I'm getting my auto proxy config from
   http://www.cl.cam.ac.uk/proxy.config

This file reads:

function FindProxyForURL(url, host)
{
   if(  shExpMatch( url, "*cgi*" ) ||
        shExpMatch( url, "snews:*" ) ||
        shExpMatch( url, "https:*" ) ||
        dnsDomainIs( host, "pgp.net") ||
        dnsDomainIs( host, "cam") ||
        dnsDomainIs( host, "ac") ||
        dnsDomainIs( host, "ac.uk" ) ||
        dnsDomainIs( host, "ja.net" ) ||
        dnsDomainIs( host, "www.avantek.co.uk" ) ||
        dnsDomainIs( host, "www.elsevier.nl" ) ||
        dnsDomainIs( host, "localhost" ) ||
        isPlainHostName( host ) ||
        isInNet( host, "131.111.0.0", "255.255.0.0") ||
        isInNet( host, "128.232.0.0", "255.255.0.0") ||
        isInNet( host, "129.169.0.0", "255.255.0.0") ||
        isInNet( host, "192.168.100.1", "255.255.255.0") ||
        isInNet( host, "127.0.0.1",   "255.255.255.255") )
        return "DIRECT";
   else return  "PROXY wwwcache.cam.ac.uk:8080; " +
                "DIRECT";
}


If I change the proxy preferences setting to "Direct connecton to the Internet"
or
"Manual proxy configuration" (with just an HTTP proxy set to wwwcache.cam.ac.uk
port 8080) then I see correct DNS behaviour: a single lookup
request is sent (and presumably the result cached by mozilla).

Austin

Comment 1

[Bradley Baetz (:bbaetz)]

This will probably end up as a dupe of bug 93845.

Comment 2

[Ian Wehrman]

i've been seeing a very noticeable performance hit because of this. as of
2001090608, this is still a major problem.

Comment 3

[Bradley Baetz (:bbaetz)]

What sites are being looked up? isInNet will cause a lookup, but that should
only happen once per host now that the last host looked up by PAC is cached.

Can you attach a full packettrace, inclusing dns requests, loading
www.google.com from a new browser session (after clearing the cache), then
reloading it?

Comment 4

[Austin Donnelly]

Attachment: packet trace A

Comment 5

[Austin Donnelly]

Attachment: packet trace B

Comment 6

[Austin Donnelly]

Attachment: packet trace C

Comment 7

[Austin Donnelly]

As requested, I've attached 3 packets traces.

This is all still with build ID 200108104 Linux ix86.

I turned on PAC, cleared my disc and memory caches and quit the browser.

I ran
   tcpdump -s2048 host wwwcache.cam.ac.uk or port domain
I then started mozilla and clicked on a link to http://www.google.com/ 
from my homepage (which happens to be local). Packet trace A shows what
happened.  The page loaded and displayed correctly.
I then clicked on the refresh button.  Packet trace B shows this.  Again
the page loaded fine.

I then shift-clicked refresh, and captured Packet trace C.
The packet traces are cut and pasted from the tcpdump output - one tcpdump
was running all through this, and all packets are present either in
trace A, B, or C.  I stopped the tcpdump some 30 seconds after the last
shift-reload experiment.  I note that no FIN packets were exchanged between
my local machine hornet and the cache (wwwcache.cam.ac.uk).   resolv0 is my
local DNS server.
Hope this helps!

Austin

Comment 8

[Bradley Baetz (:bbaetz)]

Hmm. Is this still occuring with later nightlies? [e-mail] may be having 
a different problem.

Comment 9

[Austin Donnelly]

I've just downloaded a nightly build, ID 2001090906, and tried to repeat
this bug.

As far as I can see, it is fixed.  Loading http://www.google.com/ causes only
one DNS lookup of www.google.com.  Reloading or shift+reloading doesn't
cause any further DNS lookups.
I am assuming the single initial DNS lookup is part of trying to figure out
if the user typed a valid hostname before trying to add "www." to the start,
or ".com" to the end.  This may well be a separate bug, but as far as I am
concerned this particular bug has been fixed.

I've marked this bug resolved as FIXED.
Thanks guys!

Austin

Comment 10

[Bradley Baetz (:bbaetz)]

No, its used for the IsInNet call in the pac file - we need the IP to fix that.