Chromium keeps a list of hostnames that are likely to be needed at startup (open tabs, etc.), and then begins resolving them very early during program startup. They're claiming this can speed up page loads by 250-500 (depends on DNS resolution time, obviously): http://blog.chromium.org/2008/09/dns-prefetching-or-pre-resolving.html We should so something similar. We'll need to keep a list of hostnames that are in use. And we'll need to figure out how early we can fire off the resolve requests.
Shouldn't Platform be All All?
Created attachment 497511 [details] [diff] [review] Prefetch DNS for hosts needed during startup.1 # HG changeset patch # Parent 210237f7d62607a0122d5e5fe6dc4ac14981aecb bug 580099 Prefetch DNS for hosts needed during startup Upon shutting down record the hostnames for each window/tab as well as the hostnames necessary for any imgs, scripts, or links (but not anchors) on those pages. Upon startup, as soon as the profile dir is identified, use a separate thread to load all of those in and feed them to the DNS prefetch engine. Do not store hostnames if private browsing is enabled or if browser.startup.page is not 3 (i.e. we are not restoring pages at startup). ... the storage is done with js and xul on shutdown, the dns resolution is in C++ to make it as early as possible. I think this is the first bit of xul/js I've submitted in a patch, so it will require quality reviewing - but it works and is therefore a useful stake in the ground. Network speeds are going to have everything to do with how useful this is but in my experience several hundred names were all loaded into the firefox DNS cache by the time the restore-page logic needed them.. I only saw a few that were still in 'pending' instead of full hits -but even those are going to see a benefit of having started earlier.
(In reply to comment #3) > I would think that the ordering of the prefetching would be important. For > example, it would be better to resolve the DNS in this order: > we can do that.. I was considering adding a metric for base vs included, but active vs not is a good addition too.
Created attachment 497811 [details] [diff] [review] Prefetch DNS for hosts needed during startup.2 implement load order using scheme in comment 3 - slightly refined so that images are ranked behind css/js
AFAICT, this design is based on the following assumptions that aren't obviously valid: 1. We will not be able to get use any resources out of the cache without validating them, it will be too slow to get them out of the cache, and/or there are no negative consequences of doing unnecessary DNS lookups. 2. DNS will not be used as a tracking mechanism and/or we are not concerned about tracking/privacy as far as DNS lookups go. 3. The DNS prefetching for the items in inactive tabs does not negatively impact the (perceived) loading time of the active tab. (Of course, I am also assuming that it especially important to show the active tab's content ASAP.) 4. We will attempt to load all the items on all pages. (What about ad blockers and similar things?) 5. The extra work at startup does not have a significant negative impact on other parts of startup. (The team that did the research about browser performance perception said that users care more about the (perceived) startup time of the browser than they did about page loading performance--even though improvements to page loading performance saves them more time in aggregate.) If a page is cached and the cached representation is fresh then we might not even need to do a DNS lookup for that domain. The same applies to (fresh) cached resources loaded from that page. In this case the only way we are going to speed up the loading of the page is by getting the page out of the cache sooner. But, AFAICT, this design will, if anything, make us get the page out of the cache (slightly) later.
Added "See also" link to relevant Chromium issue. Basically, doing too many DNS queries at once has severely counter-productive performance implications in some configurations. (IIRC, I have one of those 2wire routers at home that I can test with; they are very common because AT&T deploys them to their home internet customers.) We should also look for at prefetching DNS for services we contact during startup that affect the startup page loading time (e.g. the safebrowsing/url-classifier server) if and only if data from those servers would actually be needed to show the active tab.
(In reply to comment #7) > Added "See also" link to relevant Chromium issue. Basically, doing too many DNS > queries at once has severely counter-productive performance implications in > some configurations. if we think that's a common enough scenario that's an argument for some kind of different throttle and scheduler for the dns system. It doesn't really have much to do with startup. I certainly acknowledge the the chromium problem report where firefox is mentioned once in a while, but just 60 days ago I asked around both mozilla and BIND circles if anyone had heard of problems with firefox prefetching in the field and I got 0 responses. It seems likely to be linked to AAAA lookups which can be likely to timeout - some more careful thinking is probably useful there. So we should be cautious about throwing the baby out with the bathwater when making changes.
If we end up implementing captive portal checking and/or a HTTP pipelining pretest by contacting a known server, then we probably need to do those checks early and thus prefetching DNS for those servers might also be a win.
Seems like we could leverage whatever solution we come up with for bug 580104 (which seems higher priority too) to solve this--just store an entry for a bogus 'startup' host at shutdown that will return all DNS hostnames needed at startup.
And it looks like that will be leveldb...
I don't have much interest in this patch anymore. Will pass it on to steve to see if it can help him - if not he can close it.
Duping this to bug 881804, as the work on predictive connections is being discussed there.