220942 - DNS: pre-resolving hostnames found on a page

Status: RESOLVED DUPLICATE of bug 453403

(Core :: Networking, enhancement, P5)

Description

[Darin Fisher]

consider pre-resolving hostnames found on a page...

it may be of some value to pre-resolve hostnames that appear in HREFs on a page.
 we could leverage the existing prefetching service to defer these host lookups
until the browser is a near-idle state.  we'd probably also want to put a limit
on the number of resolved hostnames that we would pre-resolve in this manner. 
or something like that.  we could also meter the calls to gethostbyname if that
mattered.

BTW, this was suggested as a browser optimization from a participant at the 2003
IWCW.

Comment 1

[tenthumbs]

Bad idea. Consider a search engine results page. There might be 10, 20,
or more links which are mostly different. It's unlikely the user will
click a majority of them. Quite a few of them will be dead anyway. It's
a heavy load on the DNS server. If you limit it, how would you decide
which URL to resolve?

I guess I'm grumpy today. :-)

Comment 2

[benc]

I'm not as grouchy as tenthumbs, but I think a careful discussion of the context
of this feature is needed. 

Some kind of "if you are idle and in a bored state" detector would work well w/
this.

The seach page example is a good example of how this could be bad.

Comment 3

[Darin Fisher]

as for search results, we could limit to very few... maybe only the first 2-4
unique hostnames on the page (top-down).  or, we could do more, but at a slow
rate.  i think we can come up with something reasonable here.

benc: as for delaying until the browser is idle, that is what the prefetch
service already knows how to do (approximately, anyways).  so, that is why i
said that this should just utilize that mechanism.

Comment 4

[tenthumbs]

The biggest problem is defining "idle." Just because mozilla's idle
doesn't mean the system is, or the network, or even the user. Are you
going to examine the system's network stats to see if there's traffic?
If the machine is on a modem link you probably should because _any_
additional traffic is bad. Are you going to measure system load. You
really shouldn't do anything if it's too high. Are you going to test to
see if at least one mozilla window has the focus? It seems to me you
wouldn't want to do anything if the user isn't actively using mozilla. I
think you really need to do all of these.

Picking the first few links on the top of the page seems useless. For a
search engine response there's no reason to assume that any link is more
likely than any other.

What really bothers me though is that, on the one hand, mozilla says
it's trying to limit network traffic so it restricts the number of
connections per site but, on the other hand, the proposal here is to
generate network traffic purely on speculation. Mozilla really can't
have it both ways.

Comment 5

[Jo Hermans]

People who are worried about it, will have switched off prefetching anyway -
can't we use the same preference ? Or use one that toggles both prefetching and
pre-resolving ?

I really like the idea of pre-resolving hostnames, especially since the
DNS-rewrite has slowed down many lookups (all that IPv6 stuff - there are bugs
open on that). We just have to be careful on pages that contains many links to
all different hostnames (like search results, or links collections). Luckily,
most pages contains only a few links to 'new' hostnames. All those images that
are loaded from different ad-servers are a much bigger problem.

Note that if this gets implemented, we really need bug 208312 too (cache
negative lookups), otherwise we'll try to look up a bad hostname twice.

Comment 6

[George K.]

It looks like Google Chrome is doing this:

http://blog.chromium.org/2008/09/dns-prefetching-or-pre-resolving.html