634653 - Lots of duplicate DNS prefetch messages

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: General, defect, P5)

Description

[Alon Zakai (:azakai)]

When loading websites, we have dozens of RecvHTMLDNSPrefetch calls that reach the parent, mostly of duplicate hostnames, see for example this data:

https://bug627635.bugzilla.mozilla.org/attachment.cgi?id=512675

Only 0.01 seconds is spent on it, so this isn't very troublesome, but it does seem wasteful. Here are the hostnames we receive when loading mozilla.org:

www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.com
www.mozilla.com
mozillamessaging.com
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
blog.mozilla.com
twitter.com
www.facebook.com
blog.mozilla.com
blog.mozilla.com
blog.mozilla.com
blog.mozilla.com
blog.mozilla.com
donate.mozilla.org
weblogs.mozillazine.org
mozillalabs.com
bugzillaupdate.wordpress.com
blog.mozilla.com
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.com
www.mozilla.com
viewvc.svn.mozilla.org
wiki.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.mozilla.org
www.firefox.com
www.getthunderbird.com
www.drumbeat.org
www.mozillalabs.com
www.mozilla.org
addons.mozilla.org
input.mozilla.com
www.mozilla.org
www.mozilla.org
developer.mozilla.org
developer.mozilla.org
donate.mozilla.org
wiki.mozilla.org
contribute.mozilla.org
quality.mozilla.org
blog.mozilla.com
www.mozilla.org

Comment 1

[Doug Turner (:dougt)]

can dns resolution happen in the child (on android?)?  It seems that we only need to prime the OS, not necko.

Comment 2

[Doug Turner (:dougt)]

jason, any ideas here?

Comment 3

[Jason Duell]

I don't know whether the child can do DNS or not. It's a library call, but I'm not sure what android permission set, if anything, that maps to:

  http://developer.android.com/reference/android/Manifest.permission.html

If we're lucky, it doesn't require any special permission, or a minor one like ACCESS_NETWORK_STATE.  If it requires INTERNET, it's not going to happen.

Seems like another way we could handle this is to keep a cache in the child of hostnames we've already prefetched, and drop the prefetch if we've already recently issued one for that hostname.  We could even use a simple hashtable (map host -> last time fetched), and not have to worry about collisions as long as they're rare (it's prefetching: the worst that happens is we wind up resolving at connect time).

Giving to Patrick, since he's currently our DNS guy.  Doug, correct me if I'm wrong, but this doesn't sound urgent (i.e for initial fennec ship).

(Note that bug 561694 is close to ready, and will reduce necko IPDL traffic a lot, too.)

Comment 4

[Patrick McManus [:mcmanus]]

(In reply to comment #3)

> Seems like another way we could handle this is to keep a cache in the child of
> hostnames we've already prefetched, and drop the prefetch if we've already
> recently issued one for that hostname.  We could even use a simple hashtable
> (map host -> last time fetched), and not have to worry about collisions as long
> as they're rare (it's prefetching: the worst that happens is we wind up
> resolving at connect time).
> 

you're describing the dns cache that is already in necko - it is supposed to turn that stream of redundant requests into the more or less minimal set of network queries and OS calls.

That isn't happening? Or it is happening and the problem is how these requests get shoveled across parent/child even though the redundant ones are fufilled within necko?

I don't favor a solution that bypasses the necko cache in order to just prime the OS cache directly - longer term it makes some sense to bring the whole resolver stack inside necko (dnssec among other things).

Comment 5

[Jason Duell]

> the problem is how these requests get shoveled across parent/child

bingo.

And I agree with you that we may long-term want our own resolver.  Hence my idea to do some sort of record-keeping in the child to prune IPDL traffic if a pre-fetch has already been made.   

I don't know the DNS code that well, but perhaps one way to do this might be to run a copy of the DNS code in the child, but replace the call to gethostbyname (or whatever) with an IPDL msg to the parent.   If the existing code already does something intelligent to minimize redundant gethostbyname requests, this would presumably lower the IPDL traffic similarly.   Just a thought.

Comment 6

[Josh Aas]

This seems like a duplicate of bug 511839, or at least related. This bug contains a lot of IPC discussion, whereas the other bug doesn't seem to involve IPC.

Comment 7

[BMO Automation]

Closing all opened bug in a graveyard component