Closed Bug 580104 Opened 14 years ago Closed 11 years ago

Keep hostname cache for more aggressive DNS prefetching

Categories

(Core :: Networking, defect)

Product:
Core
Component:
Networking
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 881804

People

(Reporter: jduell.mcbugs, Assigned: sworkman)

References

Details

(Keywords: perf, Whiteboard: [parity-chrome] [parity-ie])

Chromium keeps a cache that records what other hostnames a given hostname has previously neededto resolve (i.e. www.cnn.com winds up requiring ads.cnn.com, images.cnn.com, etc.). For future requests, they can then initiate resolving DNS for those other hosts before index.html from www.cnn.com has even arrived. They call this "Adaptive Pre-resolution" in this video: http://google-chrome-browser.com/why-google-chrome-fast-spotlight-dns-pre-resolution-0 They're claiming this is a 200-300ms win. We might want to have this cache also handle keeping lists of hosts to resolve at startup? (bug 580099).
Per the dupe (thanks, Patrick), IE9 also does this.
Whiteboard: [parity-chrome] [parity-ie]
Keywords: perf
Seems like levelDB (bug 663979) could make a good backend for this.
Depends on: leveldb
Blocks: 679883
Should this be a domain -> domains mapping or a URL -> domains mapping? Do we really need a separate database for this, or could we use an existing database?
> Should this be a domain -> domains mapping or a URL -> domains mapping? Good question. The latter is more accurate but takes up more space. It would be good to investigate the tradeoff with some data. > Do we really need a separate database for this, or could we use an existing database? Any idea which database would work? We're thinking of using leveldb for perf reasons, but there might be some other DB that this could logically fit into. But I'm not aware of one.
Assignee: nobody → sjhworkman
What's the status of this bug? Are we waiting for leveldb (which has resolved into a fun argument about main-thread I/O), or has this taken a backseat to other priorities? This, plus bug 679883, seems like relatively low-hanging fruit in our "be fast" battle. We know it works and is a win.
Justin, no change in status as yet, and afaik, waiting on leveldb. FYI, there are several other DNS enhancements being tracked in bug 580093.
LevelDB will not be implemented, as it currently stands. Will you continue to work on this? The backend to use now will probably be SQLite (or is there any other option?).
Yes, we still want this fix. I'm not sure what the storage design should be--there was a long thread on dev.platform about this not long ago: https://groups.google.com/forum/?fromgroups#!searchin/mozilla.dev.platform/Storage$20in$20Gecko/mozilla.dev.platform/vYbQqkqGzlo/YYIvd-rB9ToJ Seems like it's either going to be SQLite or maybe a plain file with JSON in it. May depend on whether we keep a DB that's small enough to keep in memory, or something we want to query on disk. There's a hit rate vs latency decision to be made there.
No longer depends on: leveldb
Could this be part of HTTP cache? I plan to cache what subresources a page loads to prewarm cache entries.
It does seem like this idea ought to be part of our other predictive work (bug 881804 or followups). Nick, does this need to stay its own bug (and maybe get tracked as part of some new meta-bug for prediction stuff), or get duped into 881804? re: storing in HTTP cache. The original idea here was to index just on hostname (i.e. DNS request for 'www.cnn.com' indexes into a DNS-only DB and tells us we should also prefetch 'img.cnn.com' and 'trackme.google.com', etc). If we implement subresource caching in the HTTP cache then that could work instead. There's tradeoffs--more accuracy vs more information stored. Also, DNS subhosts might be more long-lived than cache entries (the DNS subrequests might remain accurate after a particular page's content has expired). In the absence of data, I don't have an opinion which is better.
Flags: needinfo?(hurley)
Jason, my guess is we can dupe this into 881804, given that keeping a hostname list for both startup and pageload is part of that work (whether or not we actually fire off DNS requests will be dependent on the heuristics to determine likeliness, but that's an implementation detail). I won't do the duping myself, since I'm open to other solutions to this, but I really think 881804 is the way to go.
Flags: needinfo?(hurley)
Yeah - this should be dup'd. I just spoke to Nick about it, and he's fine with discussion continuing there.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.