484569 - Convert Cache to CouchDB

Status: RESOLVED INVALID

(Core :: Networking: Cache, defect)

Description

[Andrew Hagen]

Cache needs improvements in performance and reliability. Apache's CouchDB is a document database server that seems to fit our requirements. It is written in Erlang. 

http://couchdb.apache.org/docs/intro.html

CouchDB is released under Apache License 2.0.

Comment 1

[Sylvain Pasche]

uh? from the website:
Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API. [...] CouchDB is written in Erlang, but can be easily accessed from any environment that provides means to make HTTP requests.

Aren't you thinking of using that as a replacement for Necko HTTP cache?

Comment 2

[Andrew Hagen]

Yes.

Comment 3

[Sylvain Pasche]

Well, the technology and design of CouchDB doesn't seem to match the requirements of a fast HTTP cache for a browser.
First of all, you would need to embed an Erlang interpreter which may not be the best thing from a performance, code size or complexity point of view.
Then if you have to go through HTTP for storing/fetching cache entries that's going to hurt performance wise (I guess there's a way to bypass it).
The distributed feature doesn't seem to be suited for such a cache.

Then why would you need a schema-free document oriented db? From the cache point of view, the items all match a very specific schema: metadata (headers, ...) and payload.

Unless you have a detailed technical overview of the implementation that shows how it could improve the current situation I don't see how this is going to fly.

Comment 4

[Andrew Hagen]

CouchDB is a document database. In the cache there are documents. That could include HTML files, CSS files, Javascript files, JPEG files, GIF files, PNG files, etc....  I don't see why we would need a relational database for the cache, as in bug 425730. That's overhead we should avoid if we can. 

It's important to note that "document-based databases do not store data in tables with uniform sized fields for each record." 

http://en.wikipedia.org/wiki/Document-oriented_database

We could have fields of different length. For example, for each file in the cache (or each "document"), a JSON record, we could have the following fields: URL, time/date, cache-control, content-type, content-length, the file itself, or whatever we need. Thus, we wouldn't need a schema. 

That said, we probably don't need an Erlang interpreter running. We would want to turn that off. We don't need the distributed database function, either. 

http://blog.labnotes.org/2007/09/02/couchdb-thinking-beyond-the-rdbms/

http://blog.leetsoft.com/2007/9/2/futuretalk-couchdb

Comment 5

[u7736]

+1 to "uh?"

Comment 6

[Andrew Hagen]

Resolving invalid. I like the idea of revamping how we cache data. I believe a document-oriented database would be a good choice. It looks like CouchDB is not going to serve as that database. Thanks for the comments.