663979 - try leveldb for a disk cache backend

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect)

Description

[(dormant account)]

We will likely end up using https://code.google.com/p/leveldb/ as a replacement for sqlite in a few places.

However, using leveldb to replace the network cache feels like a particularly good fit, ie the block cache files for small entries.
It's a key/value store db like our block cache. It features compression, batch io and background compaction.

Our current backend is really ill-performing, features bugs like bug 663200. Seems like it would be reasonable to give leveldb a try before we run off writing our own storage backend.

Comment 1

[Dominique De Vito]

See LevelDB Benchmarks (July 2011).

LevelDB (revision 39) is compared against SQLite3 (version 3.7.6.3) and Kyoto Cabinet's (version 1.2.67) TreeDB (a B+Tree based key-value store).

http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html

Interesting stuff.

Comment 2

[Jason Duell]

Note that we'll have to figure out the eviction story here: see bug 679852 comment 4