Closed Bug 686314 Opened 8 years ago Closed 8 years ago

IndexedDB: Compress structured clones

Categories

(Core :: DOM: IndexedDB, defect)

defect
Not set

Tracking

()

RESOLVED DUPLICATE of bug 703660

People

(Reporter: janv, Unassigned)

Details

I tested the Snappy compression library to get some numbers for the "files stored in IndexedDB" work...

The API is actually very simple to use and I tried to compress a structured clone before it's written to the sqlite db. The library is optimized for speed, not the compression ratio.

I tested it with a JS object:
var data = {
  id: 1,
  code: "101",
  value: "1.23",
  white: 1,
  blue: 20,
  red: 3343,
  green: 4,
  black: 500000
};

uncompressed data:
sqlite> select quote(data) from object_data;
X'000000000800FFFF020000000400FFFF6900640000000000000000000000F03F040000000400FFFF63006F0064006500030000000400FFFF3100300031000000050000000400FFFF760061006C0075006500000000000000040000000400FFFF31002E0032003300050000000400FFFF77006800690074006500000000000000000000000000F03F040000000400FFFF62006C00750065000000000000003440030000000400FFFF720065006400000000000000001EAA40050000000400FFFF67007200650065006E000000000000000000000000001040050000000400FFFF62006C00610063006B000000000000000000000080841E41000000000000FFFF'

the uncompressed size is: 256 bytes
the compressed size is: 158 bytes (61.72%)

there are apparently zero sequences which compress quite well

I investigated jsclone.cpp a bit and it seems that the data in a pair (tag and data) is zero for nulls, undefined objects, date objects, numbers, etc.

There's a LevelDB benchmark (it uses the same library to compress data)
http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html

"LevelDB's write performance is better with compression than without since compression decreases the amount of data that has to be written to disk. Therefore LevelDB users can leave compression enabled in most scenarios without having worry about a tradeoff between space usage and performance."

"Performance of both LevelDB and TreeDB improves a small amount when compression is disabled. Note however that under different workloads, performance may very well be better with compression if it allows more of the working set to fit in memory."
Yup, I think we should do something like this. Though if we're getting it for free once we switch to LevelDB, then we might want to focus on that.
Oops, forgot we had a bug on this...
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 703660
Component: DOM: Core & HTML → DOM: IndexedDB
You need to log in before you can comment on or make changes to this bug.