Closed Bug 1522638 Opened 8 months ago Closed 6 months ago

support bulk insert and drop for kvstore

Categories

(Toolkit :: Storage, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED
mozilla68
Tracking Status
firefox68 --- fixed

People

(Reporter: nanj, Assigned: nanj)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

As Activity Stream is planning on replacing the json cache files with kvstore, we've found these two features could be helpful for some use cases.

In particular, for a big dataset, bulk insert, i.e. put multiple key/value pairs to the store in a single transaction, will be way more efficient than calling kvstore.put separately on each key/value pair, because there is an implicit transaction involved for each call. This also benefits the consumer with the "all-or-nothing" semantics when the batch insert is desired.

As a dual feature, a drop API, which drops everything in the given store, would be better than calling delete repeatedly if the consumer wants to truncate the store in a single transaction. LMDB has mdb_drop for this, we can add it to rkv and expose it via kvstore.

Depends on: 1490496

Agreed, both bulk insert and drop would be useful. Is bulk insert different from the support for transactions requested in bug 1499238?

(In reply to Myk Melez [:myk] [@mykmelez] from comment #1)

Agreed, both bulk insert and drop would be useful. Is bulk insert different from the support for transactions requested in bug 1499238?

Yes, they are related. I was actually commenting this feature request in bug 1499238 first, but then realized that this might want a separate bug on its own.

To implement bulk insert, presumably we will be using cursor plus a write transaction. We can still hide this implementation detail in the kvstore without exposing the transaction to the user at all. And I believe it's better for now than letting the consumers do so with a transaction. I quite like the current API design in kvstore.jsm, it's neat, also hides the unnecessary complexity for the regular uses.

Still trying to wrap my head into this XPCom, hopefully, I can at least help to review some patches, or even better hack on it directly :-)

Component: General → Storage
Priority: -- → P3
Depends on: 1530467

This adds the bulk insert to kvstore as discussed in Bug 1522638

Assignee: nobody → najiang
Status: NEW → ASSIGNED
Pushed by najiang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/49ffc038ef84
Add bulk insert to kvstore r=myk,mossop,nika
Status: ASSIGNED → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.