1881542 - Update SQLite online backup API to allow callers to back up to a byte array

Status: RESOLVED WONTFIX

(Toolkit :: Storage, task)

Description

[Mike Conley (:mconley) (:⚙️)]

So the online backup API that we exposed in bug 1869060 has let us make some fairly interesting backup prototypes. Great!

One thing that's not been so great is that the prototypes all have had to use temporary file system space to for the cloned databases, which are then read back into memory to be streamed into an archive file.

It'd be really great if we could skip that first write to disk - that'd generally make things faster, and it'd also help us avoid leaving fragments of backed up databases around if we crash or abort halfway through a backup.

I think we can pull this off if we adjust the exposed API to resolve with a byte array of some kind, representing the serialized version of the cloned database.

I think we can do that by:

1. Cloning the source database to an in-memory database
2. Serializing that in-memory database to some memory region using this API: https://www.sqlite.org/c3ref/serialize.html
3. Handing back a pointer to that memory region, which can then be represented as some kind of byte array to the (presumably JS) caller.

I'm going to try doing that in this bug.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1881542 - Have the SQLite online backup mechanism return an ArrayBuffer instead of writing to a file. r?mak!

Comment 2

[Mike Conley (:mconley) (:⚙️)]

We're not going to go this route. Significantly large databases are likely to cause OOM problems with serializing for machines with low or very fragmented memory.