1887978 - Add support to QuotaManager to allow chrome-privileged and WebExtension IndexedDB databases to be backed up

Status: NEW

(Core :: Storage: Quota Manager, enhancement)

Description

[Mike Conley (:mconley) (:⚙️)]

This was something we were discussing with asuth, janv and rpl over Matrix last week.

In bug 1885369, and its parent bug 1883052, we're trying to make it possible to create runtime backups of various data stores in the user's profile directory. The aim is to make it easier to both recover from data loss / corruption due to a bug in Firefox, a crash, or a sudden power loss, as well as to make it easier to move profile data between desktop devices.

We're using the SQLite Online Backup API to clone most SQLite databases in the user profile directory, but the IndexedDB databases managed by QuotaManager seem to be much, much more complicated a case.

According to asuth, it might be possible to "teach QuotaManager to back up a set of origin directories to some target directory and optionally restore origin directories from a source directory. For the backup, I think the sanest thing would be to use directory locks to ensure that the quota clients are in an inert state.".

Here's more from that conversation:

The simplest initial pass would require obtaining an exclusive directory lock which would potentially break the WebExtensions, but since I think the "event pages" conceptually can expect to be terminated and restarted, this might be something the WebExtensions subsystem could help with. Like it could kill the extension page like it would for an upgrade, ask for the origin to be backed up, and then start the event page again. Sidebars might be less happy, although I know TST seems to upgrade okay in such situations, so maybe life-cycling the webext in the same way would be fine.

An incremental enhancement could then be to do something where wouldn't need to disrupt open IDB databases but instead queue what amounts to a write transaction to let us pause the IDB execution, although there might still be some edge cases about active refcounted blobs that could be something that would need specific handling. (This might work out okay if we make sure that on import we treat the origin as dirty and make sure we reconcile the blobs/files.)

Comment 1

[Mike Conley (:mconley) (:⚙️)]

This is now no longer something that we need for the first version of the profile backup feature. If we find that there are IndexedDB databases that do need to be backed up, we'll see if we can read-out their contents / write-in their contents on a case by case basis, rather than aim for the generalized solution that this bug entails.

Comment 2

[Mike Conley (:mconley) (:⚙️)]

(If the Storage team wants to close this bug out now, you're welcome to, unless keeping it open is still useful for your purposes)