1659464 - Provide documentation for Gecko/Firefox engineers on how to use IndexedDB from system code and related best practices

Status: NEW

(Core :: Storage: IndexedDB, enhancement, P3)

Description

[Andrew Sutherland [:asuth] (he/him)]

While many of the best practices for using IndexedDB from system code are the same as for content code, there are some important differences, and we should both document them and make sure that we have enhancements on file for cases where we can do better than the current (hopefully documented) situation.

1. Lifecycle. System code can run at times content code could never run, both before the system is fully initialized and as the system is shutting down.
2. Bucket clearing and data corruption. The Storage spec is clear that data stored in a single bucket should be cleared in its entirety. And implicitly, content won't perceive this clearing, since pages can also be reloaded when this happens. Although multiple buckets are being spec'ed right now, this means that an origin should be cleared in its entirety. For system code like remote-settings which opens databases under the auspices of the system principal, the bucket granularity is a trickier issue, but it's clear that clearing all system-principaled storage because a single thing got corrupted is likely not helpful.
3. Failures modes / assumed capabilities. Content on the web cannot assume that a browser has IndexedDB available. When writing system code for the Firefox browser with the system principal which has all capabilities defined to be available, it's a reasonable starting assumption that IndexedDB will be always be present and will always work. Unfortunately, this isn't always the case.

I propose the following in terms of documentation given the current state of things:

1. Lifecycle: Make sure to link to documentation on shutdown blockers and the need to potentially ensure that startup doesn't occur before profile-after-change.
2. Bucket clearing and data corruption: Document that corruption can happen and that it may be reported both at opening time and during usage, manifesting as an UnknownError (I think...), and that the only course of action in these cases is to delete the database. Also document that IndexedDB may automatically delete the database on its own and pretend that it never existed if the database is found to be corrupt during opening.
3. Failure modes: Document that there are ways that QuotaManager can break that will in turn break IndexedDB and these will also manifest as UnknownError. So for important data on nightly, it might be appropriate be aware that one is on nightly and the breakage might be something that auto-corrects in the next release.

I propose and have created the following in terms of enhancements that we will reference in the documentation and pursue implementing:

1. Lifecycle:
   • Bug 1659447: By default I would like Chrome-privileged IDB databases to use a Chrome-specific opening signature that explicitly takes a shutdown blocker. This is conceptually similar to proof-of-mutex that will also be used to ensure that a database isn't opened after when shutdown should have started.
   • Bug 1659454: The same thing but for system-principaled worker constructors. This also came up before with remote settings.
2. Bucket clearing and data corruption:
   • Bug 1659458: Expose an API that expresses whether QuotaManager is entirely broken or not. This should let IDB users determine if errors are indeed specific to their database or are indicative of a browser-wide problem and that there may be no benefit to trying to delete a database or even do anything with IDB. In such a case the mechanism might need to fall back to its most degraded form of operation.
   • We may also want to propose enhancements to the IDB spec to better express corruption.
3. Failure modes:
   • Bug 1659458: The same API as above exposing QM breakage.

Obviously all of this would want to happen in parallel with our extensive efforts to make IndexedDB and QuotaManager more resilient, fault-tolerant, automatically recovering, etc.