Open Bug 1996558 Opened 7 months ago Updated 1 day ago

Automatically encrypt SQLite databases on disk

Categories

(Core :: SQLite and Embedded Database Bindings, enhancement)

enhancement

Tracking

()

ASSIGNED

People

(Reporter: nwipper, Assigned: nwipper)

References

(Blocks 4 open bugs)

Details

(Whiteboard: [size=3.5])

Attachments

(17 files, 5 obsolete files)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review

Encrypt all SQLite databases based on a key derived from the primary password, if available, or a default key.

Component: Security → SQLite and Embedded Database Bindings
Blocks: 2012079
Blocks: firefox-client-mvp
No longer blocks: 2012079
Severity: -- → S1
Priority: -- → P1
Severity: S1 → --
Priority: P1 → --
Whiteboard: [size=3.5]
Attachment #9522622 - Attachment description: WIP: Bug 1996558: Encrypt SQLite databases from mozStorageService → WIP: Bug 1996558 - Encrypt SQLite databases from mozStorageService
Flags: needinfo?(emilio)
Flags: needinfo?(emilio)

Follow-up patch on top of Nikolas's WIP that tightens the architecture
of the keystore + mozStorage integration without changing the
cryptographic primitives.

mozStorageService / Connection

  • Centralize NSS init: drop scattered EnsureNSSInitializedChromeOrContent
    release-asserts from CookieService, PermissionManager, LSObject, and
    XPCOMInit. mozStorageService::Init now registers the keystore profile
    observer; Connection::initialize calls EnsureNSSInitializedChromeOrContent
    lazily on the encryption path and returns NS_ERROR_FAILURE on failure
    instead of crashing the parent process.
  • Unify Connection::initialize paths: fold initializeSecure into
    initialize(nsIFile*), have initialize(nsIFileURL*) also set
    mDatabaseEncrypted=true when it injects ?key=, so GetDefaultPageSize
    and initializeClone agree on whether the connection is encrypted.
  • Strip a leading "file:" scheme and any ?key= query from the path
    returned by PRAGMA database_list so encrypted clones never produce a
    double ?key=.

security/keystore module

  • Replace the push-model SetCurrentProfilePath (8 sites in
    nsToolkitProfileService.cpp) with a small KeyStorageObserver that
    listens on profile-do-change / profile-after-change /
    profile-before-change / xpcom-shutdown, resolves the profile dir on
    the main thread via NS_GetSpecialDirectory, and caches it under
    sKeyMutex. The observer also unregisters itself and clears its
    StaticRefPtr at xpcom-shutdown so we don't leak.
  • Bump the keystore magic to "# mozilla secure key storage v1"; reject
    unknown versions cleanly.
  • Base64-encode per-DB identifiers in the on-disk format so a Windows
    path that legitimately contains ':' cannot break the parser.
  • Replace the appending writer with a full-file rewrite via
    NS_NewSafeLocalFileOutputStream, so we get write-to-temp + rename
    atomicity. Keep the wrapped system-key bytes in memory
    (sSystemKeyWrapped) so each rewrite preserves the system-key line.
  • Fix GetKeyByPath to use InitWithPath(NS_ConvertUTF8toUTF16(...)) -
    callers (e.g. PRAGMA database_list) supply UTF-8, not native bytes.
  • Reject DBs outside the profile directory with NS_ERROR_NOT_AVAILABLE
    so mozStorage callers can fall back to plaintext for temp DBs.

Pref

  • Switch security.storage.encryption.sqlite.enabled from mirror: once
    to mirror: always (RelaxedAtomicBool) so browser-chrome tests that
    set the pref via their manifest see the updated value. All consumers
    drop the _AtStartup suffix from the accessor.

Tests

  • Document the storage/marionette tests that legitimately cannot run
    under encryption (page_size-locked tests, PBM-encryption tests, the
    backup compatibility fixtures) and explain the override in each
    manifest.
  • Make browser_connect.js robust under test-verify: use a unique DB
    name per task run and remove -wal/-shm/-journal sidecars defensively
    so iteration N+1 cannot inherit a stale WAL that obfsvfs cannot
    decrypt.
Attachment #9573748 - Attachment is obsolete: true

Lock the pref in modules/libpref/init/all.js so end users cannot toggle
it from about:config or user.js, and add it to the Preferences
enterprise-policy allowlist so admins can still drive it from
policies.json.

The pref stays declared as a StaticPref in StaticPrefList.yaml; this
patch only adds the locking and policy plumbing.

Snapshot keystore.db / cookies.sqlite presence in key::Init (before any
storage consumer can touch them), so we can still reason about the
profile's pre-storage state even after the patched encryption code has
acted in this session.

Two policy paths:

  • pref off + keystore.db existed at startup (encrypted profile, pref
    pinned off): live-flip the pref via the unlock/setDefault/relock
    dance from key::Init -- before the first DB open -- and continue
    running. Idempotent across restarts; no _Exit needed.
  • pref on + no keystore.db at startup + cookies.sqlite existed at
    startup (plaintext profile, pref pinned on by enterprise policy):
    refuse to start with a clear FATAL message and std::_Exit(1) from
    profile-after-change.

ObfuscatingVFS preserves the SQLite header verbatim, so distinguishing
encrypted from plaintext from on-disk content is unreliable; the
existence-snapshot is the only robust signal.

When a connection is opened with the encryption pref on and the file
already exists in plaintext form (SQLite header byte 20 == 0), copy
its schema, rows, indexes, views, and triggers into a sibling
.migrating file via obfsvfs and atomically swap. Idempotent on already-
encrypted files; cleans up -wal / -shm / -journal sidecars of the
plaintext source.

Wired into both Connection::initialize overloads (nsIFile* and
nsIFileURL*) right after the per-file key is fetched and before
sqlite3_open_v2. With migration in place, the case-2 fail-stop in
ApplyEncryptionPolicy is no longer reachable, so drop it (and the
profile-after-change observer wiring that called it).

Also adjust GetKeyByFile to fall back to NS_GetSpecialDirectory when
sProfilePath is empty (e.g. from the main-thread synthesized observer
fire before profile-do-change has actually run), so off-main-thread
callers get NS_ERROR_NOT_AVAILABLE rather than a crash.

Migration test added as a browser-chrome test alongside the existing
keystore tests; xpcshell isn't a viable host for these because NSS
SDR isn't initialized in stock xpcshell. The two existing tests now
also do the unlock/setDefault/relock dance to engage encryption,
since the all.js lock blocks the previous prefs = [...] override.

Attachment #9522622 - Attachment description: WIP: Bug 1996558 - Encrypt SQLite databases from mozStorageService → Bug 1996558 - Encrypt SQLite databases from mozStorageService

Follow-up to D270165 addressing reviewer comments outside security/keystore/:

  • Drop duplicate hasKey/mDatabaseEncrypted assignment and clarify the
    "outside profile" fallback comments in Connection::initialize.
  • Restore explicit return rv at the three sqlite3_open failure sites
    that the original patch had switched to NS_ENSURE_SUCCESS.
  • Strengthen the security.storage.encryption.sqlite.enabled pref
    description to flag it INTERNAL / DO NOT ENABLE pending the
    enterprise-policy gate in the rest of the stack.
  • Replace the mozilla_net_percent_encode Rust addition in
    netwerk/base/idna_glue with a call to NS_EscapeURLSpan(esc_FilePath |
    esc_Forced) inside a new shared helper, storage::PreparePathForURI,
    exported via storage/StoragePathUtil.h. NS_EscapeURLSpan covers '?',
    '#', '&', space, etc. -- not just '%'.
  • Expose obfsvfs::kObfsPageSize from ObfuscatingVFS.h so that
    Connection::GetDefaultPageSize and ObfuscatingVFS.cpp share the same
    8192 constant instead of duplicating literals.
  • Use storage::PreparePathForURI in toolkit/components/places/
    Database.cpp::AttachDatabase before building the file: URI, so paths
    containing URI-significant bytes don't produce malformed URIs.
  • Make ExtractURIPathAndQuery tolerant of bare filesystem paths. PRAGMA
    database_list returns normalized filenames (no file: prefix), so the
    encrypted-clone branch was previously returning NS_ERROR_FAILURE and
    breaking Connection::initializeClone for encrypted DBs with attached
    databases.
  • Track the pending mozIStoragePendingStatement returned by
    attachDatabase in Sqlite.sys.mjs's _pendingStatements map, matching
    the _executeStatement pattern, so shutdown can cancel in-flight
    ATTACH operations instead of leaking them.
  • Parameterize test_page_size_is_32k.js on the encryption pref (8 KiB
    when on, 32 KiB when off) and drop its pref override in xpcshell.toml.
  • Strengthen the comment in dom/indexedDB/test/marionette/
    manifest.toml about the PBM x obfsvfs interaction gap and reference a
    pending follow-up bug.
Attachment #9575591 - Attachment description: WIP: Bug 1996558 - Lock SQLite-encryption pref and route override through enterprise policy → Bug 1996558 - Lock SQLite-encryption pref and route override through enterprise policy

Delete the bespoke security/keystore module introduced by the parent
revision and route per-database obfsvfs keys through security/lockstore
using KekType::LocalKey. The lockstore collection name for each SQLite
database is the SHA-256 of the database's path relative to the active
profile directory; the underlying DEK is created extractable so that
obfsvfs can consume the raw 32-byte key via the ?key=... URI.

  • New storage/SQLiteEncryption.{h,cpp} exposes
    mozilla::storage::GetEncryptionKey and ShutdownEncryptionKeystore
    on top of lockstore_ffi.
  • Three consumers rewired: mozStorageService::Init drops the explicit
    key::Init() call; mozStorageService::Observe(xpcom-shutdown-threads)
    calls ShutdownEncryptionKeystore(); mozStorageConnection and
    toolkit/components/places/Database.cpp call GetEncryptionKey.
  • security/keystore/ deleted and dropped from security/moz.build.
  • Browser-chrome tests moved to storage/test/browser/encryption/ with
    the keystore.db existence assertion removed (lockstore manages its
    own DB).
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587240 - Attachment description: WIP: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp → Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp

Flip the YAML default so encrypted-SQLite is on out of the box for any
profile that doesn't already have the pref locked elsewhere. NOTE: the
companion pref(..., false, locked) in modules/libpref/init/all.js from
D297713 still overrides this at runtime; remove that line or flip its
value to actually observe the new default end-to-end.

  • Add browser/branding/enterprise/policies.json shipping
    security.storage.encryption.sqlite.enabled=locked-true as the
    Enterprise default. Wire it into branding-common.mozbuild so the
    packaged distribution picks it up.
  • Flip the same pref default to true,locked in all.js so the
    StaticPrefList.yaml change and all.js agree.
  • Fix FeltProcessParent::createProfile call to pass the 3rd "source"
    argument required since Bug 2026972 (D290460, 2026-05-15) added it
    to nsIToolkitProfileService.idl. Without this, Felt enters a
    start/stop loop on every launch because createProfile throws
    NS_ERROR_XPC_NOT_ENOUGH_ARGS.

Add an EncryptedDatabases marker under [Compatibility] in the profile's
compatibility.ini. The marker is read in CheckCompatibility at the very
top of XRE_mainStartup -- after profile lock acquisition but before
mDirProvider.SetProfile and any XPCOM/storage initialization. A new gate
CheckEncryptionCompatibility refuses to launch when the marker and the
launching build's security.storage.encryption.sqlite.enabled pref
disagree, so we never silently corrupt a profile by opening encrypted
databases with the wrong VFS (or vice versa).

To disambiguate "marker absent but profile has DBs", a 21-byte SQLite
header peek classifies the existing DBs as encrypted (page=8192,
reserved=32) or plaintext. Already-encrypted profiles from builds that
pre-date the marker self-heal on first launch with this code.

The marker is written from two hook points:

  • storage/SQLiteEncryption.cpp ProfileObserver on profile-after-change:
    runs before any DB is opened (per the verified startup ordering:
    profile-do-change -> policies-startup -> profile-after-change ->
    lazy mozStorageService init). Idempotent.
  • storage/SQLiteEncryption.cpp GetEncryptionKey on first-DEK-create:
    defensive backup write for xpcshell / embedding paths where the
    observer doesn't fire. Once per session.

nsIToolkitProfileService gains markCurrentProfileEncrypted(bool); the
writer (MaybeWriteEncryptionMarker in nsAppRunner) preserves existing
[Compatibility] keys, uses the same PR_TRUNCATE atomic-replace pattern
as WriteVersion, and skips the write when the on-disk value already
matches.

The gate is skipped in MOZ_BACKGROUNDTASKS mode -- those tasks operate
on profile files independently of any SQLite consumer.

Suggested by mossop. New strings under toolkit/locales for the two
refusal dialogs. New xpcshell tests cover the writer's happy path,
idempotency, and key-preservation behaviour.

Several components hand-roll a byte->lowercase-hex loop; gcp flagged (D301076) that the SQLite-encryption keystore was about to add yet another copy. Introduce one shared mozilla::HexEncode in xpcom/io (next to Base64Encode, its binary->text sibling) and move sandboxBroker's local copy onto it. mfbt would be the natural home but cannot depend on nsACString, so the helper lives in the XPCOM io/ module.

Several components hand-roll a byte->lowercase-hex loop; gcp flagged (D301076) that the SQLite-encryption keystore was about to add yet another copy. Introduce one shared mozilla::HexEncode in xpcom/io (next to Base64Encode, its binary->text sibling) and move sandboxBroker's local copy onto it. mfbt would be the natural home but cannot depend on nsACString, so the helper lives in the XPCOM io/ module.

No longer blocks: 2040365
Attachment #9592027 - Attachment is obsolete: true
Attachment #9592014 - Attachment description: Bug 1996558 - Add a shared mozilla::HexEncode helper r?#xpcom-reviewers → Bug 1996558 - Add shared mozilla::HexEncode/HexDecode helpers in xpcom/io r?#xpcom-reviewers

Flips both the all.js runtime default and the StaticPrefList.yaml default
of security.storage.encryption.sqlite.enabled from false to true, and
drops the locked attribute from the all.js pref so per-test
prefs=[...=false] opt-outs (page-size-sensitive tests that need an
unencrypted 4096-byte page size) still work.

The YAML flip is required because the encryption gate in
XRE_mainStartup runs before mDirProvider.InitializeUserPrefs(), so a
runtime override in all.js alone wouldn't be visible to the gate; the
gate reads the StaticPrefs accessor which falls back to the YAML
default until prefs are loaded.

For try-side verification of the Case-3 (encryption enabled by default)
codepath only -- not for landing.

Attachment #9575592 - Attachment is obsolete: true
Attachment #9575593 - Attachment is obsolete: true

Until this commit, mozStorage's at-rest SQLite encryption stack
bootstrapped on a LocalKey (lockstore::kek::local:sqlite). That
LocalKey's kek_bytes field stores the raw AES-256 KEK plaintext
in lockstore.keys.sqlite -- a file that is itself plaintext
because kvstore-backed databases don't go through obfsvfs. So
anyone with same-UID filesystem access could lift the wrapping
key and decrypt every per-DB DEK, defeating the encryption
layer's confidentiality story.

Promote the SQLite-encryption bootstrap to a Password KEK whose
password is the console-supplied primarySecret. PasswordKekRecord
stores [salt | iterations | AES-GCM(ciphertext)] rather than
raw bytes (lib.rs:172-186) so a lockstore.keys.sqlite leak no
longer yields the KEK without also acquiring primarySecret.

Felt parent (FeltProcessParent.sys.mjs):

  • Pre-fetch primarySecret via ConsoleClient.getPrimarySecret()
    BEFORE spawning the Firefox child. Cache via Services.felt
    .setPrimarySecret(hex) on the parent side.
  • On the very first post-spawn .then() step, push it over the
    existing Felt IPC channel via Services.felt.sendPrimarySecret(),
    ahead of sendAccessToken / sendReady (no firefoxReady gate).

Felt IPC (rust/{message.rs,utils.rs,components.rs,client.rs} +
nsIFelt.idl): mirror the SSOPassword plumbing for a new
PrimarySecret variant + PRIMARY_SECRET static + set/send/peek/clear
XPCOM surface.

Firefox child storage/SQLiteEncryption.cpp:

  • At profile-do-change, EnsurePrimarySecretCached() polls
    Services.felt.peekPrimarySecret() every 50ms up to 5s. The
    primarySecret stash mirrors sHandle / sKekRef in lifecycle:
    established at profile-do-change, cleared at quit-application.
  • GetEncryptionKey()'s bootstrap branch becomes an
    unlock-or-create against lockstore::kek::password:sqlite:
    • keystore_unlock_kek with TTL=u32::MAX ms (49.7d == effectively
      session-unlimited; lockstore's 0 means "don't cache" not
      "infinity").
    • On NS_ERROR_NOT_AVAILABLE -> keystore_create_kek with the
      same identifier; idempotent on the kek_ref dedup at
      keystore.rs:1182-1185.
    • After the first-ever create, MigrateLocalToPasswordKek()
      rotates every collection still carrying a local:sqlite
      wrapping (add_kek + switch_kek + remove_kek), then deletes
      the orphaned LocalKey record via delete_kek.
  • Cache-expiry safety: if get_dek returns NS_ERROR_NOT_AVAILABLE
    mid-session, transparently re-unlock with the cached
    primarySecret and retry once before failing.

Failure modes (validated against the existing encryption gate):

  • primarySecret never arrives -> Password KEK can't be
    unlocked -> get_dek fails -> mozStorage refuses the open
    -> CheckEncryptionCompatibility in nsAppRunner refuses launch
    via the existing "Encrypted Profile" dialog.
  • primarySecret rotated server-side -> AEAD tag failure on
    unlock surfaces as NS_ERROR_ABORT (LockstoreError::WrongPassword);
    the storage layer fails opens, gate refuses launch. Recovery
    requires SSO re-auth so the console re-issues primarySecret.
  • TTL expiry -> existing mozStorage connections survive (the
    obfsvfs key was baked into the xFile state at open time per
    mozStorageConnection.cpp); new opens hit the re-unlock path.

Out of scope: Felt's own scratch profile (T/felt-default) stays
on LocalKey -- Felt can't gate its own startup on a secret it
hasn't fetched yet. Its content sensitivity is bounded (SSO
session cookies, formhistory email, IdP URLs in places.sqlite);
high-value material (auth tokens, NSS keys) is gated behind the
SSO+primarySecret-derived NSS slot password, not the LocalKey.
Felt-profile hardening (OS keychain / ephemeral profile) is
filed as a follow-up.

Backported from enterprise-firefox 16d9ea9e4e79.

Threads an explicit key_size through keystore_create_dek (LockstoreService,
lockstore_ffi, keystore.rs, nsILockstore.idl); SQLiteEncryption passes
kDekBytes and static_asserts kDekBytes == IPCStreamCipherStrategy::KeyType size.

Also updates the lockstore gtest call sites (TestLockstoreKeystore.cpp,
TestLockstoreDatastore.cpp, TestLockstoreService.cpp) to the new 5-arg arity;
the enterprise commit updated only TestLockstoreService.cpp, leaving the other
two on the old signature (a latent build break).

Backported from enterprise-firefox ed9df1f698fa.

obfsvfs is the SQLite default VFS; obfsOpen owns the at-rest policy so keyless
sqlite3_open_v2 (rusqlite/skv/app-services) gets path-aware encryption. Per-file
key cached for shutdown journal finalization; PeekOnDiskHeader uses
page_size==8192 && reserved==32; lockstore.keys.sqlite bootstrap bypass;
no-CREATE-of-missing forwards to lower VFS.

(Enterprise Password-KEK/primarySecret + vault routing remain enterprise-only,
excluded; .cargo vendor-reorder dropped.)

Not for landing. Included in the stack so the rusqlite/skv default-VFS work can
be exercised with encryption forced ON.

Two defects broke mozIStorageAsyncConnection.backupToFileAsync when the
database is encrypted through obfsvfs:

  1. The backup writes <dest>.tmp then renames it to <dest>. The per-database
    key is keyed by the file's profile-relative path, and obfsvfs only strips
    -wal/-journal suffixes (not .tmp), so the renamed backup had no key and
    could not be reopened. RekeyEncryptedDatabaseForRename moves the DEK from
    the .tmp collection to the final one after the rename (the key bytes are
    unchanged, so the already-written ciphertext stays valid).

  2. A multi-step backup never completed. The pager validates that the source
    has not changed by reading the 16-byte change-counter region (page 1,
    offset 24) directly from the file on every shared-lock acquisition. obfsRead
    only decoded full-page reads, so that read returned ciphertext for bytes
    32-39 and never matched the decoded value the pager cached -- the pager
    concluded the file changed and restarted the backup on every step. obfsRead
    now serves that specific read from a decoded copy of page 1.

Also enables test_connection_online_backup.js (made mode-aware for the encrypted
page size) and removes its xpcshell.toml opt-out.

Instead of opting these storage/toolkit tests out of SQLite encryption, assert
the correct behavior in BOTH the encrypted and plaintext configurations
(detected via the security.storage.encryption.sqlite.enabled pref):

  • test_storage_service: a directory-as-database open surfaces NS_ERROR_FAILURE
    through obfsvfs (and records no open telemetry) vs NS_ERROR_FILE_ACCESS_DENIED
    / Glean "access" on the plain VFS.
  • test_vacuum: obfsvfs forces a fixed page size, so a VACUUM cannot change it
    (expect unchanged vs 1024); and an encrypted database keeps full auto_vacuum
    (1) rather than switching to incremental (2).
  • test_sqlite_autoVacuum: a fresh encrypted database reports full auto_vacuum
    (1), which reclaims freed pages at commit, so there is no freelist for an idle
    VACUUM to reclaim; assert that auto-reclaim behavior instead.
  • test_cache_size: the requested page size is ignored under encryption (cache
    size is KiB-based and unchanged).
  • test_sqlite_secure_delete: an encrypted database never stores the plaintext on
    disk, so the pre-delete "string is present" check only applies unencrypted;
    the post-delete absence check holds in both modes.

Verified passing both with and without SQLite encryption.

test_connection_online_backup stays opted out for now with a TODO: it exposes a
real backup+encryption defect (the backup destination's DEK is not found on
reopen), which needs a code fix rather than a test change.

Attachment #9594325 - Attachment description: Bug 1996558 - Fix backupToFileAsync for encrypted SQLite databases. r?gcp → Bug 1996558 - Disable profile backup when SQLite at-rest encryption is enabled. r?fchasen
Attachment #9587240 - Attachment description: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp → WIP: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9594326 - Attachment description: Bug 1996558 - Make SQLite-encryption-sensitive xpcshell tests encryption-aware r=gcp → WIP: Bug 1996558 - Make SQLite-encryption-sensitive xpcshell tests encryption-aware r=gcp
Attachment #9587240 - Attachment description: WIP: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp → Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp

The keystore previously closed at quit-application (AppShutdownConfirmed), the earliest shutdown phase, which forced obfsvfs to cache the raw per-file DEK (ObfsFile::aKey plus WAL/journal partner-inheritance) so it could finalize a connection's journals after the keystore was already gone. That early teardown was justified by a comment claiming LateWriteChecks fires at AppShutdownNetTeardown; it actually fires at XPCOMShutdownThreads (toolkit.shutdown.lateWriteChecksStage=2), so there is ample headroom to keep the keystore alive much later.

Keep lockstore available for late profile writes and shut it down last, at XPCOMWillShutdown -- after Places (profile-before-change) and QuotaManager/IndexedDB/DOM-storage (profile-before-change-qm) have closed their connections, and after AppShutdownTelemetry (past which nothing writes the profile). Both the storage-internal keystore handle (SQLiteEncryption) and the LockstoreService XPCOM handle now tear down on xpcom-will-shutdown; InitEncryptionKeystore refuses to register/open once already in or beyond XPCOMWillShutdown.

With lockstore alive that late, obfsvfs no longer needs to cache the DEK: WAL and journal opens fall through to the policy branch and re-derive the same per-database key from lockstore on demand (DeriveMainDbPath strips the -wal/-journal suffix), including the final checkpoint at a connection's close during shutdown. Remove ObfsFile::aKey, the partner-inheritance fast path, and the now-dead keyReady flag; pPartner is retained for checkpoint coordination.

Attachment #9596259 - Attachment description: WIP: Bug 1996558 - Shut down the SQLite encryption keystore at XPCOMWillShutdown and stop caching the per-file DEK r?gcp → Bug 1996558 - Shut down the SQLite encryption keystore at XPCOMWillShutdown and stop caching the per-file DEK r?gcp
Attachment #9593526 - Attachment is obsolete: true
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587286 - Attachment description: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9587240 - Attachment description: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp → WIP: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp
Attachment #9587240 - Attachment description: WIP: Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp → Bug 1996558 - Address review feedback for SQLite encryption (excluding keystore). r?gcp
Attachment #9587286 - Attachment description: WIP: Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp → Bug 1996558 - Replace security/keystore with security/lockstore (LocalKey). r?gcp
Attachment #9594326 - Attachment description: WIP: Bug 1996558 - Make SQLite-encryption-sensitive xpcshell tests encryption-aware r=gcp → Bug 1996558 - Make SQLite-encryption-sensitive xpcshell tests encryption-aware r=gcp
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: