2054381 - Bookmarks/history locked error at startup despite no external process holding places.sqlite — appears to be an internal ConcurrentConnection race in Database::EnsureConnection()

Status: RESOLVED FIXED

(Toolkit :: Places, defect, P2)

Description

[Sean Walsh]

Steps to reproduce:

Upon fresh boot of Firefox running on Linux Mint 22.3, Firefox will occasionally display the error "The bookmarks and history system will not be functional because one of Firefox's files is in use by another application. Some security software can cause this problem."
I'm not sure what this error coincides with, as it does not happen every time. Approximately 1 out of every 10 boots.

Actual results:

The error doesn't allow me to use my bookmarks or see browsing history.

Expected results:

Here's what I've done to troubleshoot the problem.
-Updated to the most recent version of Firefox that Linux Mint will allow. It was installed by default.
-using ext4 filesystem; meaning not NTFS/dual boot/windows related problem
-No duplicate Firefox installs
-Correct permissions exist
-Timeshift (Linux Mint's backup program) explicitly excludes /home directory where Firefox is installed
-No antivirus/sync tool holding the file
-Evidence ruling out external causes: confirmed via lsof/fuser that only Firefox's own main process ever holds the file
Reference: point to toolkit/components/places/Database.cpp, function EnsureConnection(), and the ConcurrentConnection::MaybeInterrupt() retry logic — flag that this looks like the retry occasionally not being sufficient

Comment 1

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Firefox::Bookmarks & History' component, but is not confident enough to move the bug to that component.

Comment 2

[Sean Walsh]

I've also tried following the advice in the link that accompanies the error. The advice is to delete the various sqlite files (e.g. favicons.sqlite, places.sqlite, favicons.sqlite-shm, favicons.sqlite-wal, places.sqlite-shm and places.sqlite-wal) and restart. While this does work, it is only temporary. The error appeared again a few boots later. Restarting Firefox works just as well as deleting these files.

Comment 3

[Marco Bonardo [:mak]]

We have evidence the retry reduced the problem, but probably didn't solve it completely.

We should investigate the issue and see if we can make it more lenient (either wait longer or retry more often) before giving up.

Comment 4

[Marco Bonardo [:mak]]

There's one complication, that is we can't easily make the main-thread "await" for other threads to release their lock. We'd have to sleep the main-thread, included the UI.

Comment 5

[Marco Bonardo [:mak]]

We may have to force ConcurrentConnection to await for the main connection, and if it that doesn't go up before a certain amount of time (10-15s), forcibly start it. I would have liked to keep them more independent, but we can't do it until the main thread stops doing synchronous open.

Comment 6

[Marco Bonardo [:mak]]

Attachment: Bug 2054381 - Fix ConcurrentConnection LOCKED race on Places startup. r=#places-reviewers

In the parent process, ConcurrentConnection opening places.sqlite concurrently
with EnsureConnection() causes WAL-mode lock contention (SQLITE_LOCKED or
SQLITE_IOERR), which bypasses busy_timeout. Defer the CC open until
TOPIC_PLACES_INIT_COMPLETE, which fires only after the main connection is
established. A 10s fallback timer handles the case where the notification
never arrives.

Comment 7

[Marco Bonardo [:mak]]

Attachment: Bug 2054381 - Forward places-init-complete to content processes via IPC. r=#ipc-reviewers

Content processes don't receive TOPIC_PLACES_INIT_COMPLETE, so
ConcurrentConnection has no signal to retry opening the database after an
initial failure. Forward the notification from the parent via PContent so the
existing Observe() handler can trigger the retry.

Comment 8

[Marco Bonardo [:mak]]

Dropping the IPC changes from this bug and moving them to https://bugzilla.mozilla.org/show_bug.cgi?id=2059368 since they are unnecessary for the bug fix

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/b10910c797b0
https://hg.mozilla.org/integration/autoland/rev/d6819df16882
Fix ConcurrentConnection LOCKED race on Places startup. r=places-reviewers,daisuke

Comment 10

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/d6819df16882

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/55b850373b32
https://hg.mozilla.org/integration/autoland/rev/b8154c219191
Revert "Bug 2054381 - Fix ConcurrentConnection LOCKED race on Places startup. r=places-reviewers,daisuke" for causing gtest failures at ConcurrentConnection

Comment 12

[Silaghi Andreea]

Backed out for causing gtest failures at ConcurrentConnection
Backout Link
Push with failures
Failure Log
Failure line [task 2026-08-04T22:27:25.304+00:00] 22:27:25 INFO - TEST-START | test_ConcurrentConnection.test_database_not_present

Comment 13

[amarc]

backout merged to main: https://hg.mozilla.org/mozilla-central/rev/b8154c219191

Comment 14

[Marco Bonardo [:mak]]

It's failing (timing out on event loop spinning) only on linux-tsan

This is likely because it's super slow and it takes more than 10s to startup Places, and then the ConcurrentConnection timer fires and gives up.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/3372dfb93854
https://hg.mozilla.org/integration/autoland/rev/b49c0a456811
Fix ConcurrentConnection LOCKED race on Places startup. r=places-reviewers,daisuke

Comment 16

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/b49c0a456811