1829933 - Firefox IndexedDB thread at 100% cpu constantly

Status: RESOLVED WORKSFORME

(Core :: Storage: IndexedDB, defect, P2)

Description

[Kertesz Laszlo]

Attachment: Firefox 2023-04-25 22.08 profile.json.gz

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0

Steps to reproduce:

Started Firefox

Actual results:

The main firefox thread uses 100% cpu. Profiling seems to point towards some local database issue?

Expected results:

The main Firefox thread should not use 100% cpu constantly

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Gtk' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Martin Stránský [:stransky] (ni? me)]

Can you use mozilla profiler and share that profile on-line? (There's a menu entry "share profile" in right up corner).
Thanks.

Comment 3

[Martin Stránský [:stransky] (ni? me)]

Okay, managed to open from file.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

Yes, looks like sqlite3 issue. sqlite3RunParser() calls and similar ones.

Comment 5

[Kertesz Laszlo]

Hmm. This behavior just stopped all of a sudden. Now everything works fine, no extra cpu usage.
Previously it was consistent after browser restarts, computer reboots for a few days. Was there some kind of migration of database in the background (the issue begun with the first 112 version)?

Comment 6

[Kertesz Laszlo]

Aand it's back again. Tomorrow morning firefox process is back at 100% cpu usage. Note that i leave my computer running 24/7 but even if i restart firefox this behavior is consistent.
New profiler link:
https://share.firefox.dev/42ckAAV

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

Do you possibly have some addons installed? I see some extensions related JS running in the profile and I wonder if some addon uses indexedDB very heavily.

Comment 8

[Kertesz Laszlo]

I have Ublock Origin and keepassxc.
But i started Firefox in debugging mode and still did the same.

Comment 9

[Olli Pettay [:smaug][[e-mail]]]

Could you possibly provide a performance profile which shows also the extension process, and maybe even all the processes for the relevant web pages? if you don't want to share it publicly, feel free to send to me only.
Something somewhere is triggering lots of use of IndexedDB, and that something doesn't seem to be the parent process, so we need a bit more information here.
Thanks.

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

Janv, does the profile https://bugzilla.mozilla.org/show_bug.cgi?id=1829933#c6 give any hints what might be going wrong.

Comment 11

[Jan Varga [:janv]]

Does the performance problem happen after opening a private window ?
Do you remember setting dom.indexedDB.privateBrowsing.enabled preference to true ?
Can you check what is your setting in about:preferences ?
Thanks.

Comment 12

[Kertesz Laszlo]

(In reply to Jan Varga [:janv] from comment #11)

Does the performance problem happen after opening a private window ?
Do you remember setting dom.indexedDB.privateBrowsing.enabled preference to true ?
Can you check what is your setting in about:preferences ?
Thanks.

This setting is at default (false).
The performance issue appears regardless of opening private windows or not. It happens a minute or so after startup even if i don't do anything.

Comment 13

[Jens Stutte [:jstutte]]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #10)

Janv, does the profile https://bugzilla.mozilla.org/show_bug.cgi?id=1829933#c6 give any hints what might be going wrong.

The symptoms in the profile of comment 6 are very similar to what we see in PBM in bug 1832639. We seem to be stuck at the same iteration during vacuuming. This might indicate that SQLite did not free anything making aFreelistCount never become zero. We know from the investigation of bug 1832639 that this happens if a DB does not support vacuuming, but it seems unclear why this could be the case here for non-PBM.

In any case I'd propose to have a mitigation that interrupts the idle processing if the aFreelistCount did not change between two iterations for robustness. We might consider having an assertion there to avoid obvious errors in setting up the DB on our side, too.

Note that any new incoming IDB request will interrupt our processing, too, so things should generally work as expected, we just burn quite some CPU cycles uselessly.

Comment 14

[Jens Stutte [:jstutte]]

Attachment: Bug 1829933 - Interrupt vacuum loop if nothing has been freed. r?#dom-storage-reviewers

It seems we have currently no test at all for vacuuming maintenance. We should add some testing in general.

Comment 15

[Jens Stutte [:jstutte]]

(the profile shows it is the IndexedDB thread)

Comment 16

[Jens Stutte [:jstutte]]

There is more to come here to improve vacuuming.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ad7f2c72892c
Interrupt vacuum loop if nothing has been freed. r=dom-storage-reviewers,janv

Comment 18

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/ad7f2c72892c

Comment 19

[Jan Varga [:janv]]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #10)

Janv, does the profile https://bugzilla.mozilla.org/show_bug.cgi?id=1829933#c6 give any hints what might be going wrong.

Yes, we should rework how we do the incremental vacuum in DatabaseConnection::ReclaimFreePagesWhileIdle. There's already bug 1833473 which may help with the perf problem.

Comment 20

[Jens Stutte [:jstutte]]

Hi Kertesz! Can you verify with the latest nightly if you do not see this behavior anymore (the patch should have mitigated this) ? If not we can close this bug and do the remaining work on bug 1833473. Thank you!

Comment 21

[Kertesz Laszlo]

Hello,
I am testing the build, using the same profile (i renamed the folders and i have my sessions). So far so good (4 or 5 hours), cpu usage is low when idle.

Comment 22

[Jens Stutte [:jstutte]]

Thanks! The remaining performance improvement work will happen on the other bug then.