1830648 - High memory and CPU usage with many tabs and no tab unloading

Status: UNCONFIRMED

(Core :: Performance: Responsiveness, defect, P3)

Description

[Adam Nielsen]

Attachment: memory-report.json.gz

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/111.0

Steps to reproduce:

For the last year or so, Firefox no longer seems to have any sort of process limit, or any way of unloading old processes if the tabs haven't been used for a while.

This results in it using more and more memory until the system becomes unstable, the Linux out-of-memory manager gets invoked, and random applications get killed in order to keep Firefox running.

There is no way to get Firefox to reduce its memory usage - going to about:memory and clicking the various buttons frees up only 2-3 GB of memory, whereas exiting and restarting Firefox frees up over 30 GB of memory.

Firefox also uses up a lot of CPU - around 75% - due to the sheer number of processes/tabs running in the background. Again, exiting and restarting (with the same number of tabs open) stops most of this CPU usage.

Actual results:

I have to restart Firefox frequently - sometimes 3-4 times a day - in order to keep its memory use under 4 GB. Once it goes above this, the system can freeze and programs are randomly terminated by the Linux out-of-memory reaper, sometimes including Firefox.

In the attached memory report, the "vsize" for the Firefox process is 32 GB which is just excessive, yet after exiting and restarting it, with the same number of tabs "open", the vsize is reduced to only 2 GB, and my system has regained 30 GB of free memory.

Expected results:

The behaviour a year or two ago was fine, where the number of processes was limited to some fixed amount (usually around 8 or 16). If a process was needed to view a new tab, the process for the oldest non-pinned tab would be reused. This allowed Firefox's memory to remain relatively constant, and it could run for weeks without issue.

Comment 1

[Adam Nielsen]

Attachment: memory-report-after-restart.json.gz

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

This bug was moved into the Performance component.

:a.nielsen, could you make sure the following information is on this bug?

• For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
• ✅ For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
• Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm it is recent.

Thank you.

Comment 3

[Adam Nielsen]

Attachment: support.json

The attached memory reports were created just prior to the bug report being lodged. I have attached the support report.

Note that I experience this problem on three different computers running three different profiles and slightly different Firefox versions.

On one machine which only has 16 GB of memory, I have to use systemd to hard limit Firefox to only 8 GB of RAM to avoid system instability. This means Firefox freezes on average once a day when this memory limit is reached and I have to terminate the task and restart it. If I don't hard limit Firefox in this way then the whole machine freezes when memory is exhausted (to the point where even the mouse cursor does not move) and I have to use the SysRq+F key combination to tell Linux to terminate some tasks, and invariably some important program gets killed along with Firefox.

Evidently there is a major memory leak in Firefox, and it just keeps allocating memory until it is exhausted and the system crashes.

Comment 4

[Julien Wajsberg [:julienw]]

The Performance Impact Calculator has determined this bug's performance impact to be medium. If you'd like to request re-triage, you can reset the Performance Impact flag to "?" or needinfo the triage sheriff.

Impact on browser: Renders browser effectively unusable
[x] Causes severe resource usage

Do you know if your problem happens without any addons too?

Comment 5

[Julien Wajsberg [:julienw]]

I also have a lot tabs in my Firefox, and if I try to open them all (keeping ctrl + pagedown/pageup pressed) I think I can reproduce too (94 firefox processes according to ps).

Hey Nika, what do you think about this? Would there be a way to reduce the number of processes started? Or maybe this is related to Tab Discarding instead?

Comment 6

[Nika Layzell [:nika] (ni? for response)]

Given you're forcing all tabs to be active and not discarded, thanks to Fission we are somewhat forced to keep many processes open, at least one per-site, though we currently allow up to 4 if you have 4 distinct tabs from a given site. There's not much we can do to keep the process count down beyond discarding unused tabs, which will in turn free up the processes they are using.

In the example linked in comment 3, the machine currently has 72 processes dedicated to rendering tabs, which would roughly correspond to 18 unique sites loaded, each with 4 tabs, or more sites with fewer tabs. Cross-origin iframes within a tab, or use of multiple containers / private browsing windows could also inflate the number of processes (as we isolate these sites separately).

I can't think of a way forward here other than to improve our tab discarding situation to allow us to clean up unused background tabs more often to keep the process count down. We cannot avoid spawning new processes when new sites are loaded, as we're using these processes for sandboxing reasons. Tab discarding is unlikely to help with the spam-loading situation you mention in comment 5 though, as you are explicitly requesting every tab be loaded when doing that.

Comment 7

[Adam Nielsen]

To me if you held Ctrl+PgUp to load a bunch of tabs immediately then it would be acceptable for the memory use to increase. However in my case, I leave Firefox open 24/7 so some tabs have processes still running a week or more after I last used the tab.

I don't know how hard it would be to implement, but I would love to see another two options in about:config that I could play with, that work as follows:

One to limit the global number of processes (so if it's set to say 50, then when the 51st process tries to run, whichever tab was last used the furthest in the past has its process terminated). From Nika's description in comment 6 it sounds like this is already in place for a single site (limiting it to four processes per site) so this would be a similar limit in a similar situation just with a greater number and without the origin restriction. This would solve the problem I see on low-memory machines, where I could set the global process limit quite low, say to 16, effectively capping the amount of memory used. I used to see this behaviour on older versions of Firefox, because I would visit tabs I had last used only an hour ago and they would have to reload as they had been discarded. I never used to run out of memory on those versions (even if Firefox was running for weeks at a time) so getting that behaviour back would solve this issue for low memory machines.

The second about:config option I would love to see is a process timeout/lifetime, so I could set the number of seconds until a process is terminated after its tab becomes hidden. I could set this to 48 hours, which means any tab I haven't used in the last two days would have its process terminated and memory recovered. This would solve the problem on machines with larger amounts of memory, where you can afford to have many processes open at certain times but later have that memory returned to other applications if the tabs are no longer active. I imagine this wouldn't be too onerous to implement, as you could just have a timer that runs every 15 minutes or so, checks the last-unfocused time of each tab, and discards any that exceed the user-defined lifetime.

This wouldn't change anything for someone who quickly opened a bunch of tabs (where realistically you would expect greater memory usage) but for long running browser instances it would allow unused tabs to be discarded over time, avoiding the eventual memory exhaustion that currently occurs.

Comment 8

[Gregory Pappas [:gregp]]

I just noticed that this report is about a Firefox installation on Linux. Tab unloading on Linux was disabled by default in bug 1780058.

I used to see this behaviour on older versions of Firefox, because I would visit tabs I had last used only an hour ago and they would have to reload as they had been discarded. I never used to run out of memory on those versions

Please try enabling browser.tabs.unloadOnLowMemory in about:config.

Comment 9

[Adam Nielsen]

I have enabled browser.tabs.unloadOnLowMemory and I'll report back what happens in a few days. However will this unload old tabs after some amount of time, or will it wait until system memory is low before doing anything? It sounds like it won't remove tabs until it's too late.

I noticed when changing the option, there's also browser.tabs.min_inactive_duration_before_unload. How does this work? If the default 600000 value is in milliseconds then it suggests tabs can be discarded after 10 minutes, but it doesn't quite explain how they can remain in memory, unused, for over a week.

So it sounds like even if unloadOnLowMemory does work, it would still be useful to have those two additional options to fine tune the number of running processes.

Comment 10

[Andrew McCreight [:mccr8]]

The memory report has 160 processes. The main process has 1.3GB of heap-unclassified, which is quite a bit, but it is possible that there's an immense amount of IPC overhead with that many processes, as I know we don't have memory reporters for that.

I think there are addons that will do more aggressive tab unloading for you. Maybe something there would help.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mstange, could you have a look please?

For more information, please visit BugBot documentation.