1624131 - Firefox hits 5.3GB memory cap and then cannot function

Status: NEW

(Firefox :: Session Restore, defect, P5)

Description

[ghost53947]

Attachment: memory-report.json.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

I have roughly 9k tabs suspended open. When opening firefox and it it seems to reach a memory limit of 5.3GB which then makes that firefox stops functioning completely, the windows no longer render correctly and it does not respond, if you bring the tab count down to get under that 5.3GB memory limit to about 5.1GB then it can be used again with no issues.

Actual results:

As soon as firefox hit 5300MB it stops rendering pages and stops responding properly. I have attached a memory snap shot at the time when it was misbehaving and then the screenshots of task manager after doing clear history.

Expected results:

It should just continue to work without this behaviour

Comment 1

[ghost53947]

Attachment: Memory-usage.PNG

Comment 2

[ghost53947]

Exiting firefox also becomes a problem, memory spikes to 5474536K and it causes a crash after some time.

Comment 3

[Augusto Pace]

Hi,

Unfortunately I'm not able to reproduce this issue on Windows 10. It's hard for us to reproduce this kind of issues since it could depend on your computer specs.
Anyway I'm setting component to Core - Memory Allocator for someone to take a look at this.

Comment 4

[Mike Hommey [:glandium]]

│ │ │ ├──4,063.04 MB (78.20%) -- realm([System Principal], shared JSM global)

Comment 5

[Nicolas B. Pierron [:nbp]]

My understanding would be that this would be caused by a limitation with the JS engine, which would be limited in the space available for one of its data structures. However, from the memory report my guess that would be that we have a single large Array.

Jon, is there anything obvious from the memory report?
Otherwise I think this bug belongs to the "Tabbed Browser" component, as a question of why do they need such large entries per tab.

Comment 6

[ghost53947]

Just to add to this, while it's stuck in hitting that memory limit. Firefox shows no addons. Even though the icons are shown, when going to the addons settings it shows there are non installed. In safe mode this page does show the addons.

Comment 7

[ghost53947]

Hi all,

So with regards to testing, I am more than happy to test, even build and debug the browser. My machine has 32GB of ram if that helps anyone (so memory consumption is not an issue). I do realize I have an excessive amount of tabs, but firefox's discard tab function works beautifully. I am trying to work through and process all the tabs, but when I can't open it, it is an issue. Safe mode does seem to load properly. From what I have observed is that it seems there is something that runs into a memory cap and then blocks the browser from fully initializing. Thus no extensions get properly loaded. Once that happens the memory drops sharply by 2GB in my case. The page render also halts and seems to be stuck with the tabs just infinitely loading.

I have considered using the 'Refresh' function in FF, but currently with this bug still in play when I tried that, it did not restore any tabs, I wasn't even prompted to restore any tabs when I tried that option, I restored my profile from a backup.

One thing I have noticed and this might point to where the issue is, when I am stuck and cannot properly open FF and restore it, I have gone using FF nightly and a plugin to edit the sessionstore.jsonlz4 and yanked out about 14k lines from the file and saved it. Then FF does seem to manage to start since the memory stays under 5.1GB and the browser can properly restore. Later on once I have closed about 200 tabs I will open that file again and restore the lines I yanked out, and then continue. The current size of this file is 135,569KB.

Please note that when the browser crashes it also leaves about a lot of .sqlite-sal and .sqlite-shm files lying around. Which also need to be opened with a sql lite browser and properly closed.

Using all that is mentioned above I am able to then start the browser fully, and but once closed I need to repeat the .sqlite file cleanup again.

Comment 8

[ghost53947]

To add: In safe mode, the memory did go up to over all 5.3GB to 5.4GB (at its highest 5.6GB while trying to use the Minimize memory usage) use. I then went to about:memory, did a 'Clear recent history -> Everything, CC, GC and then Minimize memory usageand the memory dropped to 1,799.2MB over all usage. I took a snap shot before seesafe-mode-before-memory-report.json.gzand afterafter-memory-report.jszon.gz`. So something drastically happens here.

Comment 9

[ghost53947]

Attachment: safe-mode-before-memory-report.json.gz

Safe mode memory usage before any free memory functions are used.

Comment 10

[ghost53947]

Attachment: safe-mode-after-memory-report.json.gz

Safe mode memory snapshot after Free memory function

Comment 11

[Jon Coppeard (:jonco)]

(In reply to Nicolas B. Pierron [:nbp] from comment #5)
Yes, it looks like we're hitting the 4GB JS heap limit. Maybe there's something the browser can do to reduce the amount of data it needs per-tab.

(On a tangent, I wonder if we should stop opening new tabs at some point?)

Comment 12

[ghost53947]

Well thus far, reducing the size of the sessionstore.jsonlz4 has actually made it possible for me to use the browser again. As far as I understood this is used to restore the tabs, so maybe it's not just the JS head limit, but actually due to trying to fully parse this massive file, and convert that back to data structures, but most tabs are in a suspended state on restore.

Comment 13

[swleefers]

I experienced an issue a few months ago that might be the same thing, with Firefox 65, 32-bits.

I had far fewer tabs open, only 17 to 20. At some point, several extensions would be silently dropped from memory (they no longer showed up in the list in about:performance, and they no longer affected any pages). I only noticed this because, during my session, I would suddenly see advertisements on websites, which ought to have been blocked by my extensions. And Lastpass stopped working. Etc. Then I saw other extensions had been dropped as well in about:performance. This happened on a regular basis, I thought perhaps when some Firefox process exceeded the 32-bit memory limit during a session. I really wasn't doing anything weird, and it was a fairly new Firefox profile on a new computer (Win 10). Restarting the browser temporarily solved the issue, until it would happen again.

Switching to Firefox 64-bits solved the issue for me, which is why I thought it was a memory issue. I was going to post a bug at the time, but I forgot.

Comment 14

[ghost53947]

So I tried to do a refresh which did succeed, but then after closing the browser and it crashing again, it went and deleted my whole previous session , 10456 tabs lost. This is really not cool. Luckily I had a backup of my sessionstore.jsonlz4 file. Can someone maybe just point me where in the code I could possibly start searching, as I realize not many users will have this issue, so I can solve it?

Comment 15

[ghost53947]

Could it possibly be there chrome://browser/content/tabbrowser-tab.js:231 causing this ?

Comment 16

[Dão Gottwald [:dao]]

(In reply to ghost53947 from comment #15)

Could it possibly be there chrome://browser/content/tabbrowser-tab.js:231 causing this ?

Not if you mean this line: https://hg.mozilla.org/mozilla-central/annotate/4c47508565732b2db909f265e222a9355ee6a810/browser/base/content/tabbrowser-tab.js#l231

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:dao, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 19

[mirh]

(In reply to swleefers from comment #13)

This happened on a regular basis, I thought perhaps when some Firefox process exceeded the 32-bit memory limit during a session. I really wasn't doing anything weird, and it was a fairly new Firefox profile on a new computer (Win 10). Restarting the browser temporarily solved the issue, until it would happen again.

Switching to Firefox 64-bits solved the issue for me, which is why I thought it was a memory issue. I was going to post a bug at the time, but I forgot.

Could report the same thing in bug 1749138
On the other hand, both OP and bug 1834379 are reportedly using the x64 version.. so there must be something else going on.

p.s. bug 1464685 comment 9 has an interesting script to quickly bloat session store