1798074 - High commit, leading to system instability

Status: UNCONFIRMED

(Core :: Graphics, defect, P1)

Description

[jkaelin]

Attachment: memory-report.json.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

Browse web with lots of tabs.

Actual results:

System hit the upper limit of commit at 47.9GB out of 48GB and starts refusing allocation requests.

Expected results:

Firefox releases commit

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:glandium, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[Mike Hommey [:glandium]]

There's only about 16.7GB of heap-mapped in the attached report, + about 1.5GB associated with the GPU, so I'm not sure where the extra 29GB used would be coming from. It would seem plausible that there could be some leak in on the GPU driver side. Moving this over to graphics for possibly more digging.

Comment 3

[jkaelin]

Attachment: Firefox 120.0.1 anonymized memory report

Current memory report. Still seems to be an issue, since I have to restart the browser occasionally when the swapping gets too bad. Currently at 55GB/102GB commit, and I only have 32GB RAM.

Some of the anonymized processes are just Youtube. I also run the NoScript and uBlock Origins extensions.

webIsolated=https://youtube.com (pid 7884)
─2,095.79 MB (00.00%) ++ commit

977.73 MB (100.0%) -- heap-committed
├──916.12 MB (93.70%) ── allocated
└───61.60 MB (06.30%) ── overhead

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:bhood, could you have a look please?

For more information, please visit BugBot documentation.

Comment 5

[Bob Hood [:bhood]]

Dropping into Graphics triage for a team look-see.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:bhood, could you have a look please?

For more information, please visit BugBot documentation.

Comment 7

[Bob Hood [:bhood]]

Can you launch Task manager (taskmgr.exe) on your machine and give us the result (as a screen shot) of the memory consumption of all the Firefox processes when it is consuming all this RAM? Please make sure you enable all of the memory column types (View -> Select Columns) so we get values for all of the following in the capture:

• Memory - Working Set
• Memory - Peak Working Set
• Memory - Working Set Delta
• Memory - Private Working Set (probably already selected)
• Memory - Commit Size
• Memory - Paged Pool
• Memory Non-paged Pool

Comment 8

[jkaelin]

Attachment: firefox90of102CommitCapture.PNG

I recently closed out all my tabs using the OneTab extension, so attempting to recreate it by loading ~800 tabs back up.

Basically turned my computer into a slideshow with all the swapping at 90GB/102GB commit. But didn't hit the commit limit this time(would have hit my old commit limit of 48GB and started crashing things).

File > Exit took about 20 minutes after taking the screenshot.

Then the computer is at 22GB/102GB commit when Restore previous session is used. (Firefox at about 800MB RAM, 1.3GB commit with the 800 tabs present but unloaded.)

Comment 9

[Kelsey Gilbert [:jgilbert]]

Can you add the "Dedicated GPU memory" and "Shared GPU memory" columns and give us another screenshot?
Columns can be added by right-clicking one of the existing column headings.

Comment 10

[jkaelin]

Attachment: firefox41of102CommitCapture.PNG

I haven't completely blown up the PC this browsing session, so the 800+ tabs are mostly unloaded.

Comment 11

[Ashley Hale [:ahale]]

Attachment: Shared GPU memory with lots of tabs on a dev machine

On one of my dev machines I can see that having a lot of tabs and windows open leads to a rather alarming amount of Shared GPU memory (66GiB) on a long-running Firefox Nightly process, attaching it for comparison. That is slightly more memory than this machine physically has.

Comment 12

[Ashley Hale [:ahale]]

Attachment: memory report for lots of tabs and windows on a dev machine

Here is the about:memory report for that dev machine, only 2.0GiB commit, so I don't think I'm encountering the same issue here.

Comment 13

[Ashley Hale [:ahale]]

I can get a decent increase in commit by playing a youtube video and a twitch stream at the same time (+300MiB) along with +3GiB of shared GPU memory reported on Task Manager - Details mode.

My guess is that ublock origin may be causing youtube to leak js objects for some RPCs that never complete, as I have seen that happen before, which can cause a rather rapid increase in commit I think, though I can't repro that leak when I try currently, it may be an issue of that nature.

Troubleshooting - you might want to try toggling it off only for youtube and see what happens to commit, as it may be that issue, if so at least it can be narrowed down a bit.

Comment 14

[Ashley Hale [:ahale]]

One more idea - if you close some of the tabs (or windows with lots of tabs), does the commit go down reasonably or does it stay high?

I just closed a bunch of tabs and my commit went from 2.0GiB to 1.3GiB, which is approximately what I expected to happen, but if there is a significant drop when closing certain tabs it may be that those tabs are doing something we should be testing for.

Oddly when closing a bunch of tabs and windows, the GPU Shared Memory did not go down in the task manager Details view, still at 66.9GiB, but the total GPU shared memory usage reported in the Performance tab of task manager is only 2.1/31.9GiB, so there is something odd about how the Details view counts GPU memory for processes.

Comment 15

[jkaelin]

Commit does go down as I close active tabs, yes.

Disabling uBlock Origin and closing Youtube tabs does seem to release commit from the GPU process at least. Was seeing drop of 24MB vs 1MB without disabling.

Comment 16

[Glenn Watson [:gw]]

Bumping to S2 as this is a major issue and we don't know how many users are affected.

Comment 17

[Ashley Hale [:ahale]]

I'm still trying to figure out a way to repro this and dig into the root cause.

Comment 18

[Bob Hood [:bhood]]

Needs a summary here, Ashley.

Comment 19

[Ashley Hale [:ahale]]

Can you go to about:support and copy the text (either of the two buttons at the top) and paste in an attachment on this bug?

I see RadeonSoftware in the task manager screenshot so I assume you have an AMD GPU, I've been investigating a decoder memory leak on AMD 24.* drivers in https://bugzilla.mozilla.org/show_bug.cgi?id=1847453#c47 so this may be a duplicate of that bug, but it would help to know what driver you are using.

I'll probably put a summary on that bug rather than this bug as the investigation there has been much more fruitful.

Comment 20

[jkaelin]

Attachment: about:support

Requested by :ahale

Comment 21

[xse]

Firefox 126.0 (Build ID 20240509170740) mozilla-MSIX Windows 11
AMD Ryzen 7 PRO 6850U with Radeon Graphics

Firefox open for a while, browsing, watching videos when computer goes

"You're running low on disk space"

gpu process sitting at over 20GB reserved memory -> about:memory screenshot

Comment 22

[Ashley Hale [:ahale]]

(In reply to jkaelin from comment #20)

Given that the D3D driver version shows up as 31.0.22023.1014 in that about:support (whereas my latest system shows 31.0.24033.1003), I am assuming that is 22.* driver series which didn't seem to have the memory leak problem on decoder devices when I tested it at one point, that said it is also Ryzen PRO graphics whereas mine is regular Ryzen graphics, so the drivers may behave differently.

I am still wondering if this memory usage is a result of that leak I described in https://bugzilla.mozilla.org/show_bug.cgi?id=1847453#c47 however.

Comment 23

[Ashley Hale [:ahale]]

(In reply to xse from comment #21)

Firefox 126.0 (Build ID 20240509170740) mozilla-MSIX Windows 11
AMD Ryzen 7 PRO 6850U with Radeon Graphics

Firefox open for a while, browsing, watching videos when computer goes

"You're running low on disk space"

gpu process sitting at over 20GB reserved memory -> about:memory screenshot

That roughly matches what I saw in https://bugzilla.mozilla.org/show_bug.cgi?id=1847453#c47 , so it is probably the same issue with decoder devices.

Comment 24

[jkaelin]

(In reply to Ashley Hale [:ahale] from comment #22)

(In reply to jkaelin from comment #20)

Given that the D3D driver version shows up as 31.0.22023.1014 in that about:support (whereas my latest system shows 31.0.24033.1003), I am assuming that is 22.* driver series which didn't seem to have the memory leak problem on decoder devices when I tested it at one point, that said it is also Ryzen PRO graphics whereas mine is regular Ryzen graphics, so the drivers may behave differently.

I am still wondering if this memory usage is a result of that leak I described in https://bugzilla.mozilla.org/show_bug.cgi?id=1847453#c47 however.

Driver is Adrenaline 23.11.1, and my GPU is a Radeon RX 6600 8GB, on Windows 10.

https://www.amd.com/en/resources/support-articles/release-notes/RN-RAD-WIN-23-11-1.html

Comment 25

[Ashley Hale [:ahale]]

NeedInfo - Can you take a look at the reporting guide I wrote in https://bugzilla.mozilla.org/show_bug.cgi?id=1902566#c0 and see if this memory leak is repeatable (i.e. gets worse each time you do the actions that trigger it)?

So far we're keeping track of a decoder device leak in https://bugzilla.mozilla.org/show_bug.cgi?id=1847453 but I am not sure if this is the same issue so it would be extremely useful to know if it's a different root cause that we need to track down.

Comment 26

[jkaelin]

Attachment: dec2024memory-report.json.gz

The bulk of the commit isn't tied up by the GPU process, it seems to be tied up by processes attached to youtube.com. Which probably tracks with orphan DOM nodes from Ublock Origin.

Possibly related, when I Restore Previous Session, it seems to be trying to load up ALL the tabs at the same time. Which leads to me getting NoScript XSS redirect notifications about "possible DDOS attempt?" from sites being hit. This is also when commit tends to spike highest, and has led to crashing out as I run out of pagefile.sys.

Steps to reproduce are probably just keep opening Youtube tabs and watching videos. Reddit may do it too, with their in-line video player.

Firefox 133.0
Ublock Origin 1.61.2
NoScript 11.5.2

Comment 27

[Bob Hood [:bhood]]

13 Feb Triage: Ashley will try to spend some time on this.

Comment 28

[Ashley Hale [:ahale]]

(In reply to jkaelin from comment #26)

Created attachment 9444230 [details]
dec2024memory-report.json.gz

The bulk of the commit isn't tied up by the GPU process, it seems to be tied up by processes attached to youtube.com. Which probably tracks with orphan DOM nodes from Ublock Origin.

In your memory report I see an extension called "MyJDownloader Browser Extension", if that is a video downloader I would say it is way more likely to be the culprit than uBlock Origin or NoScript, I've seen other video downloader extensions cause memory leaks on YouTube in past reports.

Comment 29

[jkaelin]

Alright, I'll try disabling that extension. But it's not a video downloader, it's primary use is for completing captchas from the JDownloader app when it tries to download things from various hosting sites. JDownloader can be used to download videos from Youtube, but it does that app-side.

DownThemAll is a downloader extension, but I don't think that one is particularly active on Youtube, but I'll disable it as well for now. (I don't use either of those extensions regularly, so no great loss.)

My windows install got nuked 2 weeks ago, so I'm still recovering things from that. And Windows Update is still restarting me more than I like, so I haven't been able to build up the sort of browser tab load that manifests the bug yet.

Comment 30

[Ashley Hale [:ahale]]

It still could be uBlock Origin or NoScript, if the others are not likely, I'd lean towards it being NoScript (which is a great extension, but it would be unsurprising if YouTube was not testing for issues with this extension in their development efforts).

It's also possible that the cause of this memory usage has changed over time, as other bugs have come and gone with regards to video playback memory leaks.

In terms of debugging this, we're thinking about how to do better attribution of memory usage to script from the website vs browser extensions which would make situations like this easier to debug at some point in the future.

Have you had any better luck with this issue on your new Windows install?

Comment 31

[jkaelin]

Attachment: March2025memory-report.json.gz

The new install hasn't really improved the situation that much, but I did port over my old Profile almost completely.

Comment 32

[jkaelin]

Attachment: firefox41of53CommitCapture.PNG

The GPU process (32004) is still the largest offender.

Comment 33

[jkaelin]

Attachment: 41of53FirefoxCaptureRedux.PNG

Sorry, forgot to add back the GPU memory tabs.

Comment 34

[Ashley Hale [:ahale]]

How many videos are open? In comment #33 I see 3.3GiB of dedicated gpu memory which seems normal if a bunch of videos are open, abnormal if it's only one or a few, and could be a memory leak if it increases over time with regular usage (e.g. opening and closing video tabs repeatedly).

In about:memory in comment #31 I see the first listed isolated web content process is using 8GiB of system memory, of which 4.39GiB is js-realm which I believe is scripts (so could be youtube and interactions with extensions), 8GiB of memory usage is a lot, is this increasing or stays roughly stable in normal usage?

I'm most interested in memory leaks as resolving those has a much bigger impact than optimizing the browser's memory usage (which we already put effort into), so if either of these numbers increase it would be helpful to get about:memory reports at different points in time - the about:memory page can load two reports and diff them to see what changed, which would be the biggest lead.

Comment 35

[jkaelin]

Attachment: firefox15.4of72CommitCapture.png

Hard to say. That browser session got nuked the other day when I restarted twice without restoring session after the first restart. But I think that there were a large amount of video tabs open, and that the GPU usage would have been a fair amount for the number of them. (cached video content on already watched videos)

The amount of commit used creeps up over time, but my amount of tabs open also creeps up over time. So I don't know if that's linear growth or if a memory leak is making it worse.

With a basically fresh session from today, with 3 tabs open, the total Commit in Task Manager is 15.4GB used out of 72GB available.

Comment 36

[jkaelin]

Attachment: Mar17memory-report.json.gz

3 tabs, basically fresh session, 1 video tab open

Comment 37

[jkaelin]

Attachment: Mar18memory-report.json.gz

Watched a few Youtube videos, opened a couple more tabs (2x Reddit, 1x Fandom). Commit is up by 232.93 MB since last report.

Comment 38

[Ashley Hale [:ahale]]

NeedInfo - Bug routing question; the memory report in comment #1 seems like YouTube is using an order of magnitude more memory in JavaScript than I'd reasonably expect here for a single tab, an 8GiB Web Isolated container with about 4.3GiB of js-realm that is about a quarter full - can someone more familiar with JS memory usage take a look?

Comment 39

[jkaelin]

Attachment: March21_memory-report.json.gz

Alright, had some time for this session to cook.

And I'll de-anonymize one of the Youtube links, 651MB for a 2.5 minute video is probably excessive.

1,768.05 MB (100.0%) -- explicit
├──1,264.94 MB (71.54%) -- window-objects
│ ├────651.45 MB (36.85%) -- top(https://www.youtube.com/watch?v=GaBtPPxKls8, id=2908)
│ │ ├──576.73 MB (32.62%) -- active
│ │ │ ├──576.35 MB (32.60%) -- window(https://www.youtube.com/watch?v=GaBtPPxKls8)
│ │ │ │ ├──465.54 MB (26.33%) -- js-realm(https://www.youtube.com/)
│ │ │ │ │ ├──457.12 MB (25.85%) -- classes
│ │ │ │ │ │ ├──120.12 MB (06.79%) -- class(Object)/objects
│ │ │ │ │ │ │ ├───90.53 MB (05.12%) ── gc-heap
│ │ │ │ │ │ │ └───29.59 MB (01.67%) -- gc-buffers
│ │ │ │ │ │ │ ├──25.95 MB (01.47%) ── slots
│ │ │ │ │ │ │ └───3.64 MB (00.21%) ── elements/normal
│ │ │ │ │ │ ├──111.85 MB (06.33%) -- class(Function)/objects
│ │ │ │ │ │ │ ├──102.89 MB (05.82%) ── gc-heap
│ │ │ │ │ │ │ └────8.96 MB (00.51%) ── gc-buffers/slots
│ │ │ │ │ │ ├───96.03 MB (05.43%) -- class(Call)/objects
│ │ │ │ │ │ │ ├──95.59 MB (05.41%) ── gc-heap
│ │ │ │ │ │ │ └───0.44 MB (00.02%) ── gc-buffers/slots
│ │ │ │ │ │ ├───88.98 MB (05.03%) -- class(Array)/objects
│ │ │ │ │ │ │ ├──66.46 MB (03.76%) ── gc-heap
│ │ │ │ │ │ │ └──22.52 MB (01.27%) -- gc-buffers
│ │ │ │ │ │ │ ├──22.30 MB (01.26%) ── elements/normal
│ │ │ │ │ │ │ └───0.23 MB (00.01%) ── slots
│ │ │ │ │ │ └───40.13 MB (02.27%) ++ (33 tiny)
│ │ │ │ │ └────8.42 MB (00.48%) ++ (8 tiny)
│ │ │ │ ├───67.98 MB (03.85%) -- dom
│ │ │ │ │ ├──47.78 MB (02.70%) ── orphan-nodes
│ │ │ │ │ └──20.20 MB (01.14%) ++ (7 tiny)
│ │ │ │ ├───42.73 MB (02.42%) ++ layout
│ │ │ │ └────0.09 MB (00.01%) ── property-tables
│ │ │ └────0.38 MB (00.02%) ++ (2 tiny)
│ │ └───74.72 MB (04.23%) -- js-zone(0x2ca1e611400)
│ │ ├──46.19 MB (02.61%) ++ (17 tiny)
│ │ └──28.52 MB (01.61%) -- strings
│ │ ├──24.33 MB (01.38%) ++ string(<non-notable strings>)
│ │ └───4.19 MB (00.24%) ++ (110 tiny)

Comment 40

[Steven DeTar [:sdetar]]

Jon, would you be able help Ashley in comment 38.

Comment 41

[Jon Coppeard (:jonco)]

There's no smoking gun here. Youtube is using more memory than we'd hope for but 465MB of JS as per comment 39 is not wildly excessive. Likely a leak in the page.

Comment 42

[Bob Hood [:bhood]]

Flagging as stalled because are awaiting more available cycles.

Comment 43

[jkaelin]

Attachment: Firefox90of118CommitCapture.png

About to restart Firefox for an update, current version 137.0.1, but figured I'd drop an update on the commit. Since I'm running about 6 weeks of uptime currently.

The 512GB of Dedicated GPU memory is certainly weird. Especially since I only have a 8GB AMD RX 6600.

PID 6824 doesn't show up in the about:memory report aside from a small mention in queued-ipc-messages:

0 (100.0%) -- queued-ipc-messages
├──0 (100.0%) ── content-parent(Browser, pid=-1, closed channel, 0x248fed25620, refcnt=2)
├──0 (100.0%) ── content-parent(Browser, pid=-1, closed channel, 0x24913b75320, refcnt=1)
<snip about 40 lines>
├──0 (100.0%) ── content-parent(Browser, pid=6824, open channel, 0x24900fd2a20, refcnt=47)