1931717 - [meta] High OOM rate and CC time in YouTube

Status: NEW

(Core :: Performance: Memory, defect)

Description

[Kagami Rosylight [:saschanaz] (they/them)]

We closed bug 1923864 with no further action concluding it's an actual OOM, but it seems the youtube OOM rate itself went high enough to bother users in the wild, given user report e.g. https://old.reddit.com/r/firefox/comments/1grh0mk/youtube_freezes_in_livestreams_since_132x/ . We should investigate why that's the case and how to mitigate that.

Comment 1

[Kagami Rosylight [:saschanaz] (they/them)]

Note that the reddit post says it started with FF132, but the crash report in bug 1923864 says it went higher on all channels at once on October 2024, and reports include ESR115. So it's either:

1. Something we uplifted all the way to to ESR115 in October 2024 is causing high memory usage (did we?)
2. Some YouTube change caused high memory usage by triggering an existing issue

Comment 2

[cosplayking]

I posted the Reddit thread and just noticed this was opened on Bugzilla.

To add to this:
I've had this stream open for about 3 hours: https://www.youtube.com/watch?v=Q7EbjfGS-jg - Experienced zero issues until it ended.

Yesterday/2days ago however I experienced a crash while having this livestream open: https://www.youtube.com/watch?v=R27v3IS7muc
**Related crash report: **https://crash-stats.mozilla.org/report/index/f70e7464-99e2-4ea8-b862-5f7210241115

Thus far it has only happened during livestreams and not regular videos, albeit the regular videos tend to be about 30 minutes max and mostly below that, so not sure if it may occur at random. So far it hasn't though.

I also haven't tried other streaming websites such as Twitch as I don't generally watch streams on Twitch or others.

Extensions don't seem to the cause of it either as other bug reports show people with barely if any extensions or very different non-Youtube related extensions:

Example of one with no addons:
https://crash-stats.mozilla.org/report/index/39d79629-fc2a-4a7c-b23e-0e1af0241115#tab-extensions

I thought maybe it is my 'dreaded' CPU, but it also seems to happen to MAC devices and AMD CPU's/GPU's, crashes also only happen on Youtube livestreams since a version 132.x of Firefox, or it simply coincides with changes in the background happening on Youtube as you mentioned.

I do notice the first stream is about 11 hours long, and I started watching it I think somewhere midway through... The chat is not heavily active so I doubt it has anything to do with there being too much chatting going on, causing the crash.

Unfortunately I don't know the duration of the other streams that caused crashing, it may have been a good pointer if it only happens on streams after a specific duration point.

I'll keep track to see if it happens again, note the stream URL and at least the duration of the video.

Comment 3

[Kagami Rosylight [:saschanaz] (they/them)]

Hello cosplayking, thank for the report and the post!

Would you mind trying this steps, when you watch another live stream:

1. After 30min-ish, open about:memory in Firefox address bar
2. Check "anonymize" in "Save memory reports" and click "Measure and save..."
3. Either attach the generated file here or send it to email krosylight at mozilla.com

With that we would be able to have some idea whether the memory use is actually increasing and how.

Comment 4

[cosplayking]

Attachment: memory-report.json.gz

As requested

Comment 5

[Kagami Rosylight [:saschanaz] (they/them)]

(It doesn't seem to happen with a long video, but only with actually live videos, from the current observation. So retrying with the recording of the previous live streams may not be helpful. Refer to https://www.youtube.com/watch?v=u4GlN6NtvIM when anyone else tries debugging on a live stream, which has been open for years.)

Comment 6

[Kagami Rosylight [:saschanaz] (they/them)]

(As discussed internally)

Comment 7

[cosplayking]

I just had the following stream freeze (surprisingly it happens with the same person quite often, so not exactly sure what the deal is):
https://www.youtube.com/watch?v=pzRTGcnXabA

**One thing I noticed it was trashing my disk with 120+ MB/s
https://i.imgur.com/U89IfBf.png

The tab didn't crash but is still in a frozen state after trying to click developer tools:
https://i.imgur.com/IitSaiS.png

I did the profiler thing in about:performance which automatically finishes within 5 seconds, not sure if this is as intended and if it's any useful information:
https://i.imgur.com/fpkZFPG.png
https://i.imgur.com/L19yEqC.png
https://i.imgur.com/o2lYkoK.png

Strangely enough the Youtube link of the stream it refers to is from one I watched earlier today:
https://www.youtube.com/watch?v=2mQBNqqfnHA

Will add a new memory report as well.

Comment 8

[cosplayking]

Attachment: memory-report2.json.gz

Comment 9

[Kagami Rosylight [:saschanaz] (they/them)]

Do you still have the profile tab open? Can you click the upload button at the top right and share the link?

Comment 10

[cosplayking]

(In reply to Kagami Rosylight [:saschanaz] (they/them) from comment #9)

Do you still have the profile tab open? Can you click the upload button at the top right and share the link?

https://share.firefox.dev/4fNjxP7

Comment 11

[cosplayking]

Had both open forgot to add the 2nd one:
https://share.firefox.dev/3ZaoMCp

Comment 12

[Kagami Rosylight [:saschanaz] (they/them)]

80sec jank when doing cycle collection? That sounds interesting, but unfortunately not much is recorded in either of the perf profiles.

Comment 13

[Matthew Gaudet (he/him) [:mgaudet]]

That second profile is a just profiling js::DumpHeap; I suspect an issue where a memory dump was requested, then a profile started.

Comment 14

[Kagami Rosylight [:saschanaz] (they/them)]

So I was running https://www.youtube.com/watch?v=u4GlN6NtvIM for 18 hours with the profiler on (by accident 😅, and somehow the stream just stopped for me while still being live), and although the profiler only actually recorded the last few minutes because of buffer limit, it did record that the memory usage increases without even playing any video.

https://share.firefox.dev/4fzmmmr

Comment 15

[cosplayking]

(In reply to Kagami Rosylight [:saschanaz] (they/them) from comment #14)

So I was running https://www.youtube.com/watch?v=u4GlN6NtvIM for 18 hours with the profiler on (by accident 😅, and somehow the stream just stopped for me while still being live), and although the profiler only actually recorded the last few minutes because of buffer limit, it did record that the memory usage increases without even playing any video.

https://share.firefox.dev/4fzmmmr

That is what makes this issue so frustrating to troubleshoot as it occurs so randomly, you can watch something for hours on end without issue, or even a multitude of streams without issue... and then poof out of nowhere a tab crashes OR it just happens within an hour in some random stream you had open.

For example just now I had random streams open for about 6~ hours today before the first freeze started happening.

Just had the following stream freeze:
https://www.youtube.com/watch?v=ArYStZVyIDM

The stream froze and I pressed CTRL + F5 to refresh it, as a standard refresh did nothing and no Tab crash was triggered.
Page is however stuck on the following screen: https://i.imgur.com/D6MwSyc.png

**Profiler: **https://share.firefox.dev/494DUEQ

Memory usage jumped around between 8-15GB, I think the profiler is the one that is around 13GB usage mark. The Memory report I will add is after it jumped to 15GB.

Task manager screenshot: https://i.imgur.com/6sPiWfn.png

Comment 16

[cosplayking]

Attachment: memory-report3.json.gz

Comment 17

[Markus Stange [:mstange]]

Shot in the dark: Do you have the extension "Return YouTube Dislike" installed? This extension was the culprit in bug 1920229, which was another report of a performance problem on YouTube.

Comment 18

[cosplayking]

(In reply to Markus Stange [:mstange] from comment #17)

Shot in the dark: Do you have the extension "Return YouTube Dislike" installed? This extension was the culprit in bug 1920229, which was another report of a performance problem on YouTube.

No, never used it either.
Have the following extensions:
https://i.imgur.com/aGVsYB0.png

I do have a few scripts running in Violentmonkey:
https://greasyfork.org/en/scripts/11057-block-youtube-users
https://greasyfork.org/en/scripts/13040-youtube-hide-watched-videos
https://greasyfork.org/en/scripts/423851-simple-youtube-age-restriction-bypass
https://greasyfork.org/en/scripts/498145-youtube-hd-premium
https://greasyfork.org/en/scripts/439993-youtube-shorts-redirect

Not sure if the profiler also adds the extensions + settings used, but it happened to Kagami as well albeit after a long time in a single stream. Fortunately it doesn't happen all the time as that would drive me crazy. The problem however, due to it occurring at random moments makes it harder to troubleshoot I figure. If it were an extension or userscript you would think it would happen a lot more frequently, as these would be running all the time.

Comment 19

[Markus Stange [:mstange]]

Olli, what can we do to diagnose this problem with intermittently long CC times?

Comment 20

[Markus Stange [:mstange]]

Ok so I think there are two options here:

1. The long CC times are caused by the gradual memory growth shown in Kagami's profile from 14. If this is the case, we may be able to debug it locally.
2. Or the memory growth is unrelated. The long CC times could be due to some other difference in environments.

The add-ons and user scripts could certainly be implicated. But I acknowledge that it's hard to narrow this down if the problem only appears intermittently and only after running the browser for a long time.

Comment 21

[Kagami Rosylight [:saschanaz] (they/them)]

(Comment #14 was recorded with a fresh profile with no addons)

Comment 22

[cosplayking]

(In reply to Markus Stange [:mstange] from comment #20)

Ok so I think there are two options here:

1. The long CC times are caused by the gradual memory growth shown in Kagami's profile from 14. If this is the case, we may be able to debug it locally.
2. Or the memory growth is unrelated. The long CC times could be due to some other difference in environments.

The add-ons and user scripts could certainly be implicated. But I acknowledge that it's hard to narrow this down if the problem only appears intermittently and only after running the browser for a long time.

Turned off all addons except for uBlock Origin and just had a stream freeze:
https://www.youtube.com/watch?v=I3vpoLCKrgA

Profiler: https://share.firefox.dev/4fxntmU

Will add the memory report as well.

Comment 23

[cosplayking]

Attachment: memory-report4.json.gz

Comment 24

[Markus Stange [:mstange]]

Thank you. The memory report from comment 23 has 5GB of detached windows in the Isolated Web Content (pid 19336) process:

5,685.33 MB (100.0%) -- explicit
├──5,193.50 MB (91.35%) -- window-objects
│  ├──4,991.92 MB (87.80%) -- top(none)/detached
│  │  ├──4,990.56 MB (87.78%) -- window(<anonymized-1582695448606>)
│  │  │  ├──4,983.33 MB (87.65%) -- js-realm(<anonymized-8>)

Olli, what does it mean to have detached windows?

Comment 25

[Andrew McCreight [:mccr8]]

IIRC a detached window means the window is not currently being displayed. Kind of weird for the page to be leaking a gigantic window like that (I think the lack of a bracketed number means there's not a bunch of windows, which is what I'd usually expect for a leak this large). Disclaimer: it could be our leak and not the page.

Comment 26

[Olli Pettay [:smaug][[e-mail]]]

And 'not being displayed' in this context means for example an iframe which has been removed from DOM, but someone still keeping reference to its contentWindow.

It would be useful to create CC/GC logs from about:memory as soon as one sees a detached window coming from youtube - in case memory usage isn't yet too bad, since otherwise the logs would be massive.

Comment 27

[Olli Pettay [:smaug][[e-mail]]]

The second youtube link from comment 2 creates a detached chat window rather soon, also another (non-detached). So something to do with chat iframe? But the memory use there is quite moderate for me, 40MB. Maybe if there was more chat history it would be larger?

Comment 28

[cosplayking]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #26)

And 'not being displayed' in this context means for example an iframe which has been removed from DOM, but someone still keeping reference to its contentWindow.

It would be useful to create CC/GC logs from about:memory as soon as one sees a detached window coming from youtube - in case memory usage isn't yet too bad, since otherwise the logs would be massive.

I didn't notice any pop-ups or frames opening, I'm also not using the 'Popout chat' option in Youtube. But I don't always have the Youtube stream tab active, as I'm often surfing around while listening to it in the background so I don't always directly notice the freezing happening until I think 'Hey why is it so silent?'

Perhaps I'm completely off-base but could this be triggered by PiP? Personally I have that option disabled though:
media.videocontrols.picture-in-picture.video-toggle.enabled = False

Comment 29

[BugBot [:suhaib / :marco/ :calixte]]

This bug was moved into the Performance component.

:saschanaz, could you make sure the following information is on this bug?

• ✅ For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
• For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
• Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm it is recent.

Thank you.

Comment 30

[cosplayking]

Attachment: about-support.txt

Comment 31

[Kagami Rosylight [:saschanaz] (they/them)]

There are several memory dumps here.

Comment 32

[Olli Pettay [:smaug][[e-mail]]]

I think I can reproduce high memory usage increase if I resize the window so that the chat moves to be below the video and then again increase window size so that chat is again moved to be on the right side. Need to find what is keeping the detached chat window(s) alive. At least some callback.

Comment 33

[Olli Pettay [:smaug][[e-mail]]]

The leak I see locally happens because there is an event listener (for blur event) in the top level Youtube page and it has a reference to a window object which used to be the contentWindow of an iframe, but the iframe has been removed from DOM. The iframe was for the chat feature.
And resizing the window seems to make this worse since the existing iframe is moved around, so I guess a new one is created and old one mostly destroyed.

I see high memory consumption also in Chrome when resizing, though of course there the issue might be something else in case different JS is provided to different browsers.

I've been using https://www.youtube.com/watch?v=R27v3IS7muc (from comment 2) as the testcase.

Comment 34

[Olli Pettay [:smaug][[e-mail]]]

And to be clear, the issue has been reported to Google/Youtube.

Comment 35

[cosplayking]

(In reply to Olli Pettay [:smaug][[e-mail]] from comment #34)

And to be clear, the issue has been reported to Google/Youtube.

Just found this, I have no idea if it works for this specific bug but apparently it's something that supposedly could help:
https://greasyfork.org/en/scripts/473972-youtube-js-engine-tamer

People linked towards it on the following Reddit discussion:
https://www.reddit.com/r/youtube/comments/1c8tcc8/memory_leak_with_livechat/

But these scripts were released about a year ago and I've only noticed this issue since 132.x, perhaps there's something in the script(s) that can be used to fix or temporarily bandage Firefox until Youtube fixes it?

Comment 36

[cosplayking]

For what it's worth I had a tab crash with this userscript active: https://greasyfork.org/en/scripts/473972-youtube-js-engine-tamer

https://crash-stats.mozilla.org/report/index/611e13e2-47bb-499e-a51b-fb8ed0241201

Hard to tell if it mitigates anything at all, is just snake oil or helps against different kinds of issues on Youtube.

Comment 37

[Kagami Rosylight [:saschanaz] (they/them)]

A bunch of different YouTube performance issues in my mailbox, might all have the same root cause.

Comment 38

[Kagami Rosylight [:saschanaz] (they/them)]

This has become effectively a meta bug, so let's mark it as such.

Comment 39

[Filo]

I have a gut feeling that it occurs more often when a live streamer changes a setting on their stream or takes their stream offline. Just now during a livestream I had a tab crash RIGHT after the stream ended.
https://crash-stats.mozilla.org/report/index/5dc87fe3-505e-4422-b79b-015c00250304

Perhaps this also happens when they make other changes to their stream, like changing title/bitrate or other settings you might be able to change during a livestream that affects the layout(?) or status of the broadcast.

Comment 40

[Filo]

3 crashes on this stream within 30 minutes:
https://www.youtube.com/watch?v=CaoPLrs0wQQ

https://crash-stats.mozilla.org/report/index/bp-15db4a1f-d1ae-4a29-967b-d22810250430
https://crash-stats.mozilla.org/report/index/bp-d62ab6ce-4516-401e-8c7d-479160250430
https://crash-stats.mozilla.org/report/index/bp-06be302d-0220-4d68-9825-b16830250430

Is there some clarity yet whether this is:
A: Firefox-specific issue
B: Youtube issue
C: Hardware issue

Comment 41

[Andrew S.]

Wanted to add that I encounter Youtube tabs crashing as well. It happens (as far as I see) only with streams - according to about:processes they start consuming 5-6+ GB of memory and just hang.
Refreshing the tab doesn't seem to help. Only closing it and opening a new one with the same address (restoring the tab in most cases still breaks them).
Here's the Firefox Profiler output for when this happened: https://share.firefox.dev/4dL31yE
Is there anything else needed the next time this happens? Well it is a randomish thing, so will be hard to just sit and wait for it.

Comment 42

[Andrew S.]

Here's another crash: https://share.firefox.dev/4mMjVkN
And the corresponding about:memory output: https://1drv.ms/u/c/2839e5b547e6bafd/EYgREa3IWT5HlyW0dbA2ffIBgrCqcfCOrm842xn0Jag97A?e=SMbyxJ

Comment 43

[Justin Peter]

I've been having regular issues with high memory usage on YT as well for a while. (I think mostly in the GPU process.) I typically do not watch live streams. It seems to take somewhere between a few hours and a few days until I reach the point where the GPU process is using 4+ GB and I kill it or restart the browser.

Am I best off leaving that here, in bug 1945363, or in a new report? What info can I provide that is of most help? Anything beyond a memory report?

At a glance right now, about:processes says the GPU is using 5GB, the main process 1GB, and 6 YT processes (14 tabs) with 400-700MB each.

about:memory shows 250MB in explicit allocations, 6GB commit (5GB private), and 2GB gfx (same number for webrender, 96% of that number for textures).

Comment 44

[Andrew S.]

An update on this matter - YouTube streams keep on crashing constantly after some time while the corresponding process leaks memory. That is on v145 beta.

Comment 45

[Matthew Gaudet (he/him) [:mgaudet]]

Can you link some recent crash reports?

Comment 46

[Andrew S.]

Do you mean the report that is sent automatically after a tab crashes or a profile recording?

Comment 47

[Matthew Gaudet (he/him) [:mgaudet]]

The former; you can find them at about:crashes, and the URL you would give us should look similar to https://crash-stats.mozilla.org/report/index/62248372-078c-4cf4-aef6-ea6190251020

Comment 48

[Andrew S.]

If I'm not mistaken this should be the one: https://crash-stats.mozilla.org/report/index/0307c289-f406-4171-990f-8ad040251102#tab-details

Comment 49

[Matthew Gaudet (he/him) [:mgaudet]]

Thanks Andrew. This is an ongoing issue we've been seeing where it seems like youtube (or twitch) leaks memory and it interacts poorly with our failure to allocate objects after we cross an allocation limit (which is unrelated to your machine's memory capacity). There's some investigation work we want to do regarding this. We don't currently have a great answer here unfortunately.

There are some relevant pieces we're looking into though: Bug 1998745, Bug 1989115 are both diagnostics that help clarify what we're seeing.

Comment 50

[Andrew S.]

Good luck!
If any other reports or profiles are needed, please, tell me.

Comment 51

[Filo]

Had a crash today after upgrading to Firefox 145.0:
https://crash-stats.mozilla.org/report/index/3b8dc940-7292-4f78-9880-230a40251111

I have had the same happen today, at some point a single Youtube tab with a livestream open was using 25GB(!!!) of my 64GB available RAM.
https://profiler.firefox.com/public/aede0v3gd0yetrfydd8bjjq4y223k9ff9twmym8 <-- Hit profiler on the Youtube tab, after a few seconds the RAM usage dropped to 2GB.

Haven't had these issues on 144.0.2 -- It started happening again since 145.0.

I had this issue a few months back on another Youtube version, not sure what exactly I had done or what had happened in the background but haven't experienced it since, until I updated to Firefox 145.0

Comment 52

[Elle Solomina (she/her)]

Maybe this setting helps you:

		"javascript.options.mem.gc_parallel_marking": {
			"Value": false,
			"Status": "locked"
		},
		"javascript.options.mem.gc_max_parallel_marking_threads": {
			"Value": 1,
			"Status": "locked"
		},
		"javascript.options.mem.gc_max_helper_threads": {
			"Value": 1,
			"Status": "locked"
		},

Furthermore, I highly recommend disabling all Spectre protection because it ruins optimizations.

Comment 53

[Andrew S.]

If disabling something like Spectre protection makes an application leak memory, then it is the application fault.

Comment 54

[Andrew McCreight [:mccr8]]

The Spectre JIT changes have already been disabled on platforms where we are shipping site isolation (eg on desktop) and yes it does not have anything to do with leaks. Also, you really should not turn off site isolation.

Comment 55

[Andrew S.]

A small update - it still crashes daily. (Have to note that in most cases it results in not a crash, but rather in hang/degraded performance so the tab is just reloaded without a crash report)

A few days before though I disabled Windows Multi-Plane Overlay and... it fixed the crashes and Firefox sub-processes running high on memory.

As a side note this also fixed jittery scrolling of animated pictures and video on various pages.

If you need any logs or whatever with this - notify me.

Comment 56

[Andrew S.]

After testing a bit more, it happens not to fix things, but to slow them down.

Comment 57

[mirh]

Depends also on bug 1351194, bug 1400173, bug 1739277, bug 1833070, bug 1945363, bug 1948501 and bug 1957081.