Open Bug 1837553 Opened 1 years ago Updated 3 months ago

KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts

Categories

(Core :: Audio/Video: Playback, defect)

Product:
Core
Component:
Audio/Video: Playback
Version:
Firefox 114
Platform:
Unspecified
Linux
Type:
defect

Tracking

()

Project Flags:
Performance Impact low
Tracking Flags:
Tracking Status
firefox114 --- disabled

People

(Reporter: kamalahmad22, Unassigned)

Details

(Keywords: memory-leak, perf:resource-use)

Attachments

(7 files)

Memory Report
573.45 KB, application/gzip
Details
about:support page with VAAPI enabled
53.63 KB, text/plain
Details
Output of vainfo -a
1.20 KB, text/plain
Details
MOZ_LOG="PlatformDecoderModule:5" logs
9.15 MB, text/plain
Details
memory-report.json.gz
457.29 KB, application/gzip
Details
memory-report_before.json.gz
125.35 KB, application/gzip
Details
memory-report_after.json.gz
160.71 KB, application/gzip
Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0

Steps to reproduce:

Start firefox, use it for a bit

Actual results:

Firefox is consuming almost all of the RAM on my system, to the point my swap space is also filling up. Checking about:processes reveals that "Data Decoder" is using 8GB of ram

Expected results:

This did not occur before. I believe it is happening ever since I updated Firefox

Performance Impact: --- → ?

I'm guessing that "Data Decoder" means the RDD process.

Could you please attach an about:memory report from while this is happening? Hopefully before it is taking up all of the memory, because getting a report can use up some resources so you want to do it before that. Also, feel free to check the "anonymize" box or it'll include the URLs of your open tabs.

Component: Untriaged → Performance
Flags: needinfo?(kamalahmad22)
Product: Firefox → Core
Attached file Memory Report —
Flags: needinfo?(kamalahmad22)

(In reply to Andrew McCreight [:mccr8] from comment #1)

I'm guessing that "Data Decoder" means the RDD process.

Could you please attach an about:memory report from while this is happening? Hopefully before it is taking up all of the memory, because getting a report can use up some resources so you want to do it before that. Also, feel free to check the "anonymize" box or it'll include the URLs of your open tabs.

Hi, I have attached the memory report. For some more information, it seems to be related to video somehow. If I open Youtube Shorts and scroll down to the next video, my system memory usage increases by about 3% each time I scroll down. But then closing the Youtube Shorts tab still keeps the memory usage around the same, even if I go to about:memory and press the minimize memory usage button. Closing just about every tab (except this one) brings down total memory use by about 7%, which means Firefox is still using about 6GB of memory.

This bug was moved into the Performance component.

:kamalahmad22, could you make sure the following information is on this bug?

  • For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
  • ✅ For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
  • Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm it is recent.

Thank you.

Flags: needinfo?(kamalahmad22)

Thanks for the memory report. As expected, this is in the RDD process. Unfortunately, it is all under heap-unclassified. I think the next steps here are to look at a profile and hope that something shows up, or if somebody can reproduce in a DMD build we can see where the memory is going at least.

The RDD process is used by every web page, so I think it is expected that if there's a leak that just closing the tabs won't fix it.

I'll move this to A/V playback, given that the steps involve YouTube and I expect that RDD is being use for video decoding somehow.

Status: UNCONFIRMED → NEW
Component: Performance → Audio/Video: Playback
Ever confirmed: true
Summary: High memory usage in Firefox (8GB+ used by Data Decoder) → 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts

Also, if you could more specific about what the steps to reproduce are and maybe attach your about:support that could help.

I was unable to reproduce this on Firefox 116 on MacOS. I went to a specific YouTube Short video, then scrolled down to the next video, then did that over and over again, sometimes letting the video play a bit and sometimes just scrolling as fast as I could, and my RDD process memory didn't go above 6MB. I could certainly imagine there are OS-specific differences. It looks like the reporter is on Linux.

(In reply to Andrew McCreight [:mccr8] from comment #6)

Also, if you could more specific about what the steps to reproduce are and maybe attach your about:support that could help.

I was unable to reproduce this on Firefox 116 on MacOS. I went to a specific YouTube Short video, then scrolled down to the next video, then did that over and over again, sometimes letting the video play a bit and sometimes just scrolling as fast as I could, and my RDD process memory didn't go above 6MB. I could certainly imagine there are OS-specific differences. It looks like the reporter is on Linux.

Hi, upon further inspection it seems to be related to VAAPI (hardware video acceleration for Linux). Disabling VAAPI by setting media.ffmpeg.vaapi.enabled to false seems to prevent memory from being leaked.

I have attached about:support information.

Flags: needinfo?(kamalahmad22)

Per comment7, link this to VAAPI meta bug.

Blocks: egl-linux-vaapi
Attachment #9338730 - Attachment mime type: application/octet-stream → text/plain

Does this problem still occur after setting media.ffmpeg.vaapi.force-surface-zero-copy to 1 on about:config and restarting Firefox?

status-firefox114: --- → disabled
Keywords: memory-leak
OS: Unspecified → Linux
Summary: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts → VAAPI/KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts

(In reply to Darkspirit from comment #10)

Does this problem still occur after setting media.ffmpeg.vaapi.force-surface-zero-copy to 1 on about:config and restarting Firefox?

Yes, it's still occurring. I have attached the output of vainfo -a incase it helps.

Attached file Output of vainfo -a —

Your vainfo doesn't contain any useful codecs. Fedora's ffmpeg and mesa-va-drivers packages have patented h264 disabled by default.
Users can replace those packages with ones from a third party repository: https://rpmfusion.org/Howto/Multimedia

109: Since bug 1756459, HW codecs found by VAAPI test are listed on about:support.
114: Since bug 1787182, VAAPI codecs are only tested&reported if HW decoding is enabled.
115: Since bug 1831038, VAAPI won't be used for a codec if it's not listed as HW codec on about:support. In case this bug only occurs if the VAAPI driver doesn't support any relevant codec, that change would prevent this bug.

Summary: VAAPI/KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts → VAAPI (Fedora=no h264)/KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts

(In reply to Darkspirit from comment #13)

Your vainfo doesn't contain any useful codecs. Fedora's ffmpeg and mesa-va-drivers packages have patented h264 disabled by default.
Users can replace those packages with ones from a third party repository: https://rpmfusion.org/Howto/Multimedia

109: Since bug 1756459, HW codecs found by VAAPI test are listed on about:support.
114: Since bug 1787182, VAAPI codecs are only tested&reported if HW decoding is enabled.
115: Since bug 1831038, VAAPI won't be used for a codec if it's not listed as HW codec on about:support. In case this bug only occurs if the VAAPI driver doesn't support any relevant codec, that change would prevent this bug.

Hi, good catch! The last time I checked vainfo it had H264/H265 but it doesn't anymore, as Fedora removed them by default. Replacing the VAAPI drivers with the freeworld drivers (proprietary codecs included) on Fedora seems to fix the issue. So I guess the bug only occurs if the decoder doesn't find a suitable hardware codec? But strangely, by forcing Youtube to use AV1 which my GPU doesn't support doesn't cause this memory leak issue in the same case (scrolling through Youtube Shorts).

The Performance Impact Calculator has determined this bug's performance impact to be low. If you'd like to request re-triage, you can reset the Performance Impact flag to "?" or needinfo the triage sheriff.

Configuration: Rare
[x] Causes severe resource usage

Keywords: perf:resource-use
Performance Impact: ? → low

The severity field is not set for this bug.
:jimm, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(jmathies)

From the report it looks like a problem with Fedora ffmpeg/OpenH264 backend (Fedora ships openh264 as ffmpeg decoder now).
On setup where you see the bug, can you please run Firefox as:

MOZ_LOG="PlatformDecoderModule:5" firefox > log.txt 2>&1

and attach the log here?
Thanks.

Flags: needinfo?(jmathies) → needinfo?(kamalahmad22)
Flags: needinfo?(kamalahmad22)

I have att(In reply to Martin Stránský [:stransky] (ni? me) from comment #17)

From the report it looks like a problem with Fedora ffmpeg/OpenH264 backend (Fedora ships openh264 as ffmpeg decoder now).
On setup where you see the bug, can you please run Firefox as:

MOZ_LOG="PlatformDecoderModule:5" firefox > log.txt 2>&1

and attach the log here?
Thanks.

I have attached a log with this running (swapped mesa freeworld drivers with regular drivers which don't have h264 and some other codecs) in the case where this occurs most often which is scrolling through Youtube Shorts

Not related to VA-API.

No longer blocks: egl-linux-vaapi
Summary: VAAPI (Fedora=no h264)/KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts → KDE X11/Mesa 23.1.1/Radeon RX 580: 5GB memory leak (heap-unclassified) in RDD process when scrolling on Youtube Shorts
Severity: -- → S3

Sorry, just found this in my history. Can you still reproduce this?

Hey. Just wanted to add a comment to this. I have a young kid who leaves Youtube active for long periods on the same (looping) short.
I've noticed (Firefox Stable as of 2024-08-25, Devuan, AMD CPU and GPU) that after many hours of this, Firefox is consuming 20 gigs of RAM or more, forcing swapping and hanging the rest of the system.

Exiting Firefox and restarting fixes. It's very reproducible though, if shorts are left playing long enough. It does not require scrolling. Just repeated playing.
I have noticed regular youtube can be memory hungry. I wonder if shorts can trigger increased consumption just due to more plays...

(oh, he does also scroll a lot, but I've seen this memory leak even when it's just been sitting on one video)

@nemo, thank you for confirming this. Can you please upload memory reports from about:memory before and after the leak?

alright. that'll be a weekend activity I suppose. It'll be a delicate balance because once it gets bad I can barely get the system to respond at all to a close, much less open new tabs. also we've been running that account in kiosk mode so I suppose I'll have to remove that to access tabs at all.

I forgot to do it before since I was busy with breakfast. I'll try another set again after, but I figured I might as well let you see the dramatic spike after less than half an hour of short watching and scrolling by one kid. I had to restart the browser 3 times this morning.

Attached file memory-report.json.gz —

One weird thing (for me) when looking at that memory tree ...
2,604.38 MB (100.0%) -- explicit
├──2,165.71 MB (83.16%) -- window-objects
│ ├────105.59 MB (04.05%) -- top(https://www.youtube.com
Snipping out only that section and counting it...
$ grep -P top.*youtube.com/ temp3.txt | sed 's/─/ /g' | awk '{SUM+=$3}END{print SUM}'
2151.32

This one was basically all youtube shorts. But he was only watching one.
$ grep -P top.*youtube.com/ temp3.txt | wc -l
26

26 were active though.
I'll check again to see if it gets worse, but it could simply be that youtube is simply creating many many many videos in their infinite scroller (in iframes?) and never cleaning up. It might be I thought it was happening without scrolling but he was scrolling sometimes when I wasn't looking...

Perhaps they are relying on a browser optimisation they added to chrome (i.e. - alternate browser hostile youtube again) or perhaps it's bad in all browsers, or perhaps Firefox just burns more ram for off-screen shorts. I know that Firefox years back added optimisations to unload images on large pages that weren't visible. Perhaps chrome does something similar but for these vids.

If that's the case and it's just a badly written infinite scroller, refreshing the window might be sufficient without restarting the browser... I guess I'll try that. If that fails, there really is a leak in Firefox.

Hum. Memory usage was still high when he moved over to a regular video. Refreshing did not help, about:memory still had a ton of youtube watch urls sucking up ram, despite the fact that he only had one tab open. Ok. This time I'm going to remember to take the snapshot first.

Ok. I did another run where I remember to record before launching. Just before measuring and halting, the PID looked like this:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6338 family 20 0 6960444 3.6g 80540 R 200.0 11.6 37:31.89 Isolated +
6633 family 20 0 6198916 3.1g 79568 S 0.0 10.0 14:24.77 Isolated +
6975 family 20 0 6110892 3.0g 75300 S 0.0 9.6 14:15.43 Isolated +
6808 family 20 0 5977712 2.4g 75176 R 200.0 7.8 13:50.84 Isolated +
6214 family 20 0 4133556 331208 108096 S 0.0 1.0 30:39.52 firefox-b+

Will attach the 2 .gz files next.

Before

After a couple of hours of only youtube, single tab.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: