1579595 - Firefox keeps allocating memory until all consumed when streaming live YouTube presentation

Status: UNCONFIRMED

(Core :: Audio/Video: Playback, defect)

Description

[rr_mozilla]

Attachment: about:memory while streaming FOX news "Live Now" hurricane monitor

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

Find any "Live Stream" or "Live Now" YouTube video. Should be a least an hour long. Try the YouTube channel "Lionel Nation" which usually has one every day. Another good one is Sky News (https://www.youtube.com/watch?v=siyW0GOBtbo). Memory usage will be seen to grow in task manager until all consumed then causing FF to lock up. Also had several other windows open at that time. Problem does not seem to happen as often on already uploaded podcasts.

Actual results:

After about an hour (more or less) all 6 gigs of available memory are consumed. At least one instance of Firefox in Windows 7 task manager (there are usually several instances) will have grown to over 1+ GBytes with others having escalated to a half gig or more.
Running an external garbage collector often shows a collection and return to pool total size of greater that 1 GByte. Repeated collection helps, but ultimately that tactic fails and Firefox goes "white' and becomes unresponsive. Firefox has to be forcibly killed in task manager and restarted. This happens on Win 8.1 as well. The attached file was taken as memory in the largest instance had grown to 900,000 mb from a starting point of about 50k.
Memory can be seen to grow by watching the task manager over about a 10 minute period.

Expected results:

FireFox memory usage size should remain stable during streaming if no other pages or tabs are added..

This problem has existed for a long time (years) on all of my computers and Firefox versions.

Comment 1

[Adrian Florinescu [:aflorinescu]]

I've tried to reproduce on a Win 7 Pro machine with a live now stream. I've left it running for 2 hours with no major leaks or memory spikes that I can see.
Also, the attached about:memory report doesn't speak to me: Mike, could you also please take a look?

Comment 2

[rr_mozilla]

Attachment: TakManager_2019.09.13_15h20m14s_001_.png

Task Manager shot 2:20

Comment 3

[rr_mozilla]

Attachment: TaskMnager2019.09.13_15h54m21s_004_.png

Second TakManager Shot 3:54 Note Firefox increase mem

Comment 4

[rr_mozilla]

Attachment: TaskManagerVideoOfGrowth2019.09.13_15h44m30s_003_.wmv

Video Showing growth while playing live stream. Note MANY other windows and tabs open duing this session.

Comment 5

[rr_mozilla]

Attachment: aboutMem3-23-20190913.txt at 3:23 same as taskmanager time snapshot

Simple about:memory ar 3:23 same time as Task Manager time snapshot

Comment 6

[rr_mozilla]

Attachment: aboutmem3-56-20190913.txt

About:memory 3: 54 same time as last Task manager snapshot.
Hope this helps
Note 32 windows and 80+ tabs were open during this test

Comment 7

[Mike Conley (:mconley) (:⚙️)]

I'm afraid I'm not able to find anything actionable in these memory reports. mccr8, do you see anything I don't?

Comment 8

[Andrew McCreight [:mccr8]]

Can you reproduce this in safe mode? I see that you have Video DownloadHelper and YouTube MP4 Downloader installed, which could be somehow keeping video data alive for a long time. You also have McAfee WebAdvisor, it looks like, and we've seen leak issues before with things like WebAdvisor that try to look at everything you load, so that could be an issue. Although, if it was an addon, I'd have expected the memory usage to show up somewhere under memory that the Firefox itself has allocated ("explicit").

For future reference, it would be better if you did "Measure and Save" in about:memory and uploaded the resulting report rather than copying and pasting about:memory directly.

Two of the processes each have about 4GB under address-space, so maybe that's where the bulk of the memory is going, but the copy and paste text just says "(2 tiny)" so I'm not sure what it actually is. Whatever is using up all of the memory doesn't seem to be anything explicitly allocated, and it doesn't seem to be anything we have a measurement for, aside from vsize.

Comment 9

[rr_mozilla]

Hi:

Thanks so much for continuing to pursue this problem. I'm glad to help out anyway that I can.

I deleted Video Download Helper and YouTube MP4 Downloader. I disabled McAfee web advisor. I then put my PC in safe mode.

I started the browser and took a screen capture of the task manager as well as a memory dump from the browser. I played an uploaded YouTube video from the channel "Real Coffee with Scott Adams". It seems that a live stream is not required for this problem to occur. Perhaps it just aggravates the situation. I'm not certain.

I took a snapshot of the task manager and another memory dump after about 20 minutes. As you can see from the task manager snapshots, the memory consumption problem did still manifest itself. I have attached the dumps and the screenshots.

I hope this helps. Let me know if I can be of any more assistance. Thanks. Roland.

Comment 10

[rr_mozilla]

Attachment: memory-report.json_1.gz before video start

safe mode dump before start of video

Comment 11

[rr_mozilla]

Attachment: memory-report.json_2.gz safe mode dump after 20 minutes

safemode dump after 20 minutes

Comment 12

[rr_mozilla]

Attachment: task manager Snap_2019.09.24_15h37m18s_004_.png before start

task manager snap of Firefox usage before start of video

Comment 13

[rr_mozilla]

Attachment: Tak manager Snap_2019.09.24_15h51m25s_005_.png after about 20 minutes

safe mode task manager snap after about 20 miutes

Comment 14

[Adrian Florinescu [:aflorinescu]]

rr_mozilla, seems this bug got stalled a bit.
I don't have any new ideas into further advancing this issue. What we can do though is confirm this issue still exists in the latest versions (or hope it was Cristmass and it wishfully got fixed inbetwen.) Could you please give it a spin in NIghtly75 or Release73? As a best practice, it would work best if you could use a new profile for the confirmation (you can easily achieve this just by installing a new Firefox from the above specified versions in a new unused path location)

Comment 15

[rr_mozilla]

Hi:
Thanks for keeping this bug in mind.

It is still happening under release 73. The memory increases in all instances of Firefox that show in the task manager. I have seen the largest instance go over 1.2 Gig when things just totally jam up, although the others are typically 500m + or so.

Here is a point that may help in trying to reproduce this problem on your side. In my main window I have more than (gasp) 100 tabs. The majority are unaccessed links at the time of jam up. About 1/2 are (unaccessed) YouTube links. There are 20 or so other windows which are open with a few tabs each. They have also been unaccessed at time of freeze.
I suspect there is a threshold - meaning the behavior starts to show up at 50 tabs or 100 or whatever. I would be happy to try to find that number if you can tell me a reliable way to save my entire current session so I could restore it later after I experiment to find such a threshold.
Although I originally thought that the behavior occurred only on live streams YouTube videos, I have found that it happens easily playing non-live 30 minute or longer YouTube videos. The memory grows and a freeze (memory full) occurs after about 1-2 hours of playing long videos. Live streams feel more aggressive about memory growth, but I don't have objective evidence of that,
My current strategy is to keep an eye on total usage and then run an external garbage collector periodically. When I do that, all instances of Firefox shrink remarkably often to 1/10 of their prior memory value. Regrowth starts quite quickly often going up to about half prior and then regrowth slowing down. But after a while even garbage collection becomes pointless because regrowth happens faster and faster after each garbage collection. So I restart because I cannot garbage collect fast enough to be usable. I am sure the behavior I just described seems really out of the box crazy, but I am trying to give details that may help you identify a starting point to debug.
Note: I have found this same memory growth behavior using Linux instances of Firefox.

Comment 16

[Screwtape]

This bit me today.

Steps to reproduce:

• using latest Nightly, build id 20201114215126 on 64-bit Linux
• launch in a new profile: firefox --profile /tmp/test-profile
• open https://www.youtube.com/watch?v=uNiBibOFq4s
• start playing, wait for any ads to run, then jump to 45:00
• let it play until 49:00

Expected results:

• memory usage stays relatively constant over the whole period, maybe growing a little as the video gets cached but then maintaining as steady state

Actual results:

• memory usage stays relatively constant, then starts spiking around 48:00-49:00
• pausing the video pauses the memory growth
• in about:memory, the webrender texture cache is holding onto gigabytes of memory
• garbage collection and cycle collection in about:memory do nothing
• "minimize memory usage" reduces the webrender texture cache to under 100MB; memory isn't actually released to the OS immediately (total memory usage in top does not go down) but does fall eventually if the video is resumed... but then the texture cache starts climbing again.

If I hide the chat replay before starting the video, the memory growth does not occur.

Maybe this is bug 1509911 or bug 1509656?

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 18

[Dan Collins]

I can confirm this issue exists on Windows 10 and Ubuntu 20.04.3. Until this is addressed the workaround I tell clients is to use Chrome for video streaming.