1686267 - Youtube live stream memory leak from Video DownloadHelper

Status: NEW

(WebExtensions :: General, defect, P3)

Description

[karegun]

Attachment: memory-report.json.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

Simply watching a youtube livestream broadcast triggers this memory leak. Had no problems with other streaming sites.

Actual results:

Memory consumption of firefox rises until my computer runs out of memory.

Expected results:

At least forcing the GC to collect free memory should have worked.
about:memory's GC did not help.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Jon Coppeard (:jonco)]

Most memory consumption (2,244.00 MB) is from the "Video DownloadHelper" extension. It's probably that causing the problem. Try turning that off and see if it makes a difference.

Comment 3

[karegun]

This is amazing. Thankyou for your investigation.

Comment 4

[Andrew McCreight [:mccr8]]

It would be a good idea to report this issue to the developers of the addon. Maybe it is doing some kind of caching in case you want to save the video?

Comment 5

[Alex Cornestean]

Hello,

I’ve attempted to reproduce the issue on the latest Nightly (86.0a1/20210113213439), Beta (85.0b9/20210114193053) and Release (84.0.2/20210105180113) under Windows 10 x64.

I’ve started watching a livestream on YouTube while the Video DownloadHelper add-on was installed and enabled. I’ve monitored the memory consumption of Firefox during this time and it never spiked above 1200 MB, constantly fluctuating between 900 MB and 1200 MB. Please note that I've watched the stream for about 15-20 minutes. If this time period is not enough please let me know and I'll test this for a more extended period of time.

Disabling the add-on did not help much in reducing memory consumption. It maybe reduced it with 100 MB or so.

I’ve tested this with the latest version of the add-on.

Comment 6

[Luca Greco [:rpl] [:luca] [:lgreco]]

is there any particular option you may have set on the VideoDownloaderHelper addon preferences that may be needed to let Alex to be able to reproduce the same issue?

Comment 7

[karegun]

Hello

I haven't altered any settings for the addon.
So it should be the default settings.

I tested again with a local news live stream channel and was not able to reproduce the problem.

I've noticed that the news livestream was a recorded video streaming live,
and the livestream I had problems with is an actual live broadcast.

I switched to a poker 24/7 live broadcast on Youtube and thought I was reproducing the problem.
The memory assigned to firefox linearly increases up to about 2GB which was far from the usual memory consumption problem, taking up to 4GB for firefox alone.
This time the memory consumption increased much slower and firefox freed some memory time to time

I then noticed that the poker streaming had no live chatting, and the local news had only a few chats comming up.
I switched to a live game streaming channel with consistent chattings comming up and the memory occasionally spiked to 3GB.
Firefox still performs some kind of GC and frees memory before hitting 4GB, but that might have been possible only because of the there weren't enough chats.

I still cannot assure you that it is because of the chats + VideoDownloadHelper.
It might be depending on the streaming system the streamers link to Youtube.
This is as far as I can tell you.

Comment 8

[Alex Cornestean]

Hello,

Revisited the issue on account of the new information regarding the correlation between the memory usage and the chat activity in the live stream.

So I started watching a Fortnite stream (due to likely high chat activity, which was the case) and I observed several things:

1. During the start of me watching the stream, memory usage went up to almost 2 GB and after a few minutes at that level, Firefox freed about 1 GB. This happened a few times more over the next minutes.
2. 15-20 minutes into the stream, usage went up to 2.6 GB and Firefox freed once more about 1 GB after a few minutes.
3. After another 15-20 minutes, memory usage grew to 8.6 GB and remained there. This was the point in the stream when chat activity seemed to intensify. At this point, I decided to disable the extension to see whether the memory usage was indeed caused by the add-on. At first, memory usage fluctuated slightly but then went down quickly to ~2 GB.

Tested this on the latest Release.

Hope this helps.

Comment 9

[Andrew McCreight [:mccr8]]

Thanks for the investigation.

Looking at the memory report, it looks like the memory usage is a ton of JS classes in the extension process. The behavior described in comment 8 sounds like it could be an issue with Firefox not GCing enough, though I guess you'd really want to hit the "GC" and "CC" buttons in about:memory a few times each to see if that is the case.

Jon, this might be interesting to you. I'm not sure if the bug should be left in this component or maybe moved over to JS somewhere.

Comment 10

[Andrew McCreight [:mccr8]]

This from the memory report is interesting: string(length=49, copies=2895485, "content/images/icon-action-quick-download2-64.png")

Hard to tell if this is a leak in the extension or if the GC could be working more aggressively. I don't recall much time being spent on the WebExtension process in terms of tuning the GC, so maybe something is going wrong there.

Comment 11

[Luca Greco [:rpl] [:luca] [:lgreco]]

Adding Bug 1655690 as a see also, because it may be relevant while investigating this issue.

Comment 12

[Jon Coppeard (:jonco)]

(In reply to Andrew McCreight [:mccr8] from comment #9)
I don't know anything about how GC is triggered in extension processes. From the JS engine standpoint, the triggers based on heap usage are exactly the same though.

Does triggering GC from about:memory cause the memory use be reduced significantly? If so it's likely to be a scheduling issue. If not it's likey a leak.