1516943 - MediaCache can use lots of disk space for every content process

Status: NEW

(Core :: Audio/Video: Playback, defect, P3)

Description

[Konomi]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0

Steps to reproduce:

Watch one of the videos from ccc.de for example: https://media.ccc.de/v/35c3-9375-silivaccine_north_korea_s_weapon_of_mass_detection


Actual results:

When you check /tmp it is slowly filling up with the video eventually the video exceeds the size of the /tmp mount the tab crashes and Firefox leaves /tmp full until you close it.


Expected results:

Firefox should not stream the entire video to /tmp but only parts of it so it does not fill up /tmp.

Comment 1

[Andreas Pehrson [:pehrsons]]

Chris, is this the workings of the MediaCache, or possibly something else?

Comment 2

[Konomi]

Also follow up, I'd just like to say my tmpfs is a memory based one and here is the output of the mount command for it.

tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=524288k)

and fstab.

tmpfs    /tmp    tmpfs    rw,nodev,nosuid,relatime,size=512M    0    0

I suspect setting a smaller tmpfs size would make this bug even appear even more frequently.

Comment 3

[Chris Pearce [:cpearce (Not reading bugmail)]]

Konomi: You could try opening about config and change media.cache_size to a much smaller value; say drop a zero off the value. That should reduce the amount of data Firefox stores on disk, and make things better for you.

(In reply to Andreas Pehrson [:pehrsons] from comment #1)

Chris, is this the workings of the MediaCache, or possibly something else?

The media cache was written at a time when Firefox had only one process, so every process just creates a new one as needed. So in multi-process Firefox we end up with one media cache per content process. Each content process' cache can be up to 512MB in size. Data from active media elements in a content process is stored in the process' cache, and purged when the media element is shutdown.

So if someone ends up with a bunch of different tabs in different content process; which all start downloading large media files, we could potentially end up caching a lot of data in /tmp, potentially way more than 512mb.

I don't imagine the configuration with such a small(!) tmp storage would be common, but that's an untested assumption.

In the multi-e10s world, this gets worse. It would be interesting to collect telemetry to try to quantify how much of a problem this will be.

Comment 4

[Andreas Pehrson [:pehrsons]]

Thanks for the explanation.

It doesn't sound too bad given that the cache is purged on media element shutdown. I don't see it as very common to keep multiple tabs around with finished-but-not-yet-shutdown media elements in them. Still, I'm marking this confirmed and P3 for the sake of adding telemetry.

Comment 5

[User Dderss]

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #3)

Data from active media elements in a content process is stored in the process' cache, and purged when the media element is shutdown.

So if someone ends up with a bunch of different tabs in different content process; which all start downloading large media files, we could potentially end up caching a lot of data in /tmp, potentially way more than 512mb.

Thanks, Christopher, but what should be done to the settings to deal with kind of an opposite issue?

Sometimes sites like Reddit are so slow in terms of delivering their videos that it can take forever to load them even though the video itself is small (like here it is just two and a half minutes: https://www.reddit.com/r/PewdiepieSubmissions/comments/amvsf2/i_remade_bitch_lasagna_in_fortnite_using_the_new/)?

The goal is to make FF to load the whole video into the cache -- however slowly it might happen -- while user is doing something else in other tabs. Then the user should be able to see the whole video.

As it is now HTML5 video playback only works by loading small "blobs" and then stops requesting new data until the nearest few seconds are played by the viewer. And, what is even worse, it quickly purges all of the previous data that was already played back so there is no way for users to eventually see the video in its entirety in a continuous playback once it will finally load up to its very last second.

What settings in "about:config" can be changed to force FireFox to keep small videos fully cached (maybe up to certain size limit)? Better yet, FF should see that he video is loaded very slowly so it can adjust the caching rules for both forwards (to never stop loading more until the end of video is reached) and backwards (to not clear it) on the fly. For that, the byte-stamps that define the blob batches via "m3u8" file for videos should be ingored.

Otherwise to watch the slow Reddit videos I have to do this:

1. use Web Inspector on the video tag element to find the source for the video tag, which "HLSPlaylist.m3u8",

2. download it, find in it a name of files with the settings list with the best bitrate/resolution version of the video: "HLS_540_v4.m3u8" (yes, Reddit is this bad, this is the best resolution it can do; this is the file which FF would eventually use as a guideline about caching, and it is a disaster; it should be overruled if some setting is activated) and "HLS_AUDIO_160_K_v4.m3u8" for audio,

3. download them, and find the name of the actual video and audio files for the chosen resolution: "HLS540.ts" and "HLS_AUDIO_160_K.ts"

4. download them, and concat/joinmux them with tsMuxeR_2.6.12 utility.

The whole process it takes time, but is the entire video and it can be watched it continusouly and smoothly from the very beginning to its end without interruptions and endlessly waiting for when/if the next part of the "blob" will be loaded.

Thank you in advance.

Comment 6

[Neha Kochar [:neha]]

This doesn't block Fission.