1568286 - Silent parts of YouTube videos use more CPU at slow or fast speeds

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Synetech]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3861.0 Safari/537.36

Steps to reproduce:

1. Find a YouTube video that has a portion that is silent (not just quiet, but actually has no audio data at that part). Time-lapses are often a good source of silent video, as are outros (the end—or start?—of all videos also seems to do it).

Here's a video that does it right at the beginning: youtu.be/DT0tZ8GF3uI

2. Play the silent part at something other than 1× speed (anything other than 1× works)

Actual results:

The CPU load goes through the roof until the video starts playing audio again.

Expected results:

Nothing. The CPU load should have remained unchanged.

Comment 1

[Synetech]

(Silent videos play fine on other sites at speeds other than 1×; it's only YouTube videos that seem to do it.)

Comment 2

[Paul Adenot (:padenot)]

I can't repro on Linux or OSX, which Firefox version can you repro on?

Comment 3

[Synetech]

I've been dealing with this since I switched back to Firefox as my default in 2016. I don't know what version it was, but it was definitely before Quantum (57).

I'm experiencing it right now on a laptop with Intel graphics, but I also get it on a laptop with Nvidia graphics, and I've seen it on a couple of desktops as well. (They're all Windows 7 and 10; I can probably check XP if needed.)

Also, yes, it happens if I start in safe mode or use a fresh profile.

Comment 4

[Paul Adenot (:padenot)]

I can't repro on Windows either. This is really confusing.

jya, do you know what string one should put in the profiler so that the reporter can profile this on their machine?

Comment 5

[Paul Adenot (:padenot)]

Attachment: Firefox Profiler configuration

Reporter, can you install the profiler by going to https://profiler.firefox.com/, open it (this adds an icon that's a gray circle), and configure it like the attached image, with the the custom threads like this:

GeckoMain,Compositor,MediaPDecoder,MediaTimer,MediaPlayback,MediaDecoderStateMachine

and the start it, reproduce the problem by playing a video, capture the profile in the profiler windows, hit "publish" in the top right corner (green button), and give us the address? This will help us understand what Firefox is doing in this particular scenario on your setup. I've been trying on 4 computers with various OSes, it all works as intended.

Comment 6

[Synetech]

I tried the profiler:

1. Opened the video youtube.com/embed/DT0tZ8GF3uI (which starts off silent) in embedded mode to eliminate all other aspects
2. Resized browser window to ~640x360 to minimize load as much as possible
3. Loaded the video, then paused it
4. Set it to 1.5× speed, manually set it to 144p, set it to the start (0:00) and waited a few seconds for it to load the 144p video
5. Started the profiler and let it run for ~3 seconds until CPU usage fell back to ~0%
6. Let profiler run for ~5 seconds to get an idle baseline
7. Let the silent part of the video play (at 1.5× speed, it lasted about 10-11 seconds, and spikes the CPU the whole time, ~50%)
8. Once the sound kicked in, I let it run for another 10-11 seconds (CPU usage was ~1%-3%)
9. Paused the video
10. Let the profiler run for another ~5 seconds (should match the first 5 seconds)
11. Saved the resulting profile: https://perfht.ml/2Mp9yVh (I don't see any CPU load in the graph. 😕)

I also tried it with a different video (https://youtu.be/pQf5ukMlmWI) that switches back and forth between sound and silence several times. While I was testing that one, I noticed the speaker icon in the tab fading in and out each time the sound came and went. I can't help but wonder if the problem is related to the browser's audio-detection; maybe when it is checking to see if a tab has audio and needs to display sound, it gets messed up if the sound is being played back to fast. 🤔

Comment 7

[Paul Adenot (:padenot)]

There is no CPU spike because you didn't record the threads on which we do the expensive work. Have you copy/pasted the thread list:

GeckoMain,Compositor,MediaPDecoder,MediaTimer,MediaPlayback,MediaDecoderStateMachine

in the box, and restarted the profiler (there is a button at the bottom) ?

I can't help but wonder if the problem is related to the browser's audio-detection; maybe when it is checking to see if a tab has audio and needs to display sound, it gets messed up if the sound is being played back to fast. 🤔

That'd also be my guess, but we really need the measurement to be sure.

Comment 8

[Synetech]

Oops, I forgot that part. I did it again and you can clearly see all kinds of mountains in the graph during the silent part and then they disappear once the audio kicks in: https://perfht.ml/2MpLiSO

Comment 9

[Paul Adenot (:padenot)]

This is really strange: on your setup, when the audio is not playing, the code seem to spin trying to get the audio output position. This leads to lots of mutex contention, but I think this is the consequences of a real bug.

I think HaveEnoughDecodedAudio is the culprit. Maybe youtube is letting the compressed audio under-run intentionally when the video is silent ?

Comment 10

[Paul Adenot (:padenot)]

Do you understand what is going on here? Can you repro maybe?

Comment 11

[Alastor Wu [:alwu]]

When we're playing inaudible part of the video, MDSM is actually decoding silent audio and play them via the audio sink. The function calls of HaveEnoughDecodedAudio() are normal because we have to check the amount of decoded audio data and then decide when we should dispatch another audio decoding task.

However, it's really weird to wait such a long time on the mutex. I also tried to reproduce this issue [1] on Win10, and there are indeed some small peaks happen when MDSM tries to do somthing, but all of them finish within a short time period and will happen on both audible and inaudible part. But interesting point is, the memory usage in inaudible part is larger than the one in audible part.

In the profiller result in comment 8 [2], I found another weird thing which I couldn't explain why, and I don't know if it's related with this issue. MDSM ran on MediaPlayback threads instead of MDSM thread, so we could see multiple tasks were running at the same time in different threads. Perhaps it's a reason to cause locking the mutex? In theory, MSDM should run on MDSM thread [3] only.

[1] My result : https://perfht.ml/33LUc3p
[2] Problem result : https://perfht.ml/2MpLiSO
[3] https://bit.ly/2z8e4iZ

Comment 12

[Dan Minor [:dminor]]

Sounds like this is confirmed by the reporter's profile. Thank you!

Comment 13

[Synetech]

I think this is fixed, at least for me. It seems to have gotten fixed in 79, and is likely due to the WebRender update ( https://wiki.mozilla.org/Platform/GFX/WebRender_Where ) allowing Firefox to more properly support older integrated Intel graphics—in Windows 7—(which describes my old laptop).

I can't speak for every configuration, but for me at least, it's now fixed and I can have an audio-less video running at a speed other than 1× without it pegging the CPU. Hopefully any other configurations that have this issue will be fixed with further WebRender updates.

Comment 14

[Paul Adenot (:padenot)]

This was fixed in bug 1634295 (nothing to do with webrender), I forgot to update this bug, thanks for dropping by and confirming!