1138967 - Playing 4K YouTube HTML5 MSE videos seriously slows down the browser

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P2)

Description

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

Reproducible with: 
- Firefox 37 Beta 2 - BuildID: 20150302192546
- latest Firefox 38 Aurora - BuildID: 20150303004530
- latest Firefox 39 Nightly - BuildID: 20150303030230

Reproducible on: Windows 7 x64 & x86, Windows 8 x64, Mac OS X 10.8.5

Steps to reproduce:
1. Open Firefox with a clean profile and open a 4K youtube video (e.g. https://www.youtube.com/watch?v=-YGDyPAwQz0).
2. From the settings button select "2160p4K".

Expected results:
Video plays at 4K resolution without performance issues.

Actual results:
Video plays at 4K resolution but with visible performance issues (very low framerate). It seems it's the browser that slows down as other tabs are also very slow to interact with (e.g. loading content, scrolling).

Notes:
- Video plays with MSE enabled (Dash=Yes)
- CPU does not seem to go up while experiencing these issues (CPU use is easily under 10% and does not go up)
- Firefox 36 has the same issues when setting "media.mediasource.enabled"=true, while 35.0.1 has some issues buffering so cannot really determine whether it also has this issue or not
- Flash plays 4K resolution without issues

Comment 1

[Lawrence Mandel [:lmandel] (use needinfo)]

Tracking as this looks like a regression when switching from the YouTube Flash player to MSE playback.

ni rillian for prioritization.

Comment 2

[Ralph Giles (:rillian)]

Florin, can you share the graphics hardware/driver info from your reproduction machines? e.g. the 'Graphics' section of about:support? If it's universal it might be a bottleneck in our painting code, but playback performance is also going to depend on the system's graphics.

What resolution does 'stats for nerds' show with 4k flash? I was under the impression YouTube limited their flash offerings to lower resolutions and framerates.

Comment 3

[Leman Bennett [Omega]]

I also see this. The video buffers considerably and triggers the warning script on YouTube that monitors playback.

Graphics
Adapter Description	AMD Radeon HD 7900 Series
Adapter Description (GPU #2)	Intel(R) HD Graphics 4000
Adapter Drivers	aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
Adapter Drivers (GPU #2)	igdumdim64 igd10iumd64 igd10iumd64 igdumdim32 igd10iumd32 igd10iumd32
Adapter RAM	3072
Adapter RAM (GPU #2)	Unknown
ClearType Parameters	D [ Gamma: 2200 Pixel Structure: R ClearType Level: 50 Enhanced Contrast: 100 ] D [ Gamma: 2200 Pixel Structure: R ClearType Level: 50 Enhanced Contrast: 100 ] D [ Gamma: 2200 Pixel Structure: R ClearType Level: 100 Enhanced Contrast: 50 ]
Device ID	0x6798
Device ID (GPU #2)	0x0162
Direct2D Enabled	true
DirectWrite Enabled	true (6.3.9600.17415)
Driver Date	11-20-2014
Driver Date (GPU #2)	12-18-2014
Driver Version	14.501.1003.0
Driver Version (GPU #2)	10.18.10.4061
GPU #2 Active	false
GPU Accelerated Windows	1/1 Direct3D 11 (OMTC)
Subsys ID	32111682
Subsys ID (GPU #2)	0000000c
Vendor ID	0x1002
Vendor ID (GPU #2)	0x8086
WebGL Renderer	Google Inc. -- ANGLE (AMD Radeon HD 7900 Series Direct3D11 vs_5_0 ps_5_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d 1.1
AzureContentBackend	direct2d 1.1
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0


(In reply to Ralph Giles (:rillian) from comment #2)
> 
> What resolution does 'stats for nerds' show with 4k flash? I was under the
> impression YouTube limited their flash offerings to lower resolutions and
> framerates.

YouTube limits 60FPS videos, not 4K. 4K in Flash works but it drops a few frames occasionally.

Comment 4

[Sheila Mooney]

I can reproduce this with this video on both my Mac and Windows machines. 

On my Mac, I tried a number of other 4K videos and couldn't find a similar problem. On the PC, I found this issue with several other videos. Here are a couple.

https://www.youtube.com/watch?v=Cx6eaVeYXOs
https://www.youtube.com/watch?v=iNJdPyoqt8U

Comment 5

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

(In reply to Ralph Giles (:rillian) from comment #2)
> Florin, can you share the graphics hardware/driver info from your
> reproduction machines? e.g. the 'Graphics' section of about:support? If it's
> universal it might be a bottleneck in our painting code, but playback
> performance is also going to depend on the system's graphics.

This does indeed seem like a universal problem (especially since it was confirmed in comment 3 and comment 4). Just to confirm, here's the graphics we used to reproduce this:
- Windows 7 x64 - nVIDIA GeForce GT 610 (with the latest driver delivered by NVIDIA GeForce Experience)
- Windows 7 x86 - Intel HD 2500
- Windows 8 x64 - AMD Radeon HD 5450

> What resolution does 'stats for nerds' show with 4k flash? I was under the
> impression YouTube limited their flash offerings to lower resolutions and
> framerates.

Resolution: 3840 x 2160 @ 30

Comment 6

[Sheila Mooney]

I tested this on other browsers. Chrome has the same performance issues on both the Mac and Windows. I tried a few 4K videos. I can't reproduce the problem on Safari.

Comment 7

[Leman Bennett [Omega]]

This kind of looks like a buffering issue. Internet Explorer 11 is a lot smoother but suffers from occasional buffering according to the network monitor in Task Manager. 

Every time the scrubber reaches the end of the buffer, the video stalls while waiting for the next chunk. During that time, network activity bottoms out and then spikes again. When the spike happens, the video resumes. On Chromium based browsers, the buffer happens consistently, but there seems to be an acceleration issue since the video takes significantly more CPU to decode and play which makes sense why the playback is choppy.

Comment 8

[Lawrence Mandel [:lmandel] (use needinfo)]

I'm marking this as wontfix for 37. Although this may come across as a regression from the user perspective, there is a small user base for 4K videos currently and this is not going to block the MSE release. I would really like to see this fixed in 38 and am leaving this as tracking 38 as a result.

Comment 9

[Matt Woodrow (:mattwoodrow)]

Attachment: [WIP] Upload video frames on the client side

Comment 10

[Matt Woodrow (:mattwoodrow)]

Attachment: [WIP] Move image data copy off the decoder thread

This one seems to make a pretty noticeable improvement, but still not enough to maintain 30fps on my machine.

Comment 11

[Matt Woodrow (:mattwoodrow)]

Attachment: Part 1: Remove ISharedImage

Comment 12

[Matt Woodrow (:mattwoodrow)]

Attachment: Part 2: Move image data copy off the decoder thread

The existing code (which ended up creating a SharedPlanarYCbCbrImage) made a copy of the image data while blocking the decoder thread.

This is a large copy for 4k video, and we want to avoid blocking the decoder for that long.

This version will instead trigger the PlanarYCbCr path in ImageClient::UpdateImage, and do the copy on the ImageBridge thread.

This results in a fairly large fps improvement with 4k video on my machine.

Comment 13

[Matt Woodrow (:mattwoodrow)]

Attachment: Part 3: Upload on the client side

This patch makes us upload to d3d directly from the ImageBridge thread, instead of allocating and copying into memory/shmem (only to do the same upload on the compositor).

As well as removing a copy, this increases the amount of time between upload and compositing, so should reduce the amount we need to block waiting for it to complete.

roc is currently working on patches that will provide a set of video frames to the compositor instead of the current one, so that should improve the upload times futher.

With all 3 patches applied, I can now play software decoded 4k video without any frame drops on my machine.

Comment 14

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 8579027 [details] [diff] [review]
Part 2: Move image data copy off the decoder thread

Review of attachment 8579027 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/fmp4/wmf/WMFVideoMFTManager.cpp
@@ +402,1 @@
>                                              mImageContainer,

Nit: indentation of hanging args don't line up with '('.

Comment 15

[Nicolas Silva [:nical]]

Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

Review of attachment 8579026 [details] [diff] [review]:
-----------------------------------------------------------------

yay!

Comment 16

[Nicolas Silva [:nical]]

Comment on attachment 8579027 [details] [diff] [review]
Part 2: Move image data copy off the decoder thread

Review of attachment 8579027 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/layers/IMFYCbCrImage.h
@@ +16,5 @@
> +class IMFYCbCrImage : public PlanarYCbCrImage
> +{
> +public:
> +  IMFYCbCrImage(IMFMediaBuffer* aBuffer, IMF2DBuffer* a2DBuffer)
> +    : PlanarYCbCrImage(nullptr)

This is a bit sketchy. Some of PlanarYCbCrImage's methods expect to have a non-null RecycleBin for example. I'd feel better if you override the PlanarYCbCrImage methods that don't work with this class to MOZ_ASSERT.

Comment 17

[Nicolas Silva [:nical]]

Comment on attachment 8579028 [details] [diff] [review]
Part 3: Upload on the client side

Review of attachment 8579028 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/layers/d3d11/TextureD3D11.h
@@ +77,5 @@
>    bool mNeedsClear;
>    bool mNeedsClearWhite;
>  };
>  
> +class SharedYCbCrTextureClientD3D11 : public TextureClient

nit: how about calling this one DXGIYCbCrTextureClientD3D11 (TextureClients are all shared and the TextureHost is names like this)?

Comment 18

[Nicolas Silva [:nical]]

Cool stuff.

Comment 19

[Matt Woodrow (:mattwoodrow)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a3c8482c41f6
https://hg.mozilla.org/integration/mozilla-inbound/rev/215046aa1f38
https://hg.mozilla.org/integration/mozilla-inbound/rev/d6242b24bc47

Comment 20

[Matt Woodrow (:mattwoodrow)]

Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

Approval Request Comment
[Feature/regressing bug #]: HTML5 Video
[User impact if declined]: Poor fps (especially with 4k video) when using software decoding
[Describe test coverage new/current, TreeHerder]: Lots of manual testing, not covered by treeherder.
[Risks and why]: Fairly low risk change to the way we upload video frames, much faster.
[String/UUID change made/needed]: None.

Comment 21

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

We want this patch because it improves the performance of software decoding across the board. This is especially important because a most AMD cards fall back to software decoding.

Declining it will not break anything with YouTube set to auto mode bit it will tend to use a lower resolution.

Comment 22

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a3c8482c41f6
https://hg.mozilla.org/mozilla-central/rev/215046aa1f38
https://hg.mozilla.org/mozilla-central/rev/d6242b24bc47

Comment 23

[Lawrence Mandel [:lmandel] (use needinfo)]

Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

We're going to take some risk on this patch that has only recently landed on m-c for the performance benefit on YouTube with MSE. Worst case we'll need to pull this fix out of the RC on Monday. Beta+ Aurora+

Florin - Can you please include YouTube playback on AMD devices in your exploratory testing for Beta 7?

Anthony - As you have a 4K monitor, can you please test 4K video playback with Beta 7?

Comment 24

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #23)
> Florin - Can you please include YouTube playback on AMD devices in your
> exploratory testing for Beta 7?

Will do.

Comment 25

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/c1356c27fa1b
https://hg.mozilla.org/releases/mozilla-beta/rev/07e266d45703
https://hg.mozilla.org/releases/mozilla-beta/rev/0c23dcbc6bf7

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/126db33d2905
https://hg.mozilla.org/releases/mozilla-aurora/rev/77b336a4baf6
https://hg.mozilla.org/releases/mozilla-aurora/rev/22a4edc0cfa8

Comment 27

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

I've tested the patch with nVidia, AMD and Intel graphics cards and they all work with a 1080 monitor but all the graphics cards struggle to drive a 4k monitor. We get similar results with software decoding to Chrome which uses VP9 software decoding. I'll do further testing with 37b7.

Comment 28

[Trevor Saunders (:tbsaunde)]

I pushed https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f to fixup mingw / case sensitive file systems unfortunately with the wrong bug number oops

Comment 29

[Trevor Saunders (:tbsaunde)]

I pushed https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f to fix mingw / case sensitive file systems (with the wrong bug number oops)

Comment 30

[Trevor Saunders (:tbsaunde)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f (wrong bug number oops)

Comment 31

[Trevor Saunders (:tbsaunde)]

and git bz sucks sorry

Comment 32

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/824009fbc42f

Comment 33

[Robert Kaiser]

Looks like this is the probable cause of a crash increase in 37.0b7, tracked in bug 1146315, bug 1116812 and bug 1146313.

Comment 34

[Matt Woodrow (:mattwoodrow)]

Backed part 3 of this out of branches because of the regressions, will back out of m-c when it reopens.

https://hg.mozilla.org/releases/mozilla-aurora/rev/5fe9f09fe578
https://hg.mozilla.org/releases/mozilla-beta/rev/2592523e1eb0
https://hg.mozilla.org/releases/mozilla-release/rev/6d7a2555b021

Comment 35

[Joel Maher ( :jmaher ) (UTC -8)]

this caused a slight (2%) win7 private bytes perf regression:
http://alertmanager.allizom.org:8080/alerts.html?rev=41a819532cfb&showAll=1&testIndex=0&platIndex=0

the regression went away after this was backed out.  I saw it on aurora, not sure about beta, release.

Comment 36

[Lawrence Mandel [:lmandel] (use needinfo)]

Now that part 3 of this bug has been backed out of 37 it will not be fixed in the 37 release.

Comment 37

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

While testing today Firefox 37 RC 2 (BuildID=20150326190726) on Windows 7 x64, I noticed that despite slowing down the browser, 4k videos do not take much CPU and memory. For my machine it's usually ~5-10% CPU and ~300-400 Mb.

Sometimes though, I noticed that the CPU and memory usage spike up to ~75% CPU and ~1Gb memory very quickly. I saw this happen in a fairly consistent manner when I left the 4K video play in a tab and then just opened/navigated/closed some other pages in other tabs. After 3-5 minutes the CPU and memory usage just shoot up.

I feel the issue described above is most likely related to this bug so I did not file a separate bug. I think we should revisit this scenario after we get a fix for this bug. Please let me know if you think this may be a separate issue that needs to be filed separately.

Comment 38

[Sylvestre Ledru [:Sylvestre]]

Trevors, could you also backout it for 38? Thanks

Comment 39

[Sylvestre Ledru [:Sylvestre]]

Sorry for the confusion, Kairo mentioned that it has been done already.

Comment 40

[Robert Kaiser]

FWIW, I do not see any reason for backing this out in 38, the jump in crashes I saw in comment #33 has been mitigated by some blocklisting for the few affected gfx adapters.

Comment 41

[Matt Woodrow (:mattwoodrow)]

I think we can close this now, we don't have anything else actionable to do here.

Sylvestre, can we stop tracking this?

Comment 42

[Liz Henry (:lizzard) (relman/hg->git project)]

KaiRo, I think also we kept it backed out because of comment 35 - performance regressions noted in talos.

Matt, it sounds like we may have fixed some broad issues but are left with an intermittent edge case in Win 7 x64. Would that be useful, to keep it clear that we still have an issue?

Comment 43

[Sylvestre Ledru [:Sylvestre]]

Too late anyway for 38.

Comment 44

[Liz Henry (:lizzard) (relman/hg->git project)]

Dropping tracking for this for 39 since the main issue seems to be solved. Florin, if you're still seeing the problem in comment 37 please file a new bug.