The default bug view has changed. See this FAQ.

Playing 4K YouTube HTML5 MSE videos seriously slows down the browser

RESOLVED FIXED

Status

()

Core
Audio/Video
P2
major
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: FlorinMezei, Assigned: mattwoodrow)

Tracking

(Blocks: 1 bug)

Trunk
Points:
---
Dependency tree / graph
Bug Flags:
qe-verify +

Firefox Tracking Flags

(firefox37+ wontfix, firefox38 affected, firefox39 affected, firefox40 affected)

Details

(URL)

Attachments

(3 attachments, 2 obsolete attachments)

(Reporter)

Description

2 years ago
Reproducible with: 
- Firefox 37 Beta 2 - BuildID: 20150302192546
- latest Firefox 38 Aurora - BuildID: 20150303004530
- latest Firefox 39 Nightly - BuildID: 20150303030230

Reproducible on: Windows 7 x64 & x86, Windows 8 x64, Mac OS X 10.8.5

Steps to reproduce:
1. Open Firefox with a clean profile and open a 4K youtube video (e.g. https://www.youtube.com/watch?v=-YGDyPAwQz0).
2. From the settings button select "2160p4K".

Expected results:
Video plays at 4K resolution without performance issues.

Actual results:
Video plays at 4K resolution but with visible performance issues (very low framerate). It seems it's the browser that slows down as other tabs are also very slow to interact with (e.g. loading content, scrolling).

Notes:
- Video plays with MSE enabled (Dash=Yes)
- CPU does not seem to go up while experiencing these issues (CPU use is easily under 10% and does not go up)
- Firefox 36 has the same issues when setting "media.mediasource.enabled"=true, while 35.0.1 has some issues buffering so cannot really determine whether it also has this issue or not
- Flash plays 4K resolution without issues
Tracking as this looks like a regression when switching from the YouTube Flash player to MSE playback.

ni rillian for prioritization.
status-firefox37: --- → affected
status-firefox38: --- → affected
status-firefox39: --- → affected
tracking-firefox37: --- → +
tracking-firefox38: --- → +
tracking-firefox39: --- → +
Flags: needinfo?(giles)
Blocks: 778617
Flags: needinfo?(giles)
Florin, can you share the graphics hardware/driver info from your reproduction machines? e.g. the 'Graphics' section of about:support? If it's universal it might be a bottleneck in our painting code, but playback performance is also going to depend on the system's graphics.

What resolution does 'stats for nerds' show with 4k flash? I was under the impression YouTube limited their flash offerings to lower resolutions and framerates.
Flags: needinfo?(florin.mezei)
I also see this. The video buffers considerably and triggers the warning script on YouTube that monitors playback.

Graphics
Adapter Description	AMD Radeon HD 7900 Series
Adapter Description (GPU #2)	Intel(R) HD Graphics 4000
Adapter Drivers	aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
Adapter Drivers (GPU #2)	igdumdim64 igd10iumd64 igd10iumd64 igdumdim32 igd10iumd32 igd10iumd32
Adapter RAM	3072
Adapter RAM (GPU #2)	Unknown
ClearType Parameters	D [ Gamma: 2200 Pixel Structure: R ClearType Level: 50 Enhanced Contrast: 100 ] D [ Gamma: 2200 Pixel Structure: R ClearType Level: 50 Enhanced Contrast: 100 ] D [ Gamma: 2200 Pixel Structure: R ClearType Level: 100 Enhanced Contrast: 50 ]
Device ID	0x6798
Device ID (GPU #2)	0x0162
Direct2D Enabled	true
DirectWrite Enabled	true (6.3.9600.17415)
Driver Date	11-20-2014
Driver Date (GPU #2)	12-18-2014
Driver Version	14.501.1003.0
Driver Version (GPU #2)	10.18.10.4061
GPU #2 Active	false
GPU Accelerated Windows	1/1 Direct3D 11 (OMTC)
Subsys ID	32111682
Subsys ID (GPU #2)	0000000c
Vendor ID	0x1002
Vendor ID (GPU #2)	0x8086
WebGL Renderer	Google Inc. -- ANGLE (AMD Radeon HD 7900 Series Direct3D11 vs_5_0 ps_5_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d 1.1
AzureContentBackend	direct2d 1.1
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0


(In reply to Ralph Giles (:rillian) from comment #2)
> 
> What resolution does 'stats for nerds' show with 4k flash? I was under the
> impression YouTube limited their flash offerings to lower resolutions and
> framerates.

YouTube limits 60FPS videos, not 4K. 4K in Flash works but it drops a few frames occasionally.

Comment 4

2 years ago
I can reproduce this with this video on both my Mac and Windows machines. 

On my Mac, I tried a number of other 4K videos and couldn't find a similar problem. On the PC, I found this issue with several other videos. Here are a couple.

https://www.youtube.com/watch?v=Cx6eaVeYXOs
https://www.youtube.com/watch?v=iNJdPyoqt8U
(Reporter)

Comment 5

2 years ago
(In reply to Ralph Giles (:rillian) from comment #2)
> Florin, can you share the graphics hardware/driver info from your
> reproduction machines? e.g. the 'Graphics' section of about:support? If it's
> universal it might be a bottleneck in our painting code, but playback
> performance is also going to depend on the system's graphics.

This does indeed seem like a universal problem (especially since it was confirmed in comment 3 and comment 4). Just to confirm, here's the graphics we used to reproduce this:
- Windows 7 x64 - nVIDIA GeForce GT 610 (with the latest driver delivered by NVIDIA GeForce Experience)
- Windows 7 x86 - Intel HD 2500
- Windows 8 x64 - AMD Radeon HD 5450

> What resolution does 'stats for nerds' show with 4k flash? I was under the
> impression YouTube limited their flash offerings to lower resolutions and
> framerates.

Resolution: 3840 x 2160 @ 30
Flags: needinfo?(florin.mezei)

Comment 6

2 years ago
I tested this on other browsers. Chrome has the same performance issues on both the Mac and Windows. I tried a few 4K videos. I can't reproduce the problem on Safari.
This kind of looks like a buffering issue. Internet Explorer 11 is a lot smoother but suffers from occasional buffering according to the network monitor in Task Manager. 

Every time the scrubber reaches the end of the buffer, the video stalls while waiting for the next chunk. During that time, network activity bottoms out and then spikes again. When the spike happens, the video resumes. On Chromium based browsers, the buffer happens consistently, but there seems to be an acceleration issue since the video takes significantly more CPU to decode and play which makes sense why the playback is choppy.

Updated

2 years ago
Priority: -- → P2
I'm marking this as wontfix for 37. Although this may come across as a regression from the user perspective, there is a small user base for 4K videos currently and this is not going to block the MSE release. I would really like to see this fixed in 38 and am leaving this as tracking 38 as a result.
status-firefox37: affected → wontfix
(Assignee)

Comment 9

2 years ago
Created attachment 8578310 [details] [diff] [review]
[WIP] Upload video frames on the client side
(Assignee)

Comment 10

2 years ago
Created attachment 8578411 [details] [diff] [review]
[WIP] Move image data copy off the decoder thread

This one seems to make a pretty noticeable improvement, but still not enough to maintain 30fps on my machine.
(Assignee)

Comment 11

2 years ago
Created attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage
Assignee: nobody → matt.woodrow
Attachment #8578310 - Attachment is obsolete: true
Attachment #8578411 - Attachment is obsolete: true
(Assignee)

Updated

2 years ago
Attachment #8579026 - Flags: review?(nical.bugzilla)
(Assignee)

Comment 12

2 years ago
Created attachment 8579027 [details] [diff] [review]
Part 2: Move image data copy off the decoder thread

The existing code (which ended up creating a SharedPlanarYCbCbrImage) made a copy of the image data while blocking the decoder thread.

This is a large copy for 4k video, and we want to avoid blocking the decoder for that long.

This version will instead trigger the PlanarYCbCr path in ImageClient::UpdateImage, and do the copy on the ImageBridge thread.

This results in a fairly large fps improvement with 4k video on my machine.
Attachment #8579027 - Flags: review?(nical.bugzilla)
(Assignee)

Updated

2 years ago
Attachment #8579027 - Flags: review?(cpearce)
(Assignee)

Comment 13

2 years ago
Created attachment 8579028 [details] [diff] [review]
Part 3: Upload on the client side

This patch makes us upload to d3d directly from the ImageBridge thread, instead of allocating and copying into memory/shmem (only to do the same upload on the compositor).

As well as removing a copy, this increases the amount of time between upload and compositing, so should reduce the amount we need to block waiting for it to complete.

roc is currently working on patches that will provide a set of video frames to the compositor instead of the current one, so that should improve the upload times futher.

With all 3 patches applied, I can now play software decoded 4k video without any frame drops on my machine.
Attachment #8579028 - Flags: review?(nical.bugzilla)
Comment on attachment 8579027 [details] [diff] [review]
Part 2: Move image data copy off the decoder thread

Review of attachment 8579027 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/fmp4/wmf/WMFVideoMFTManager.cpp
@@ +402,1 @@
>                                              mImageContainer,

Nit: indentation of hanging args don't line up with '('.
Attachment #8579027 - Flags: review?(cpearce) → review+
Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

Review of attachment 8579026 [details] [diff] [review]:
-----------------------------------------------------------------

yay!
Attachment #8579026 - Flags: review?(nical.bugzilla) → review+
Comment on attachment 8579027 [details] [diff] [review]
Part 2: Move image data copy off the decoder thread

Review of attachment 8579027 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/layers/IMFYCbCrImage.h
@@ +16,5 @@
> +class IMFYCbCrImage : public PlanarYCbCrImage
> +{
> +public:
> +  IMFYCbCrImage(IMFMediaBuffer* aBuffer, IMF2DBuffer* a2DBuffer)
> +    : PlanarYCbCrImage(nullptr)

This is a bit sketchy. Some of PlanarYCbCrImage's methods expect to have a non-null RecycleBin for example. I'd feel better if you override the PlanarYCbCrImage methods that don't work with this class to MOZ_ASSERT.
Attachment #8579027 - Flags: review?(nical.bugzilla) → review+
Comment on attachment 8579028 [details] [diff] [review]
Part 3: Upload on the client side

Review of attachment 8579028 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/layers/d3d11/TextureD3D11.h
@@ +77,5 @@
>    bool mNeedsClear;
>    bool mNeedsClearWhite;
>  };
>  
> +class SharedYCbCrTextureClientD3D11 : public TextureClient

nit: how about calling this one DXGIYCbCrTextureClientD3D11 (TextureClients are all shared and the TextureHost is names like this)?
Attachment #8579028 - Flags: review?(nical.bugzilla) → review+
Cool stuff.
(Assignee)

Comment 19

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/a3c8482c41f6
https://hg.mozilla.org/integration/mozilla-inbound/rev/215046aa1f38
https://hg.mozilla.org/integration/mozilla-inbound/rev/d6242b24bc47
(Assignee)

Comment 20

2 years ago
Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

Approval Request Comment
[Feature/regressing bug #]: HTML5 Video
[User impact if declined]: Poor fps (especially with 4k video) when using software decoding
[Describe test coverage new/current, TreeHerder]: Lots of manual testing, not covered by treeherder.
[Risks and why]: Fairly low risk change to the way we upload video frames, much faster.
[String/UUID change made/needed]: None.
Attachment #8579026 - Flags: approval-mozilla-beta?
Attachment #8579026 - Flags: approval-mozilla-aurora?
We want this patch because it improves the performance of software decoding across the board. This is especially important because a most AMD cards fall back to software decoding.

Declining it will not break anything with YouTube set to auto mode bit it will tend to use a lower resolution.
https://hg.mozilla.org/mozilla-central/rev/a3c8482c41f6
https://hg.mozilla.org/mozilla-central/rev/215046aa1f38
https://hg.mozilla.org/mozilla-central/rev/d6242b24bc47
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox39: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla39
Comment on attachment 8579026 [details] [diff] [review]
Part 1: Remove ISharedImage

We're going to take some risk on this patch that has only recently landed on m-c for the performance benefit on YouTube with MSE. Worst case we'll need to pull this fix out of the RC on Monday. Beta+ Aurora+

Florin - Can you please include YouTube playback on AMD devices in your exploratory testing for Beta 7?

Anthony - As you have a 4K monitor, can you please test 4K video playback with Beta 7?
Flags: needinfo?(florin.mezei)
Flags: needinfo?(ajones)
Attachment #8579026 - Flags: approval-mozilla-beta?
Attachment #8579026 - Flags: approval-mozilla-beta+
Attachment #8579026 - Flags: approval-mozilla-aurora?
Attachment #8579026 - Flags: approval-mozilla-aurora+
status-firefox37: wontfix → affected
(Reporter)

Comment 24

2 years ago
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #23)
> Florin - Can you please include YouTube playback on AMD devices in your
> exploratory testing for Beta 7?

Will do.
Flags: needinfo?(florin.mezei)
https://hg.mozilla.org/releases/mozilla-beta/rev/c1356c27fa1b
https://hg.mozilla.org/releases/mozilla-beta/rev/07e266d45703
https://hg.mozilla.org/releases/mozilla-beta/rev/0c23dcbc6bf7
status-firefox37: affected → fixed
https://hg.mozilla.org/releases/mozilla-aurora/rev/126db33d2905
https://hg.mozilla.org/releases/mozilla-aurora/rev/77b336a4baf6
https://hg.mozilla.org/releases/mozilla-aurora/rev/22a4edc0cfa8
status-firefox38: affected → fixed
I've tested the patch with nVidia, AMD and Intel graphics cards and they all work with a 1080 monitor but all the graphics cards struggle to drive a 4k monitor. We get similar results with software decoding to Chrome which uses VP9 software decoding. I'll do further testing with 37b7.
(Assignee)

Updated

2 years ago
Depends on: 1145513
(Reporter)

Updated

2 years ago
Flags: qe-verify+
I pushed https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f to fixup mingw / case sensitive file systems unfortunately with the wrong bug number oops
I pushed https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f to fix mingw / case sensitive file systems (with the wrong bug number oops)
https://hg.mozilla.org/integration/mozilla-inbound/rev/824009fbc42f (wrong bug number oops)
and git bz sucks sorry
https://hg.mozilla.org/mozilla-central/rev/824009fbc42f
Flags: needinfo?(ajones)

Comment 33

2 years ago
Looks like this is the probable cause of a crash increase in 37.0b7, tracked in bug 1146315, bug 1116812 and bug 1146313.
(Assignee)

Comment 34

2 years ago
Backed part 3 of this out of branches because of the regressions, will back out of m-c when it reopens.

https://hg.mozilla.org/releases/mozilla-aurora/rev/5fe9f09fe578
https://hg.mozilla.org/releases/mozilla-beta/rev/2592523e1eb0
https://hg.mozilla.org/releases/mozilla-release/rev/6d7a2555b021
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
status-firefox37: fixed → affected
status-firefox38: fixed → affected
status-firefox39: fixed → affected
this caused a slight (2%) win7 private bytes perf regression:
http://alertmanager.allizom.org:8080/alerts.html?rev=41a819532cfb&showAll=1&testIndex=0&platIndex=0

the regression went away after this was backed out.  I saw it on aurora, not sure about beta, release.
Now that part 3 of this bug has been backed out of 37 it will not be fixed in the 37 release.
status-firefox37: affected → wontfix
(Reporter)

Comment 37

2 years ago
While testing today Firefox 37 RC 2 (BuildID=20150326190726) on Windows 7 x64, I noticed that despite slowing down the browser, 4k videos do not take much CPU and memory. For my machine it's usually ~5-10% CPU and ~300-400 Mb.

Sometimes though, I noticed that the CPU and memory usage spike up to ~75% CPU and ~1Gb memory very quickly. I saw this happen in a fairly consistent manner when I left the 4K video play in a tab and then just opened/navigated/closed some other pages in other tabs. After 3-5 minutes the CPU and memory usage just shoot up.

I feel the issue described above is most likely related to this bug so I did not file a separate bug. I think we should revisit this scenario after we get a fix for this bug. Please let me know if you think this may be a separate issue that needs to be filed separately.
status-firefox40: --- → affected
Target Milestone: mozilla39 → ---
Trevors, could you also backout it for 38? Thanks
Flags: needinfo?(tbsaunde+mozbugs)
Sorry for the confusion, Kairo mentioned that it has been done already.
Flags: needinfo?(tbsaunde+mozbugs)

Comment 40

2 years ago
FWIW, I do not see any reason for backing this out in 38, the jump in crashes I saw in comment #33 has been mitigated by some blocklisting for the few affected gfx adapters.
(Assignee)

Comment 41

2 years ago
I think we can close this now, we don't have anything else actionable to do here.

Sylvestre, can we stop tracking this?
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED
KaiRo, I think also we kept it backed out because of comment 35 - performance regressions noted in talos.

Matt, it sounds like we may have fixed some broad issues but are left with an intermittent edge case in Win 7 x64. Would that be useful, to keep it clear that we still have an issue?
Flags: needinfo?(matt.woodrow)
Too late anyway for 38.
tracking-firefox38: + → ---
Dropping tracking for this for 39 since the main issue seems to be solved. Florin, if you're still seeing the problem in comment 37 please file a new bug.
tracking-firefox39: + → ---
Flags: needinfo?(matt.woodrow)
You need to log in before you can comment on or make changes to this bug.