1147748 - [video perf] Choppy playback with YouTube 4K videos (Plays worse with DXVA)

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[Trevor Rowbotham [:rowbot]]

Attachment: profile.zip

Playback of 4K videos on YouTube using the HTML5 player are sometimes really choppy and unwatchable.  I have attached a profile, so hopefully that is of some use to figure out what is going on.

Adapter Description	NVIDIA GeForce GTX 660
Adapter Drivers	nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um
Adapter RAM	2048
Asynchronous Pan/Zoom	wheel input enabled
Device ID	0x11c0
Direct2D Enabled	true
DirectWrite Enabled	true (6.2.9200.16492)
Driver Date	3-13-2015
Driver Version	9.18.13.4788
GPU #2 Active	false
GPU Accelerated Windows	1/1 Direct3D 11 (OMTC)
Subsys ID	30693842
Vendor ID	0x10de
WebGL Renderer	Google Inc. -- ANGLE (NVIDIA GeForce GTX 660 Direct3D11 vs_5_0 ps_5_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d 1.1
AzureContentBackend	direct2d 1.1
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0

Comment 1

[Trevor Rowbotham [:rowbot]]

Attachment: gpu_usage.jpg

Shows my gpu usage during the choppy playback.

Comment 2

[Nicolas Silva [:nical]]

Which version of gecko? Matt Woodrow recently landed some big improvements for 4k video playback.

Comment 3

[Trevor Rowbotham [:rowbot]]

Sorry, should have specified that earlier.  I am using the latest nightly on Windows 7.  Playback is still choppy even after Bug 1138967 and Bug 1131638.  I also tested a build from 2015-03-22 which is from before part 3 of Bug 1138967 was backed out and it was choppy in that build as well.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

Can you try out the videos here: http://people.mozilla.org/~jmuizelaar/4k/

Do they both play poorly? You may need to download them to get them to work well as it seems the normal video streaming code may not work that well.

Comment 5

[Trevor Rowbotham [:rowbot]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #4)
> Can you try out the videos here: http://people.mozilla.org/~jmuizelaar/4k/
> 
> Do they both play poorly? You may need to download them to get them to work
> well as it seems the normal video streaming code may not work that well.

The mp4 video seems to play fairly well for the most part.  There were only a few noticeable hiccups while playing it.  The webm video was terrible, it would play for a few seconds, pause for 5 seconds then resume playback 5 seconds ahead of when it paused.  The YouTube video linked in this bug is using mp4 though.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

(In reply to Trevor Rowbotham from comment #5)
> (In reply to Jeff Muizelaar [:jrmuizel] from comment #4)
> > Can you try out the videos here: http://people.mozilla.org/~jmuizelaar/4k/
> > 
> > Do they both play poorly? You may need to download them to get them to work
> > well as it seems the normal video streaming code may not work that well.
> 
> The mp4 video seems to play fairly well for the most part.  There were only
> a few noticeable hiccups while playing it.  The webm video was terrible, it
> would play for a few seconds, pause for 5 seconds then resume playback 5
> seconds ahead of when it paused.  

Yeah, it seems broken. I've filed bug 1147888 about that.

> The YouTube video linked in this bug is
> using mp4 though.

Does the downloaded one play better than https://www.youtube.com/watch?v=goQMXfiS-dA?

How about setting media.windows-media-foundation.use-dxva to false. What impact does that have on playback?

Comment 7

[Trevor Rowbotham [:rowbot]]

Attachment: gpu_usage_smooth_playback.jpg

Just an observation.  Sometimes, after refreshing the page when viewing the YouTube video in this bug my gpu usage looks like this (see attachment).  As you can see from this image, both GPU Load and Memory Controller Load are much lower than what is seen in comment #0.  Additionally, GPU Memory usage is less than half of what it is when the video playback is choppy.  The other notable difference is that Video Engine Load is at 0% when it is smooth.  If I am not mistaken, the Video Engine Load measurement represents hardware accelerated decoding (e.g. DXVA).

Comment 8

[Trevor Rowbotham [:rowbot]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #6)
> (In reply to Trevor Rowbotham from comment #5)
> > (In reply to Jeff Muizelaar [:jrmuizel] from comment #4)
> > > Can you try out the videos here: http://people.mozilla.org/~jmuizelaar/4k/
> > > 
> > > Do they both play poorly? You may need to download them to get them to work
> > > well as it seems the normal video streaming code may not work that well.
> > 
> > The mp4 video seems to play fairly well for the most part.  There were only
> > a few noticeable hiccups while playing it.  The webm video was terrible, it
> > would play for a few seconds, pause for 5 seconds then resume playback 5
> > seconds ahead of when it paused.  
> 
> Yeah, it seems broken. I've filed bug 1147888 about that.
> 
> > The YouTube video linked in this bug is
> > using mp4 though.
> 
> Does the downloaded one play better than
> https://www.youtube.com/watch?v=goQMXfiS-dA?
> 
> How about setting media.windows-media-foundation.use-dxva to false. What
> impact does that have on playback?

The downloaded video does play much better than watching it on YouTube.

Setting media.windows-media-foundation.use-dxva to false seems to make this much better.  Explicitly setting this to false, my GPU usuage looks like the image from comment #7, which I guess means that after a page refresh Firefox was disabling DXVA automatically, but it didn't auto disable it every time.

It seems strange that DXVA would cause performance to be worse.  Bug 1133382 probably accounts for why I am seeing more than double the GPU memory used with DXVA enabled.  Could the additional color conversions be the cause of the DXVA performance issues?

Comment 9

[Trevor Rowbotham [:rowbot]]

Just thinking out loud here.  I remember when D2D1.1 first landed, it wasn't available for canvas to use and there was a mismatch where content used D2D1.1 and canvas had to use D2D1.0 and this caused a lot of performance regressions.  My browser is using D3D11 for content and DXVA uses D3D9; Is it possible that this mismatch between backends is causing the poor playback performance with DXVA enabled? Bug 875247 was originally going to implement DXVA in the D3D11 backend, but it got WONTFIXED.

Comment 10

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Hm, seems like bug 875247 should get reopened; mwoodrow even rebased the patch recently.  Even though it's not a solution for Vista (who cares) or Win 7, it would be no worse than now; Win 8+ users would benefit.  Even if it's not a full fix for the performance issues, only using D3D11 throughout would be better than mixing devices since I think this is one of the last users of D3D9 when D3D11 is available.

Comment 11

[Jeff Muizelaar [:jrmuizel]]

Certainly having it around for experimentation would be valuable.

Comment 12

[Jeff Muizelaar [:jrmuizel]]

Does the video play well in Chrome and IE? What does Chrome's about:gpu give?

Comment 13

[Trevor Rowbotham [:rowbot]]

Attachment: chrome about:gpu

Chrome reports that video decoding is being hardware accelerated, however, their GPU log is giving me the following message when I play a video on YouTube:

[25352:16388:0326/161015:ERROR:gpu_video_decode_accelerator.cc(272)] : NOT IMPLEMENTEDHW video decode acceleration not available.

It would appear that the decoding of videos are not being hardware accelerated in Chrome.  At least not on my computer.  The video plays smoothly in Chrome just like it does in Firefox when DXVA is disabled.  I don't think I can force DXVA on in Chrome to test how it performs in that situation.

It is difficult to say how IE performs.  IE on my computer is using Flash.  Disabling flash only gives me 360p and 720p as options in the HTML5 player on YouTube.  I tested the flash player in Firefox and that seemed to work ok and I believe Flash uses DXVA.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

(In reply to Trevor Rowbotham from comment #13)
> 
> It is difficult to say how IE performs.  IE on my computer is using Flash. 
> Disabling flash only gives me 360p and 720p as options in the HTML5 player
> on YouTube.  I tested the flash player in Firefox and that seemed to work ok
> and I believe Flash uses DXVA.

Can you load the mp4 file in IE directly?

Comment 15

[Matt Woodrow (:mattwoodrow)]

It's interesting that loading the mp4 file directly would perform differently, we should be using the same decoder pipeline in both cases.

The patches I landed (all are still in nightly) only affect non-DXVA performance, so it's nice to know that performance in that case is reasonable.

Comment 16

[Trevor Rowbotham [:rowbot]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #15)
> It's interesting that loading the mp4 file directly would perform
> differently, we should be using the same decoder pipeline in both cases.
> 
> The patches I landed (all are still in nightly) only affect non-DXVA
> performance, so it's nice to know that performance in that case is
> reasonable.

It's possible that MSE has some affect on it.  Loading a file directly in a plain <video> tag would avoid MSE, would it not?

Comment 17

[Trevor Rowbotham [:rowbot]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> Can you load the mp4 file in IE directly?

From irc:
18:26   trevor_r	IE can't play the file.  Thinks the video format isn't supported
18:26	jrmuizel	hmm, weird
18:27	trevor_r	It won't play your Bosnian Landscape video either
18:29	trevor_r	That's interesting.  Windows Media Player won't play the file either, but I know Firefox relies on Windows Media Player codecs to play mp4 files and the file plays just fine in Firefox
18:30	jrmuizel	it may be the container format or perhaps the file extension??
18:31	trevor_r	Maybe
18:38	trevor_r	Opera doesn't like the file either, but it plays in chrome

Comment 18

[Jeff Muizelaar [:jrmuizel]]

(In reply to Trevor Rowbotham from comment #13)
> Created attachment 8584053 [details]
> chrome about:gpu
> 
> Chrome reports that video decoding is being hardware accelerated, however,
> their GPU log is giving me the following message when I play a video on
> YouTube:
> 
> [25352:16388:0326/161015:ERROR:gpu_video_decode_accelerator.cc(272)] : NOT
> IMPLEMENTEDHW video decode acceleration not available.

That's very confusing. From my reading of the chrome code, that message should only be output when running on a version of Windows before Windows 7.

Comment 19

[Jeff Muizelaar [:jrmuizel]]

Interesting fact according to this page the maximum resolution on win7 is 1920 × 1088. https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815%28v=vs.85%29.aspx

Comment 20

[Jeff Muizelaar [:jrmuizel]]

It may be worth trying to characterize the performance of your gpu using DXVA Checker (http://bluesky23.yukishigure.com/en/DXVAChecker.html)

This thread has a bunch of info about other people trying to do the same on various gpus:
http://forum.doom9.org/showthread.php?t=163110

Comment 21

[Trevor Rowbotham [:rowbot]]

> That's very confusing. From my reading of the chrome code, that message should only be output when
> running on a version of Windows before Windows 7.
Yes, that is odd. Since you mentioned this, I've been googling it, but to no avail.  I thought maybe my card or driver was being blacklisted, but disabling their blacklist in about:flags didn't seem to make a difference.  While testing, I noticed that the ads on YouTube got DXVA accelerated, but not the actual video.

> Interesting fact according to this page the maximum resolution on win7 is 1920 × 1088.
> https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815%28v=vs.85%29.aspx
I was just looking at that link about 10 minutes before you posted this and noticed that same info.  I guess its possible that DXVA is having a meltdown with the 4K resolution.  Although, I don't have any problems with DXVA and 1440p.

Comment 22

[Trevor Rowbotham [:rowbot]]

Attachment: dxva_checker.png

According to [1] my video card uses the VP5 hardware video decoder which has support for decoding H.264 with a resolution of up to 4032x4080 and MPEG-1/MPEG-2 with a resolution of up to 4032x4048 pixels.  Since 4K has a resolution of 4096x2160, I guess my video card isn't capable of hardware decoding 4K since the max width it supports is 4023.  Based on [1] you need at least a GTX 745 or higher for 4K hardware video decoding support.  Assuming that wikipedia is correct, does this make sense to everyone?

That DXVA checker program also seems to report that my video card only supports a maximum of QFHD, which has a resolution of 3840x2160.  This still doesn't explain why Chrome is giving me an error about hardware decoding not being available since I get this message on any YouTube video I try.  Interestingly enough, I don't get that error when playing a local video file in Chrome.  None of the locally played 4K files in Chrome will use DXVA, but it will use it on other non-4K files that are played locally.

[1] http://en.wikipedia.org/wiki/Nvidia_PureVideo#Table_of_GPUs_containing_a_PureVideo_SIP_block

Comment 23

[Jeff Muizelaar [:jrmuizel]]

DXVA Checker supports checking whether the device creation at a particular resolution works. From the 'Decoder Device' tab click 'Device Creator...' select H264_VLD_NoFGT as the device and try the different resolutions and report the results.

Comment 24

[Trevor Rowbotham [:rowbot]]

Attachment: dxva_checker_trace_log.png

The highest resolution that DXVA Checker allowed me to test was 3840x2160(QFHD).  Using the trace log, it would appear that Firefox successfully creates a DXVA device for 4K videos.

Comment 25

[Trevor Rowbotham [:rowbot]]

Just some notes from looking at Chrome bugs:

According to [1], Chrome disables DXVA for resolutions > 1080p on Windows due to faulty faulty drivers and expensive texture copying.

[2] seems to have the same thought process I had in comment #9 where the D3D11 -> D3D9 device sharing may be causing performance issues.

From what I can gather [3] seems to suggest that hardware decoding using windows media foundation has issues with high profile h.264 videos.  [3] seems targeted at flash though, but other commenters seem to indicate that it affects playback using the HTML5 player as well.

According to [4], there seems to be a bug in DXVA where it reports that it is ready for more work before it has finished with its current work resulting in it getting more work than it can handle.

[1] https://code.google.com/p/chromium/issues/detail?id=464774
[2] https://code.google.com/p/chromium/issues/detail?id=439764#c14
[3] https://code.google.com/p/chromium/issues/detail?id=337606#c203
[4] https://code.google.com/p/chromium/issues/detail?id=150925

Comment 26

[Trevor Rowbotham [:rowbot]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #18)
> (In reply to Trevor Rowbotham from comment #13)
> > Created attachment 8584053 [details]
> > chrome about:gpu
> > 
> > Chrome reports that video decoding is being hardware accelerated, however,
> > their GPU log is giving me the following message when I play a video on
> > YouTube:
> > 
> > [25352:16388:0326/161015:ERROR:gpu_video_decode_accelerator.cc(272)] : NOT
> > IMPLEMENTEDHW video decode acceleration not available.
> 
> That's very confusing. From my reading of the chrome code, that message
> should only be output when running on a version of Windows before Windows 7.

Figured out that this error message appears whenever Chrome plays a Webm file.  YouTube in Chrome serves Webm files by default and hardware decoding of Webm is nonexistent, which is why I was seeing this message when trying to test 4K YouTube videos.

Comment 27

[Ralph Giles (:rillian)]

Let's just block dvxa for high resolutions on this device.

Comment 28

[Ralph Giles (:rillian)]

Does that make sense to you, Matt?

Comment 29

[Jeff Muizelaar [:jrmuizel]]

Shouldn't we just not use DXVA for resolutions higher than supported on all devices?

Comment 30

[Matt Woodrow (:mattwoodrow)]

I believe chrome disables DXVA for 4k video on all devices, maybe we should just do the same.

Comment 31

[Trevor Rowbotham [:rowbot]]

So, I just retested this and I am having great difficulty getting choppy playback on 4K videos now.  I have updated my graphics card drivers since reporting this bug, which could explain why I'm not seeing this anymore.  Looking through the release notes for the driver, nothing seems to jump out that says it would have solved a problem like this, but it is possible that it was a driver bug.

I'll try installing the previous driver tomorrow to see if I can confirm if this was a driver bug or not.

What I am now observing is that on the release build the video seems to play smoothly with DXVA, however, if the video has to stop to buffer or if I try to seek, DXVA gets switched off.  On nightly, 99% of the time, DXVA gets switched off immediately when switching to 4K, however, the few times I've gotten it to play with DXVA on nightly, it seemed to play smoothly.  Like on the release build, DXVA in nightly would get switched off as soon as I seeked or the video stopped to buffer.  I don't recall actually testing this on release at the time I reported the bug so I'm not sure if what I am seeing on the release build now is different from what was occurring then.  Bug 1131638 is likely the cause for DXVA getting switched off unexpectedly due to buffering/seeking since there is probably some weirdness going on with frames when networking gets involved and Firefox interprets this as invalid frames.

In regards to my own comments in comment #22, YouTube is serving 4K videos with a resolution of 3840x2160, which is a resolution that my card can handle, so its not an issue of my card not supporting hardware decoding at that resolution.

When I was testing 4K videos locally, I noticed that media players like VLC did not use DXVA when attempting to play those 4K video files.  Windows Media Player seems to refuse to play any 4K video on Windows 7 (Windows Media Player on my Windows 8.1 VM will play 4K videos).

Jeff, the link in comment #19 states that the 1920x1088 resolution limit for Windows 7 applies to both software and DXVA decoding.  That link also says DXVA can be used if the underlying hardware supports the resolution.  So, I'm not sure if that information is really all that meaningful since both the software and hardware decoders supposedly have the same limitation.

Disabling DXVA when playing 4K video is probably the safest thing to do here, though I hate to lose support for hardware acceleration.  I know that when I was experiencing the choppy playback, that I also experienced Bug 1138967, which is obviously bad from a user experience standpoint.  Disabling DXVA for 4K video would put Firefox on parity with Chrome and other native media players.

One concern of disabling DXVA for 4K video across the board is memory usage of the Firefox process.  I noticed that plugin-container uses about 250MB of memory with DXVA enabled and about 800MB+ with it disabled.

Comment 32

[Trevor Rowbotham [:rowbot]]

I installed my previous driver and got some mixed results.  I think what was happening with the previous driver is that the video card never wanted to leave its lowest power state so the gpu never increased its speed when playing the 4K video and it struggled as a result.  The gpu speed seems to increase with this new driver which likely explains why playback is smoother.  It is still probably a good idea to disable DXVA for 4K video at the very least.

Comment 33

[Jean-Yves Avenard [:jya]]

not an MSE issue

Comment 34

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Is this still an issue?

Comment 35

[Trevor Rowbotham [:rowbot]]

This is no longer an issue for me.  I'll go ahead and mark this fixed.