1389297 - VP9 4:4:4 playback fails with nVidia hardware acceleration on Windows

Status: RESOLVED DUPLICATE of bug 1322234

(Core :: Audio/Video: Playback, defect, P2)

Description

[Thomas Daede [:TD-Linux]]

A user reported that 4:4:4 VP9 fails on a Windows 1080ti system.

Test with the following video on Nightly:

https://bug1190939.bmoattachments.org/attachment.cgi?id=8643143

The video works fine with software decoding.

Comment 1

[Bulk Bug Changes for mreavy's org]

Mass change P1->P2 to align with new Mozilla triage process

Comment 2

[James Cheng[:JamesCheng]]

(In reply to Thomas Daede [:TD-Linux] from comment #0)
> A user reported that 4:4:4 VP9 fails on a Windows 1080ti system.
> 
> Test with the following video on Nightly:
> 
> https://bug1190939.bmoattachments.org/attachment.cgi?id=8643143
> 
> The video works fine with software decoding.

I can reproduce it only on Windows but my video card is not 1080ti.

Set "media.hardware-video-decoding.enabled = false" then close the firefox and restart it, this video plays well.

Comment 3

[James Cheng[:JamesCheng]]

Attachment: hw decode on

Upload a screenshot with "media.hardware-video-decoding.enabled" set as true.

Comment 4

[James Cheng[:JamesCheng]]

Hi jya,

Does WMF for VP9 decoder not support for YUV 4:4:4 chroma subsampling as H.264?

I cannot find related article on MSDN like H.264 ( https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815(v=vs.85).aspx ).

In addition,

How could I check the chroma subsampling in VP9?

Could you please indicate me that where is the code in gecko? Or any recommended tool that can easily parse the information inside WebM.

Thanks.

Comment 5

[Jean-Yves Avenard [:jya]]

I know the AMD doesn't. It only supports VP9 profile 0.

In any case, I'm fairly certain the compositor doesn't support those YUV formats, so it's a moot point.
We don't support YUV 4:4:4 with H264 either, I can't tell from your question if you think it is or not.
it's explicitly disabled there:
http://searchfox.org/mozilla-central/source/dom/media/platforms/wrappers/H264Converter.cpp#258

The check the subsampling, you'll need to decode the vp9 bytestream, like we did for H264 with the SPS decoder.
Chromium has a vp9 parser there: https://cs.chromium.org/chromium/src/media/filters/vp9_parser.cc
this only checks for change in colorspace however.
I haven't spent any time on the matter, so I don't know much about it.

It's something will need at some stage anyway if we want to use the VP9 MediaDataDecoder with WebRTC.

Comment 6

[James Cheng[:JamesCheng]]

Thanks,

I know we explicitly disabled YUV444 for H.264 and it is listed in MSDN so I can find the evidence that we should disable it.

But for VP9, I cannot find any clue on MSDN, it sounds supporting YUV444 should count on the capability of video card.

Just list the video information from ffprobe

ffprobe ~/Desktop/tos444.webm 
ffprobe version 3.4 Copyright (c) 2007-2017 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.37)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from '/Users/jacheng/Desktop/tos444.webm':
  Metadata:
    encoder         : Lavf56.36.100
  Duration: 00:00:02.00, start: 0.000000, bitrate: 448 kb/s
    Stream #0:0: Video: vp9 (Profile 1), yuv444p(tv), 640x268, SAR 1:1 DAR 160:67, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)

Comment 7

[Jan Goyvaerts]

Firefox is perfectly capable of playing VP9 4:4:4 video when VP9 hardware acceleration is not available at all, or if hardware acceleration is disabled in the settings.  The problem only occurs when VP9 hardware acceleration is available.  Firefox should be able to determine that the video uses 4:4:4 chroma subsampling and fall back to software playback if the hardware supports VP9 but not 4:4:4.

On my own (Windows) systems, VP9 4:4:4 videos appear black if hardware acceleration is enabled and the video card is a GTX 950 or GTX 1080.  On systems with older Nvidia cards, which do not support VP9 in hardware at all, VP9 4:4:4 videos always play correctly in Firefox.

What makes this issue annoying is that hardware acceleration is enabled by default even if the PC is more than powerful enough to handle playback in software.

4:4:4 subsampling is very useful for videos that show a lot of fine detail, such as screen recordings of applications showing small text.  Text on colored backgrounds (think syntax highlighting or "acrylic material") is much crisper with 4:4:4 subsampling.

Comment 8

[Jean-Yves Avenard [:jya]]

It's a catch 22, to determine the profile we have to decode the stream first. And if the hardware reports supporting vp9 windows will use it.
As the first decoding fail, we can't tell what the vp9 profile is.

We currently don't have fallback available from HW down to software decoding.

Comment 9

[Jan Goyvaerts]

If you can't/won't fall back to software when hardware decoding fails, then you'll have to add an extra step to the playback process.  Before actual playback starts, use the software decoder to determine the VP9 profile, perhaps by playing back the first frame to an off-screen buffer.  Then decide whether actual playback should use the hardware decoder or the software decoder.

Some videos, such as screen recordings with small text using ClearType, look so much better in 4:4:4 than in 4:2:0.  But we can't use it because with Firefox's default settings, everyone with an Nvidia card from the last few years in their PC won't be able to watch it.  Telling people to completely turn off hardware acceleration is not very user friendly.  So I'd really like to see this fixed.

Comment 10

[Jan Goyvaerts]

Chrome has the same issue, which I reported here: https://bugs.chromium.org/p/chromium/issues/detail?id=888873

Seems their plan is to disable hardware acceleration for VP9 profiles 1 and 3 because the Nvidia hardware does not correctly decode these, and does not return a proper error code either.

Comment 11

[Jean-Yves Avenard [:jya]]

This was done in bug 1322234