1859880 - Investigate using VPXDecoder for decoding VP9 in WebRTC

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, task, P2)

Description

[Andreas Pehrson [:pehrsons]]

There are several issues with decoding VP9 SVC streams with MediaDataDecoder codecs. Both FFmpegVideoDecoder and AppleVTDecoder have issues reported on them.

We could use libwebrtc's libvpx decoder as that is basically the libwebrtc reference codec. It runs in the content process however.

The MediaDataDecoder stack also has VPXDecoder which IIRC can handle SVC just as well, and would run in the RDD process. Let's verify this and see if we can make it the default in webrtc.

Comment 1

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1859880 - Don't use AppleVTDecoder for VP9 in WebRTC. r?#media-playback-reviewers!, r?#webrtc-reviewers!

There are reports that Apple's VideoToolbox VP9 decoder, which is always in
hardware, cannot always decode SVC streams properly. For now disable it for all
low-latency VP9 decoding -- this is mainly WebRTC.

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1859880 - Don't use FFmpegVideoDecoder for VP9 in WebRTC. r?#media-playback-reviewers!, r?#webrtc-reviewers!

There are reports that FFmpegVideoDecoder cannot always decode SVC streams
properly. For now disable it for all low-latency VP9 decoding -- this is mainly
WebRTC.

Comment 3

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1859880 - Fix hw decode of WebRTC streams with WMF on Windows. r?#media-playback-reviewers!

This patch should fix hw decode with WebRTC for all codecs supported by WMF.

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1859880 - Don't use WMFVideoMFTManager for VP9 in WebRTC. r?#media-playback-reviewers!,#webrtc-reviewers!

WMFVideoMFTManager cannot decode SVC streams with multiple spatial layers.
For now disable it for low-latency VP9 decoding -- this is mainly WebRTC.

Depends on D191447

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dfc1f56553ae
Don't use AppleVTDecoder for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib
https://hg.mozilla.org/integration/autoland/rev/6691b076a6a2
Don't use FFmpegVideoDecoder for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib
https://hg.mozilla.org/integration/autoland/rev/8b79442bf220
Fix hw decode of WebRTC streams with WMF on Windows. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/2653ea413889
Don't use WMFVideoMFTManager for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib

Comment 6

[Norisz Fay [:noriszfay]]

Backed out for causing bustages on FFmpegDecoderModule.h

Backout link

Push with failures

Failure log

Comment 7

[Andreas Pehrson [:pehrsons]]

A rebase issue.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/015c26be53dd
Don't use AppleVTDecoder for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib
https://hg.mozilla.org/integration/autoland/rev/f3322110cf34
Don't use FFmpegVideoDecoder for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib
https://hg.mozilla.org/integration/autoland/rev/7e8153f9bf53
Fix hw decode of WebRTC streams with WMF on Windows. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/0546c85daa8d
Don't use WMFVideoMFTManager for VP9 in WebRTC. r=media-playback-reviewers,webrtc-reviewers,padenot,jib

Comment 9

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/015c26be53dd
https://hg.mozilla.org/mozilla-central/rev/f3322110cf34
https://hg.mozilla.org/mozilla-central/rev/7e8153f9bf53
https://hg.mozilla.org/mozilla-central/rev/0546c85daa8d