1928120 - VideoDecoder Does Not Support vp8

Status: RESOLVED FIXED

(Core :: Audio/Video: Web Codecs, defect)

Description

[Jason]

Steps to reproduce:

I tried to instantiate a VideoDecoder for "vp8" and get the error "The given encoding is not supported."

I check if the config to see if it is supported and the result says that it is.

var config = {
  codec: "vp8"
};
var result = await VideoDecoder.isConfigSupported(config);
console.log(result)

I then try to configure the decoder with that config

var handleFrame = function(frame) {};
var init = {
  output: handleFrame,
  error: (e) => {
    console.log(e.message);
  },
};
var config = {
  codec: "vp8", hardwareAcceleration: "no-preference"
};
var decoder = new VideoDecoder(init);
await decoder.configure(config);

Actual results:

I get an error

The given encoding is not supported.

Expected results:

The VideoDecoder should be configured to decode vp8 frames.

Comment 1

[C.M.Chang[:chunmin]]

...
[Child 58627: Main Thread]: D/WebCodecs Created a VideoInfo for decoder - VideoInfo: display size: -1x-1 stereo mode: 0image size: -1x-1 codec specific config: 0 bytesextra data: 0 bytesrotation: 0colors: COLOR_8color range: LIMITEDalpha present: false
...
[vp8 @ 0x1058afc00] [IMGUTILS @ 0x16b3ba5b0] Picture size 4294967295x4294967295 is invalid
...
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) state change: Configuring -> Error
...
[Child 58627: Main Thread]: E/WebCodecs VideoDecoder 10caae500 Close on error: NS_ERROR_DOM_ENCODING_NOT_SUPPORTED_ERR
...

From the log capturing when running the example code, the configuration fails because the size (width and height) is not set in advance. The size check fails in avcodec_open2 during initialization. Setting codedWidth and codedHeight along with the codec fix the issue, e.g. decoder.configure({codec: 'vp8', codedWidth: 320, codedHeight: 240}).

Setting a default size could cause trouble like bug 1872311. One approach is to defer the decoder creation until the decode data is provided. MediaChangeMonitor has a mechanism to delay the creation of the decoder. I'll investigate if this is a feasible option, or if there is a simpler way to fix this.

The full log is here:

[Child 58627: Main Thread]: D/WebCodecs VideoDecoder::IsConfigSupported, config: vp8
[Child 58627: Main Thread]: D/WebCodecs IsSupportedVideoCodec: vp8
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 ctor
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500, Configure: codec vp8
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 enqueues configure #4 (vp8)
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 starts processing configure #4 (vp8)
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 has no DecoderAgent to destroy
[Child 58627: Main Thread]: D/WebCodecs Create a VideoInfo from vp8hw accel: no-preference config
[Child 58627: Main Thread]: D/WebCodecs Created a VideoInfo for decoder - VideoInfo: display size: -1x-1 stereo mode: 0image size: -1x-1 codec specific config: 0 bytesextra data: 0 bytesrotation: 0colors: COLOR_8color range: LIMITEDalpha present: false
[Child 58627: Main Thread]: D/WebCodecs IsSupportedVideoCodec: vp8
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) ctor
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 creates DecoderAgent #4 @ 104a87920 and its shutdown-blocker
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 now blocks message-queue-processing
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) state change: Unconfigured -> Configuring
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) is creating a decoder (mime: video/vp8) - PreferSW: no, low-latency: no, create-decoder-params: CreateDecoderParams @ 16ba141b0: mConfig = TrackInfo: id:2 kind:main label: language: enabled:true trackid: 2 mimetype:video/vp8 duration:{0,1000000} media time:{0,1000000} crypto:None rendered externaly: false type:Video, mImageContainer = 10caea000, mError = null, mKnowsCompositor = 10aa0dd60, mCrashHelper = 0, mUseNullDecoder = no, mWrappers = MediaChangeMonitor, mType = 2, mOnWaitingForKeyEvent = no, mOptions = Default|KeepOriginalPts, mRate = 0.000000, mMediaEngineId = None, mTrackingId = None, mEncryptedCustomIdent = false
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[Child 58627: Main Thread]: D/MediaDecoder MediaChangeMonitor::Create, params = CreateDecoderParams @ 16ba13f40: mConfig = TrackInfo: id:2 kind:main label: language: enabled:true trackid: 2 mimetype:video/vp8 duration:{0,1000000} media time:{0,1000000} crypto:None rendered externaly: false type:Video, mImageContainer = 10caea000, mError = null, mKnowsCompositor = 10aa0dd60, mCrashHelper = 0, mUseNullDecoder = no, mWrappers = MediaChangeMonitor, mType = 2, mOnWaitingForKeyEvent = no, mOptions = Default|KeepOriginalPts, mRate = 0.000000, mMediaEngineId = None, mTrackingId = None, mEncryptedCustomIdent = false
[Child 58627: Main Thread]: D/MediaDecoder updated params = CreateDecoderParams @ 16ba13b80: mConfig = TrackInfo: id:2 kind:main label: language: enabled:true trackid: 2 mimetype:video/vp8 duration:{0,1000000} media time:{0,1000000} crypto:None rendered externaly: false type:Video, mImageContainer = 10caea000, mError = null, mKnowsCompositor = 10aa0dd60, mCrashHelper = 0, mUseNullDecoder = no, mWrappers = MediaChangeMonitor, mType = 2, mOnWaitingForKeyEvent = no, mOptions = Default|KeepOriginalPts, mRate = 0.000000, mMediaEngineId = None, mTrackingId = None, mEncryptedCustomIdent = false
[Child 58627: Main Thread]: D/MediaDecoder MediaChangeMonitor::CreateDecoder, current params = CreateDecoderParams @ 16ba13a50: mConfig = TrackInfo: id:2 kind:main label: language: enabled:true trackid: 2 mimetype:video/vp8 duration:{0,1000000} media time:{0,1000000} crypto:None rendered externaly: false type:Video, mImageContainer = 10caea000, mError = null, mKnowsCompositor = 10aa0dd60, mCrashHelper = 0, mUseNullDecoder = no, mWrappers = Empty, mType = 2, mOnWaitingForKeyEvent = no, mOptions = Default|KeepOriginalPts, mRate = 0.000000, mMediaEngineId = None, mTrackingId = None, mEncryptedCustomIdent = false
[Child 58627: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/vp8
[RDD 58620: MediaSupervisor #2]: D/PlatformDecoderModule FFmpeg decoder supports requested type 'video/vp8'
[RDD 58620: MediaSupervisor #2]: D/PlatformDecoderModule FFmpeg decoder supports requested type 'video/vp8'
[RDD 58620: MediaSupervisor #2]: D/FFmpegVideo FFVPX: FFmpegVideoDecoder::FFmpegVideoDecoder MIME video/vp8 Codec ID 139
[Child 58627: RemVidChild]: D/PlatformDecoderModule RemoteMediaDataDecoder[10c5e2460] ::RemoteMediaDataDecoder: 10c5e2460 is created
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) has created a decoder, now initialize it
[RDD 58620: MediaPDecoder #1]: D/FFmpegVideo FFVPX: FFmpegVideoDecoder, init, IsHardwareAccelerated=0
[RDD 58620: MediaPDecoder #1]: D/FFmpegVideo FFVPX: Initialising FFmpeg decoder
[RDD 58620: MediaPDecoder #1]: D/FFmpegVideo FFVPX:   codec vp8 : On2 VP8
[RDD 58620: MediaPDecoder #1]: D/FFmpegVideo FFVPX: Set get_buffer2 for customized buffer allocation
[vp8 @ 0x1058afc00] [IMGUTILS @ 0x16b3ba5b0] Picture size 4294967295x4294967295 is invalid
[RDD 58620: MediaPDecoder #1]: D/FFmpegVideo FFVPX:   Couldn't open avcodec for vp8
[Child 58627: Main Thread]: E/WebCodecs DecoderAgent #4 (104a87920) failed to initialize the decoder
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) state change: Configuring -> Error
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500, DecoderAgent #4 configure #4 (vp8) has been rejected. now unblocks message-queue-processing
[Child 58627: Main Thread]: E/WebCodecs VideoDecoder 10caae500, DecoderAgent #4 failed to configure: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005) - MediaResult mozilla::FFmpegDataDecoder<46465650>::InitSWDecoder(AVDictionary **): Couldn't open avcodec
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 destroys DecoderAgent #4 @ 104a87920
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920), shutdown in Error state
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) state change: Error -> Unconfigured
[Child 58627: Main Thread]: D/WebCodecs DecoderAgent #4 (104a87920) dtor
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 now has its message queue unblocked
[Child 58627: Main Thread]: E/WebCodecs VideoDecoder 10caae500 Close on error: NS_ERROR_DOM_ENCODING_NOT_SUPPORTED_ERR
[Child 58627: RemVidChild]: D/PlatformDecoderModule RemoteMediaDataDecoder[10c5e2460] ::~RemoteMediaDataDecoder: 10c5e2460 is released
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500, DecoderAgent #4's shutdown has been resolved. Drop its shutdown-blocker now
[Child 58627: Main Thread]: D/WebCodecs VideoDecoder 10caae500 removes shutdown-blocker #4 before getting any notification. DecoderAgent #4 should have been dropped

Comment 2

[C.M.Chang[:chunmin]]

Attachment: Bug 1928120 - Add a testcase

Comment 3

[C.M.Chang[:chunmin]]

Attachment: Bug 1928120 - Defer VP8 decoder configuration when image size is unspecified

This patch modifies the VPXChangeMonitor::CanBeInstantiated() to
postpone creating and initializing the VP8 decoder until the decode()
step if the image size is unknown during the configure step.

Currently, if FFmpegVideoDecoder for VP8 (via libvpx) does not receive
an image size beforehand, it will fail in Init, causing VideoDecoder
to return an error.

To prevent this situation, the patch updates
VPXChangeMonitor::CanBeInstantiated to check whether the image size is
known in advance. If it is not, decoder creation and initialization is
deferred until Decode, where the image size can be retrieved from the
input data.

This change ensures that the VP8 decoder can be successfully
configured even when the image size is initially omitted.

Depends on D232914

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5f9a1eab5a8
Add a testcase r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/759adf3f99df
Defer VP8 decoder configuration when image size is unspecified r=media-playback-reviewers,alwu

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/49862 for changes under testing/web-platform/tests

Comment 6

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/a5f9a1eab5a8
https://hg.mozilla.org/mozilla-central/rev/759adf3f99df

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot