1757788 - MediaCapabilities API does not have consistent results when passing a contentType without the codec's configuration parameters.

Status: UNCONFIRMED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Zaggy1024]

Attachment: Video support test.html

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:99.0) Gecko/20100101 Firefox/99.0

Steps to reproduce:

Create a MediaCapabilities.decodingInfo query with a video configuration with the codec specified in contentType, but with no codec parameters. For example:

await navigator.mediaCapabilities.decodingInfo({type: 'file', video: {contentType: 'video/mp4;codecs=avc1', width: 1920, height: 1080, bitrate: 1000000, framerate: 30}});

or

await navigator.mediaCapabilities.decodingInfo({type: 'file', video: {contentType: 'video/mp4;codecs=vp9', width: 1920, height: 1080, bitrate: 1000000, framerate: 30}});

Attached is a test file with decodingInfo from many different codec strings that helps show the issue clearly.

Actual results:

Codec string 'avc1' returns unsupported (MediaCapabilitiesInfo { supported: false, smooth: false, powerEfficient: false }), whereas 'vp9' returns supported (MediaCapabilitiesInfo { supported: true, smooth: true, powerEfficient: false }).

Expected results:

Both should either return supported or unsupported, to remain consistent.

From my reading of the spec, even that is unclear on what should happen in these cases. I believe it only refers to valid mimetypes, so presumably unspecified parameters should be considered unsupported.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[John Lin [:jhlin][:jolin]]

The inconsistency is caused by MP4Decoder::GetTracksInfo() rejecting av1[1](ExtractH264CodecDetails() always checks additional parameters[2]). However, it doesn't do so for vp9[3].

[1] https://searchfox.org/mozilla-central/source/dom/media/mp4/MP4Decoder.cpp#117-135
[2] https://searchfox.org/mozilla-central/source/dom/media/VideoUtils.cpp#454
[3] https://searchfox.org/mozilla-central/source/dom/media/mp4/MP4Decoder.cpp#96-106

Comment 3

[Zaggy1024]

It should be noted that this also affects HTMLVideoElement.canPlayType, such that canPlayType("video/mp4;codecs=avc1") will not return "maybe", but instead empty string "". This will indicate to sites that H264 is unsupported, if they don't specify codec parameters.

Patching VP9 to be consistent with this will also cause VP9 to read as unsupported by canPlayType, so as part of a patch to fix this, it would be good to rework canPlayType to return "maybe" in cases where a codec that requires parameters to instantiate a decoder is provided with no parameters.