Add MediaSource.isTypeSupported() support for video width, height, and bitrate parameters

NEW
Unassigned

Status

()

Core
Audio/Video: Playback
P3
normal
2 years ago
a year ago

People

(Reporter: jya, Unassigned)

Tracking

(Depends on: 1 bug, Blocks: 1 bug, {dev-doc-needed, feature})

Trunk
dev-doc-needed, feature
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox42 wontfix, firefox43 affected, firefox44 affected, firefox45 affected)

Details

(Reporter)

Description

2 years ago
This would be particularly useful with youtube which uses MediaSource.IsTypeSupported to query if a particular codec is supported.

Example:
IsTypeSupported(aType=video/webm; codecs="vp9")OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=640)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=99999)[not supported] 
IsTypeSupported(aType=video/webm; codecs="vp9")OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; height=360)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; height=99999)[not supported] 
IsTypeSupported(aType=video/webm; codecs="vp9")OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; framerate=30)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; framerate=9999)OK 
IsTypeSupported(aType=video/webm; codecs="vp9")OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; bitrate=300000)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; bitrate=2000000000)OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.4d4015"; width=426; height=240)OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.4d401e"; width=640; height=360)OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.4d401e"; width=854; height=480)OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.4d401f"; width=1280; height=720)OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.640028"; width=1920; height=1080)OK 
IsTypeSupported(aType=audio/mp4; codecs="mp4a.40.2")OK 
IsTypeSupported(aType=video/mp4; codecs="avc1.4d400c"; width=256; height=144)OK 
IsTypeSupported(aType=audio/webm; codecs="vorbis")OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=426; height=240)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=640; height=360)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=854; height=480)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=1280; height=720)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=1920; height=1080)OK 
IsTypeSupported(aType=video/webm; codecs="vp9"; width=256; height=144)OK 

An extra parameter is provided: width or height . If we know a particular resolution can't be decoded properly or fast enough we should report "NO" there.

YouTube appears to use the decoder that supports the highest resolution supported for a particular video. E.g. if the highest resolution provided by a video is 1080p and webm reports as only supporting 720p while h264 reports supporing 1080p: h264 will be used.
Priority: -- → P2
The MSE spec just says isTypeSupported() should take a valid MIME type string, which may include some combination of media type, media subtype, and codecs. It doesn't describe the width, height, and bitrate parameters that YouTube is using, but these parameters would be very useful because the VPx codec strings don't provide profile information. We currently have no way to tell the website that a user's laptop can play 1080p H.264 (in hardware) and 720p VP9 (in software), but can't keep up with 1080p VP9.

https://w3c.github.io/media-source/#widl-MediaSource-isTypeSupported-boolean-DOMString-type
status-firefox42: --- → wontfix
status-firefox43: --- → affected
status-firefox44: --- → affected
Keywords: feature
Summary: Reports when a decoder can't support a particular resolution → Add MediaSource.isTypeSupported() support for video width, height, and bitrate parameters
(Reporter)

Comment 2

2 years ago
The MSE spec clearly states on what conditions we should return "NO"

"5. If the MediaSource does not support the specified combination of media type, media subtype, and codecs then return false."

it doesn't state that we can return "NO" based on any mimetype parameters.

The MSE spec must be amended if we want to support this (which would be very useful)
YouTube says these codec parameters were an extension for smart TVs and are not implemented in desktop Chrome. They are interested in standardizing something in this area to improve content support checks.
Blocks: 1230265
Depends on: 1259212
If we want to roll VP9 out to more users we could return 'no' based the following limits:

>=75 FPS => 720p60
>=40 FPS => 720p30
Keywords: dev-doc-needed
need-infoing Eric for updating the MDN doc pages which are currently mentioning the old canRecord... method
Flags: needinfo?(eshepherd)
(In reply to Soledad Penades [:sole] [:spenades] from comment #5)
> need-infoing Eric for updating the MDN doc pages which are currently
> mentioning the old canRecord... method

So... MediaSource.isTypeSupported() is already documented.

However, I do see MediaRecord.canRecordMimeType() instead, which does need to be fixed. I will slot that in as soon as I can.
Flags: needinfo?(eshepherd)
Note that we should likely wait until this lands before documenting it.
(In reply to Jean-Yves Perrier [:teoli] from comment #7)
> Note that we should likely wait until this lands before documenting it.

Yes, documenting these parameters should wait. However, :sole is correct that the canRecordMimeType method no longer exists on MediaRecord; that's already implemented, so this should be updated on MDN, here: 

https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/canRecordMimeType

and

https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder should be updated to link to a new page https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/isTypeSupported which documents that method.
Mass change P2 -> P3
Priority: P2 → P3
Bug 1176218 implements the support for YouTube parameters (and uses them to disallow resolutions that we know are too high for different versions of Windows.)

I think we can keep this bug open to work on documentation, and standardization (or not) of these parameters.
Depends on: 1176218
You need to log in before you can comment on or make changes to this bug.