1988848 - Support more codecs in Matroska (Opus, Vorbis, VP8, VP9, AV1, and HEVC)

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, task, P1)

Description

[Alastor Wu [:alwu]]

Currently we already have AVC and AAC support in Matroska, I'm going to add more codecs in this bug.

---

In this bug, we will add support for Opus, Vorbis, VP8, VP9, AV1, and HEVC in Matroska. FLAC and MP3 are not included yet, as they require additional plumbing and have very limited usage.

Comment 1

[Tim Langhorst]

This seems especially important since existing code seems to expect supported codecs to work across all (supported) containers: https://github.com/jellyfin/jellyfin-web/blob/6c3a3a7205e543bca2eece03984a8fc93d9afd89/src/scripts/browserDeviceProfile.js#L614
In this case Jellyfin specifically would assume every codec that works in mp4 also works in matroska. Missing support could therefore break playback in this case at least (not sure about other apps, this is the only peoductive mime checking code I know, but it might also affect other apps). So it might be good to have it done until it ships in non-nightly.

Comment 2

[Alastor Wu [:alwu]]

Attachment: mkv_codec_usage.png

Per our initial Glean probe, we will start adding HEVC, VPX, Opus, AV1 and Vorbis support in this bug.

Comment 3

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part1 : add opus to Matroska supported codecs.

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part2 : add hevc to Matroska supported codecs.

Also replace a new test file output_hevc.mkv because the old
one is gbrp which we don't support. The new one is yuv420.

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part3 : add vp8 and vp9 to Matroska supported codecs.

Also change the vp9 test file from grba to yuv420.

Comment 6

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part4 : add vorbis to Matroska supported codecs.

Comment 7

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part5 : add av1 to Matroska supported codecs.

Also changed the test file from gbrp to yuv420.

Comment 8

[Alastor Wu [:alwu]]

Attachment: Bug 1988848 - part6 : ensure both current and legacy Matroska mime type work

Per [1], x-matroska is a legacy MIME type, and the updated MIME type removes
the x prefix. We should support both.

We’ve also added new test cases to verify that using these MIME types with
HTMLMediaElement.canPlayType() returns the correct result.

[1] https://www.rfc-editor.org/rfc/rfc9559#name-media-types

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/aaeea8540d0a
https://hg.mozilla.org/integration/autoland/rev/b25e77301dfc
part1 : add opus to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/968994015f08
https://hg.mozilla.org/integration/autoland/rev/842bfea9b76e
part2 : add hevc to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/e6a54251e126
https://hg.mozilla.org/integration/autoland/rev/fc1286db447f
part3 : add vp8 and vp9 to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/c9dfdedd2212
https://hg.mozilla.org/integration/autoland/rev/ab76a4cb594e
part4 : add vorbis to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/c671a8443e9a
https://hg.mozilla.org/integration/autoland/rev/a3067fb26c56
part5 : add av1 to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/1ccca6b0ac75
https://hg.mozilla.org/integration/autoland/rev/9f90fc06a565
part6 : ensure both current and legacy Matroska mime type work r=media-playback-reviewers,padenot

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/7c174f2a0cdc
https://hg.mozilla.org/integration/autoland/rev/592f88b586e4
Revert "Bug 1988848 - part6 : ensure both current and legacy Matroska mime type work r=media-playback-reviewers,padenot" for causing xpcshell failures on test_mediasniffer_ext.js

Comment 11

[Cristina Horotan [:chorotan]]

Backed out for causing xpcshell failures

Backout link

Push with failures

Failure log

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/f5d7f93eeccd
https://hg.mozilla.org/integration/autoland/rev/b166d3bd8d53
part1 : add opus to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/313ac2e27f5a
https://hg.mozilla.org/integration/autoland/rev/e65f4b93eeab
part2 : add hevc to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/d34529210eb4
https://hg.mozilla.org/integration/autoland/rev/fa804997096d
part3 : add vp8 and vp9 to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/a00f53beaa46
https://hg.mozilla.org/integration/autoland/rev/a524c8ade3d6
part4 : add vorbis to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/03ecf5752ef1
https://hg.mozilla.org/integration/autoland/rev/b9cc4b3c7471
part5 : add av1 to Matroska supported codecs. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/2c971d7076cb
https://hg.mozilla.org/integration/autoland/rev/cd184ca12a25
part6 : ensure both current and legacy Matroska mime type work r=media-playback-reviewers,padenot

Comment 13

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/b166d3bd8d53
https://hg.mozilla.org/mozilla-central/rev/e65f4b93eeab
https://hg.mozilla.org/mozilla-central/rev/fa804997096d
https://hg.mozilla.org/mozilla-central/rev/a524c8ade3d6
https://hg.mozilla.org/mozilla-central/rev/b9cc4b3c7471
https://hg.mozilla.org/mozilla-central/rev/cd184ca12a25