Open Bug 1422891 (matroska) Opened 8 years ago Updated 9 days ago

[meta] Support mkv|matroska|video/x-matroska in Firefox

Categories

(Core :: Audio/Video: Playback, task, P3)

task

Tracking

()

Webcompat Priority P2

People

(Reporter: tim.langhorst, Assigned: alwu)

References

(Depends on 11 open bugs, )

Details

(4 keywords, Whiteboard: [webcompat])

User Story

Currentlt Firefox supports Opus, Vorbis, VP8, VP9, AV1 and HEVC in Matroska. The feature is controlled by the pref `media.mkv.enabled`, which is enabled by default since Firefox 145.

Attachments

(1 file)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0 Build ID: 20171122140526 Steps to reproduce: Try to embed a mkv file: <video controls> <source src="video_file.mkv" type="video/x-matroska"> </video> Actual results: Firefox says it's not supported. Expected results: Firefox should support it, because there are many codecs today and if i use for example h264 and opus together the only container supporting it is matroska converting opus is no option, because is the de facto best audio codec and converting the video to vp9 would be nice, but I neither have the time to do that on my cpu (would take more than 2200 hours) nor a hardware encoder. This is specific to me, but I think I'm not the only one, who encounters such problems.
Component: Untriaged → Audio/Video
Product: Firefox → Core
See Also: → 476727
(In reply to Tim Langhorst from comment #0) > Firefox should support it, because there are many codecs today and if i use > for example h264 and opus together the only container supporting it is > matroska Both H.264 and Opus are supported in MP4.
No mp4 does not support opus,only vp9 inofficially. Tested with ffmpeg.
Component: Audio/Video → Audio/Video: Playback
Matroska file are actually really nice to use, because: 1. There are very good tools (mktoolnix(-gui), mkvmerge, Mkclean) 2. Support for most features a container need and almost any codec (even yuv4) 3. Streaming support 4. Open source tool and standard
Implementation shouldn't be hard because WebM essentially is matroska: "The WebM container is based on a profile of Matroska." - Wikipedia, WebM (Also you can for example use matroska tools for webm because of this fact.) And sorry for the Typos.
Hey, I'm a webmaster trying to share content, I chose matroska and I just discovered that firefox couldn't embed it, while it works for chromium, even outdated versions (from debian jessie). Please consider supporting mkv's containers ? Please ? :( At least I would like an explanation on the technical reason here, must be one, because I'm really surprised.
Alfredo, Is it possible that we can add some telemetry to know how many attempts to play mkv format?
Flags: needinfo?(ayang)
caniuse has no statistics about this so I think that's very difficult. The only thing I can do is to underline it's features again. And of course the fact that with WebM already some kind of Matroska support is included.
The only thing I can help with is to link the library: https://github.com/Matroska-Org/libmatroska
See Also: → 1429986
Flags: needinfo?(ayang)
(In reply to Alfredo Yang (:alfredo) from comment #9) > Bug 1429986 Sweet!
(In reply to j.rios from comment #5) > Hey, > I'm a webmaster trying to share content, I chose matroska and I just > discovered that firefox couldn't embed it, while it works for chromium, even > outdated versions (from debian jessie). > > Please consider supporting mkv's containers ? Please ? :( > At least I would like an explanation on the technical reason here, must be > one, because I'm really surprised. There are no technical reasons here. We just hope the internet can be as simple as possible and hesitate to support more formats. But if it is really popular, we will consider supporting it. Bug 1429986 is to check how many attempts to play mkv on Firefox and we will check if we should support it or not.
This is fair. Thanks. I too, crave for a simpler web. I'm basically a JS hater. My surprise originally comes from the fact that I always saw heavy-browsers as "universal fileviewers", or at least, for videos. I really doubt telemetry will reveal any "heavy need", in fact I only stumbled on the issue recently, which means that I never knew. I guess my best shot is to make a turn for "webm" the "Google/Chrome format" ; guess I shouldn't feel wrong about it, it's CC-BY after all.
(In reply to Blake Wu [:bwu][:blakewu] from comment #11) > (In reply to j.rios from comment #5) > > Hey, > > I'm a webmaster trying to share content, I chose matroska and I just > > discovered that firefox couldn't embed it, while it works for chromium, even > > outdated versions (from debian jessie). > > > > Please consider supporting mkv's containers ? Please ? :( > > At least I would like an explanation on the technical reason here, must be > > one, because I'm really surprised. > There are no technical reasons here. We just hope the internet can be as > simple as possible and hesitate to support more formats. But if it is really > popular, we will consider supporting it. Bug 1429986 is to check how many > attempts to play mkv on Firefox and we will check if we should support it or > not. Exactly, that, why I proposed matroska because mp4 supports that, WebM that and ogg that, but if you have matroska you'll never need anything different, because matroska supports it all (that I think why ripped movies are mostly in mkv). The only choice you have to use JavaScript players, that support dash, have bad quality or have hundreds of different formats.
I want to add this table which shows who's the best: https://en.wikipedia.org/wiki/Comparison_of_video_container_formats
Flags: webcompat?
Whiteboard: [webcompat]

It seems that when google chrome records H.264 videos (MediaRecorder API) it only supports it inside matroska container and firefox can't play those files. I think it is pretty serious reason to consider matroska support besides other reasons.
Here I found the chromium issue about their implementation: https://bugs.chromium.org/p/chromium/issues/detail?id=601636

(In reply to Ignl from comment #16)

It seems that when google chrome records H.264 videos (MediaRecorder API) it only supports it inside matroska container and firefox can't play those files. I think it is pretty serious reason to consider matroska support besides other reasons.
Here I found the chromium issue about their implementation: https://bugs.chromium.org/p/chromium/issues/detail?id=601636

Well I tried to play a mkv file that is on my cloud (I use Nextcloud) and the file launch on chrome but without sound. But in the other hand, on Firefox, the player says that he can't handle this format.

So it would be cool, if Firefox supports mkv videos because as Tim said, it's usefull to have it when you are in a streaming situation like me.
Also, it would be even more cool if it supports subtitles.

PS: Here is a illustration https://imgur.com/a/mzjmcCv

(In reply to Mathieu Heurtevin from comment #17)

Well I tried to play a mkv file that is on my cloud (I use Nextcloud) and the file launch on chrome but without sound. But in the other hand, on Firefox, the player says that he can't handle this format.

same use case here. MKV support would be very useful.

Migrating Webcompat whiteboard priorities to project flags. See bug 1547409.

Webcompat Priority: --- → ?

See bug 1547409. Migrating whiteboard priority tags to program flags.

(In reply to Blake Wu [:bwu][:blakewu] from comment #11)

(In reply to j.rios from comment #5)

Hey,
I'm a webmaster trying to share content, I chose matroska and I just
discovered that firefox couldn't embed it, while it works for chromium, even
outdated versions (from debian jessie).

Please consider supporting mkv's containers ? Please ? :(
At least I would like an explanation on the technical reason here, must be
one, because I'm really surprised.
There are no technical reasons here. We just hope the internet can be as
simple as possible and hesitate to support more formats. But if it is really
popular, we will consider supporting it. Bug 1429986 is to check how many
attempts to play mkv on Firefox and we will check if we should support it or
not.

MediaRecorder API for Safari is coming (h264 of course). For cross browser interoperability h264 will be used but because Chrome puts it in Matroska container Firefox won't play it. Because of that basically a choice for a webmaster comes down to Chrome+Safari or Chrome+Firefox support. That's really not good for Firefox. Once some novel websites that leverages MediaRecorder API doesn't work on Firefox users will be forced to switch browsers. I think it would be very smart to preemptively add Matroska support ASAP.

The webm files output by MediaRecorder at Firefox and Chromium are dissimilar in more measurable ways than one. Firefox contains cues and duration and has a size greater than Chromium https://stackoverflow.com/questions/54651869/why-does-firefox-produce-larger-webm-video-files-compared-with-chrome/54652768. mkvmerge is not able to merge a webm file output by Firefox with a webm file output by Chromium https://gist.github.com/guest271314/f942fb9febd9c197bd3824973794ba3e , https://gist.github.com/guest271314/17d62bf74a97d3f111aa25605a9cd1ca. Not sure if it is proper to compare the output of MediaRecorder between Firefox and Chrome. There are several differences in the implementations.

https://wiki.mozilla.org/Media/openh264 states

OpenH264 is sandboxed for security reasons

though it is not clear at http://www.openh264.org/faq.html or https://github.com/cisco/openh264 which specific security issues are being referenced.

Is the reason the h264 codec is not available for for MediaRecorder at Firefox a licensing issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1143631; https://pagure.io/fesco/issue/1359; https://bugzilla.redhat.com/show_bug.cgi?id=1155499; https://www.mpegla.com/wp-content/uploads/n-10-08-26.pdf.)?

(In reply to Blake Wu [:bwu][:blakewu] from comment #11)

There are no technical reasons here. We just hope the internet can be as
simple as possible and hesitate to support more formats.

That is a valid point https://superuser.com/questions/1281836/what-does-matroska-have-which-webm-doesnt-that-made-the-differentiation-necess.

Chromium currently supports video/x-matroska. The WebM version of Matroska is very limited; it can move no faster nor output a file having any less size outside the scope of VP8 or VP9, while the container supports multiple codecs. Different combinations of containers and codecs provide solutions for different uses cases. Limiting options does not necessarily result in simplicity. The opposite can occur; where users attempt to find ways around the deficiencies of the default and exclusive containers and codecs available.

One practical reason to at least support x-matroska;codecs=avc1 and/or video/x-matroska;codecs=h264 is that the file size of the resulting webm file at Chromium using codecs=h264 or codecs=avc1 (openh264) is less than either codecs=vp8 or codecs=vp9. The quality (pixelation; aspect ration) of the images within video where h264 is used greater than vp8 or vp9. Is there a reason that openh264 is not supported for MediaRecorder mimeType?

Hi,

Just found this bug after filing my own (I searched for 'webm', not for 'matroska').
But it might be worth referring to it here: https://bugzilla.mozilla.org/show_bug.cgi?id=1562862

Basicly I made a small patch to allow 'webm' files (or matroska if you wish) to contain the h.264 video codec.
It is a small 30 line patch

(In reply to Jeroen Vreeken from comment #23)

Hi,

Just found this bug after filing my own (I searched for 'webm', not for 'matroska').
But it might be worth referring to it here: https://bugzilla.mozilla.org/show_bug.cgi?id=1562862

Basicly I made a small patch to allow 'webm' files (or matroska if you wish) to contain the h.264 video codec.
It is a small 30 line patch

Hi. There is evidently confusion involved in this matter. If *oogle Chrome and Chromium have abandoned the premise of supporting only VP8 and Vorbis, specifically implemented at MediaRecorder API, Mozilla should be able to follow suit and in fact go further than any presumed limitations which WebM purportedly specifies.

Basicly I made a small patch to allow 'webm' files (or matroska if you wish) to contain the h.264 video codec.
It is a small 30 line patch

Instead of patching unofficial codecs into webm, it would make far more sense to just make mkv happen, it's basically the same as webm but with all the codecs officially supported (so no 100 patches to add support for unofficial codecs but instead just one bigger to support them all (only at container level of course)). I don't know any codec that doesn't fit into mkv (and all of the ones Firefox supports work). With webm this goes on and on with for example subtitle and audio codecs.

Why recreate something that already exists?

I already changed the patch to allow the video/x-matroska mimetype.
This way you can already use matroska with the currently supported codecs (including h.264).
Additional codecs not currently supported would need some more work per codec anyway as decocing is done seperate from container/demuxer support

Re https://bugzilla.mozilla.org/show_bug.cgi?id=1562862#c11

One practical reason to at least support x-matroska;codecs=avc1 and/or video/x-matroska;codecs=h264 is that the file size of the resulting webm file at Chromium using codecs=h264 or codecs=avc1 (openh264) is less than either codecs=vp8 or codecs=vp9

https://plnkr.co/edit/MDFCT5?p=preview

See Also: → 1606822

With communities like Plex or Jellyfin, MKV is the most widely used container format. And since most people use a web browser to stream their content, Firefox not supporting MKV is a real problem. One the one hand, you really want to use Firefox for all of its benefits. On the other hand, having the server direct-play the files instead of having to transcode them would greatly improve playback and server performance.
There are thousands of users out there that would benefit from native MKV support on a daily basis.

I'm actually really surprised that firefox doesn't support this, and is even hesitant to. Like the post above me points out plex and jellyfin are very large and growing every day and the overwhelming majority of media that these serve are mkv files. I don't understand why the file container that has basically been the standard for home media for well over 10 years isn't supported. Now with the case of plex and jellyfin it isn't /that/ big of a deal because you can just remux it into another container, but I've actually come across a usecase where I actually need mkv support in my browser. It's really sad for me, but I may need to rebase my workflow around a chromium based solution if this isn't supported which I would really love to avoid.

It should be possible to reverse the offer and answer at https://gist.github.com/guest271314/04a539c00926e15905b86d05138c113c to stream H264 and Opus from Chrome to Firefox using RTCPeerConnection.

I am honestly disappointed that Firefox isn't supporting Matroska, altough it is the standard for plex or jellyfin and supported by Chrome or Android. My server wouldn't need to transcode all my videos everytime I want to watch trough Firefox.

Note, technically Firefox does support playback of Matroska files with VP8, VP9, Opus codecs at HTMLMediaElement, e.g.,

     input.onchange = async ({target:{files:[file]}}) => {
       console.log(file);
       video.src = URL.createObjectURL(file);
     }

and

    video.src = '/path/to/media.mkv';

just not when navigated to at address bar and loaded as a media document or set directly; Chrome and Chromium also exhibit that behaviour; Chrome supports playback of video/x-matroska;codecs=h264 recording with MediaRecorder and playback at HTMLVideoElement, "tip-of-tree" Chromium does not, unless built by a distribution that affirmatively enables OpenH264 support.

Is the ultimate goal to playback H264/AVC1 in a Matroska container?

See for interoperability see also

Note: If this is ever implemented for MediaRecorder, e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=601636, CodecPrivate is evidently a required element for H264 in Matroska container, see https://bugs.chromium.org/p/chromium/issues/detail?id=1161944.

(In reply to Jeroen Vreeken from comment #42)

'if ever implemented' is not the problem here. I posted patches 2 years ago to make it work. Never got a single technical reply.
This is a purely political thing.
So I recommend everybody to just leave mozilla to rot in peace and use Palemoon instead as they were actually interested in a solution.

Developers in the field can build this from scratch, write the specification, if necessary, and implement this - without an official patch applied to Mozilla source code.

For example, Web Speech API does not specify a) capture of speech synthesis audio output https://lists.w3.org/Archives/Public/public-speech-api/2017Jun/0000.html; or b) SSML input parsing https://github.com/guest271314/SSMLParser; thus browsers by default stable release means do not implement either. It might take some time to arrange all of the pieces to achieve the requirement, achievable nonetheless with or without waiting on any individual or institution to do anything.

It took several years for Chrome to implenent variable pixel dimension encoding and decoding, consistently, without crashing

Here we have several options. I will share a few that I have considered and which are certainly possible using Native Messaging https://github.com/guest271314/native-messaging-mkvmerge at any browser that supports that API (Chrome, Chromium, Firefox, Nightly, I have not tried at Brave yet), or WebTransport https://github.com/guest271314/webtransport/blob/main/webTransportAudioWorkletWebAssemblyMemoryGrow.js (not implemented at Nightly that I am aware of).

  • Use mpv controlled from JavaScript at the browser, precedence is mpv.js
  • Use native WebRTC or just enough SDP to connect and stream from the machine controlled by JavaScript at the browser, several applications support SDP and stream creation, e.g., aoirtc, gstreamer, pulse audio webrtc plugin
  • Use ogv.js
  • Write everything from scratch from and in the field (deliberately avoiding creation and usage of terms of art that might confuse some users https://github.com/guest271314/SpeechSynthesisRecorder/issues/18)

What do you want to do?

All of the above can be explored concretely while maintaining this issue and others and using the browsers to find workarounds that might be capable of achieving the requirement within a given API's capabilities, yet possibly outside of the scope of any written specification that implementers decide to adopt https://github.com/guest271314/SpeechSynthesisRecorder/issues/17, https://github.com/guest271314/captureSystemAudio even if the specification is already composed by an individual outside of the consortium of organizations that are not necessarily aligned or consistent: Competing interests, some disclosed. I am well-suited to the navigate in the domain of politics. I deal with facts, not fiction or folklore. Which could very well get an individual banned for thousands of years from contributing to organizations that cannot answer questions which would expose their ignorance of the subject-matter https://github.com/guest271314/banned/issues/2 which their censorship of the term "Negro" does anyway that is, just because an individual has a title, letters, or represents some would-be lauded institution is attempting to dictate what words are "not allowed" (effectively trying to control language, while carefully omiting words they use, typical special-interest proaganda) does not mean they are the smartest people in the room, in fact in this case I am the expert on the subject matter of the fraud of "race theory", they should known better, that is why no individual or institution in the known universe has been able to repudiate the conclusions where I settled the matter using the scientific method, nor will anyone ever be able to Race theory is a fraud https://guest271314.medium.com/race-theory-is-a-fraud-45802992f5bb : End of their story.

"Choose the lesser of the evil people, and the devil still gon' win
It could all be over tomorrow, kill our masters and start again"

  • Run the Jewels, A Report to the Shareholders/Kill Your Masters

is there a particular reason you are introducing irrelevant political nonsense into this discussion?

(In reply to xxrishoxx from comment #44)

is there a particular reason you are introducing irrelevant political nonsense into this discussion?

I do not post "nonsense" of any kind, so I have no idea what you are referring to.

All humans activities are political in nature.

I am pointing out that just because someone has a shiny little badge, title, or extra letters they print behind their name does not mean they are the smartest people in the room, or will make the correct decision, even if there are no sound alternatives, given the continuance of their self-interest might lay in the balance. Therefore, I reject individuals' and institutions' frauds, fairy tales, fiction, and folklore at the outset an proceed with achieving the requirement by any means. That is a sound counter to organizations' omissions, non-disclosure, failure to implement or fix an API for years.

I do not beg. I ask. In the meantime I roll my own. That is what I am saying. That requires the temerity to challenge whatever system is in place directly, yet without rancor, rather with facts, which settle all disputes. Truth does not need explanation, untruth does.

Re-read both comments. Intellectual property is a political animal: I own X idea. If I had not written my own patent claims I would have been paying at least $475 per hour for writing the archaic language of claiming ownership of thoughts. I roll my own. I have have done so up to SCOTUS, twice, by myself, where I would have been paying at least that much per hour over the course of years had I not done my own reasearch and written my own briefs, motions, answers, etc.

There is no technical reason that Matroska container including the various codecs supported by the standard/specification (EBML, etc.) cannot be implemented immediately at Firefox https://github.com/ietf-wg-cellar/matroska-specification/blob/master/codec_specs.md.

The decision not to is clearly purely political, or if you prefer, based on the risk of action by the intellectual property "owners", for example, should something go wrong, that is, the insurance game. And depending on which issue or bug you read the political matter was settled re H264 and AAC, or the matter is still ripe.

https://bugs.chromium.org/p/chromium/issues/detail?id=601636#c29

H.264 is not supported in the default Chromium build.

Anyone who turns on H.264 when building Chromium takes on the responsibility of complying with all relevant licenses; the default Chromium builders don't do this.

https://bugs.chromium.org/p/chromium/issues/detail?id=601636#c32

Copyright is not the problem. Patents are.

For some indication of what the problem might be, check out https://www.mpegla.com/programs/avc-h-264/

compare

https://bugs.chromium.org/p/chromium/issues/detail?id=1072056#c8

Yup, using a JS / wasm MP4 muxer should be possible, either from that API or by transcoding a MediaRecorder-produced webm container with H.264 video.

For audio, AAC-LC is actually now patent-unencumbered (the spec dates from 1997) and is actually the profile used for the vast majority of aac encoded content. RedHat have been shipping a modified fdk-aac for a few years now with the more recent features stripped out, which could be built for web assembly: https://src.fedoraproject.org/rpms/fdk-aac-free

So it's technically possible to get client-side mp4/h264/aac recording implemented on Chrome now without requiring patent licenses, but it still quite a bit of work to get all the parts together. This bug relates to adding support for "video/mp4" directly in MediaRecorder which would make life so much easier and avoid the need for all the support wasm / js that would be required.

see also

https://bugs.chromium.org/p/chromium/issues/detail?id=980822

A webm file should never be created with a h264 track. Only containers such as mp4, matroska, mpeg (ts or ps) can include a h264 track.

and

https://bugs.chromium.org/p/chromium/issues/detail?id=980822#c12

Notwithstanding its absence from the standard, there's a good use case for the MIME type video/webm; codecs="avc1.42E01E" : generating webcam-sourced video for decoding with Broadway (https://github.com/mbebenita/Broadway).

There's also a valid argument that boxing formats (webm / mp4 / mov) should be implemented orthogonally to the codec bitstreams they contain.

Please don't remove this MIME type from Chromium's MediaRecorder capabilities.

The matter is still being sorted out re patent claims. Wait, or proceed, in the field as developers in spite of Chrome or Firefox? To proceed requires a decision to confront the conflicts above, and perhaps unintended consequences. I am prepared for that, politcally, physically, and if necessary supernaturally. We can build this whole thing in a GitHub repository and implement it ourselves, without asking absentee technocrats or patent attorney for permission. If you do not get what I am saying here, I don't know what to told you.

My use case is to record (server-side) WebRTC-compatible audio and video. MKV is an obvious choice that supports all of Opus, VP8, VP9 and H264 in Matroska container here and today, while MP4 compatibility with those codecs is not great.

For instance this is what happens when I try to mux VP8 into MP4 (which is technically supported) using ffmpeg that is available in Ubuntu 20.10 repos:

[mp4 @ 0x5621a55b3d00] Could not find tag for codec vp8 in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument

It would be be much easier if I didn't have to implement support for multiple containers just because Firefox can't include +1 library on top of already supported and related WebM. For many people this is just one more reason to not use Firefox, which makes me, Firefox user, very sad.

This bug is more relevant now with AV1 support in Firefox. The only two container formats for AV1 are Matroska and the ISO Base Media File Format (video/mp4). Since the latter doesn't support modern audio codecs such as opus, Matroska is the only other option for combining audio and video in a single file.

Why is .mkv not yet supported. It seems to become the standard in video content encoding and delivery due to the vastly extended features compared to .webm.
Hence, I think adding support should be far up the list.

Even YouTube and other platforms seem to target .mkv for HDR content delivery.

If this is something that the team isn't opposed to, I'm happy to lend my development time towards this.

(In reply to Blake Wu [:bwu][:blakewu] from comment #6)

Alfredo,
Is it possible that we can add some telemetry to know how many attempts to
play mkv format?

I will comment on this briefly. I think you may find that the data you get may be skewed due to selection bias here, as both the Chromium (link to comment here) and Firefox do not fully support the container. As a result, people who may desire to use the format may be forced to remux or otherwise repackage their content.

Flags: needinfo?(bvandyk)

Even YouTube and other platforms seem to target .mkv for HDR content delivery.

Is this the case? YT appear to allow Matroska as an input format, but I have not seen them serve media in it. I see them use WebMs.

(In reply to Chris Tam from comment #50)

If this is something that the team isn't opposed to, I'm happy to lend my development time towards this.

Redirecting to :jimm.

Flags: needinfo?(bvandyk) → needinfo?(jmathies)

(In reply to guest271314 from comment #38)

Note, technically Firefox does support playback of Matroska files with VP8, VP9, Opus codecs at HTMLMediaElement, e.g.,
just not when navigated to at address bar and loaded as a media document or set directly; Chrome and Chromium also exhibit that behaviour; Chrome supports playback of video/x-matroska;codecs=h264 recording with MediaRecorder and playback at HTMLVideoElement, "tip-of-tree" Chromium does not, unless built by a distribution that affirmatively enables OpenH264 support.

Is the ultimate goal to playback H264/AVC1 in a Matroska container?

I couldn't make it work.

But yes, MKV support would be really nice. At least partial support with VP8, VP9 and h264 codecs.

Attached video test_matroska_audio.mkv

(In reply to nicomu.net from comment #52)

(In reply to guest271314 from comment #38)

Note, technically Firefox does support playback of Matroska files with VP8, VP9, Opus codecs at HTMLMediaElement, e.g.,
just not when navigated to at address bar and loaded as a media document or set directly; Chrome and Chromium also exhibit that behaviour; Chrome supports playback of video/x-matroska;codecs=h264 recording with MediaRecorder and playback at HTMLVideoElement, "tip-of-tree" Chromium does not, unless built by a distribution that affirmatively enables OpenH264 support.

Is the ultimate goal to playback H264/AVC1 in a Matroska container?

I couldn't make it work.

But yes, MKV support would be really nice. At least partial support with VP8, VP9 and h264 codecs.

The last time I tested it was possible to set src of HTMLMediaElement to .mkv in HTML and the media would play.

On Firefox 90 this message is logged to console

HTTP “Content-Type” of “video/x-matroska” is not supported. Load of media resource blob:https://fiddle.jshell.net/0e434e1b-b3b4-41cd-9c8c-8a4f4a2890dc failed. _display

Cannot play media. No decoders for requested formats: video/x-matroska

We can still play the media in spite of the warning by setting Content-Type of response or type of new File or Blob to 'audio/webm;codecs=opus'.

<!DOCTYPE html>

<html>
  <head> 
    <title>Test Matroska with Opus audio track playback on Firefox 90 and Chromium 95</title>
  </head>

  <body>
    <input type="file" accept=".mkv" />
<!--
Firefox 90 warning for .mkv file with one Opus audio track
HTTP “Content-Type” of “video/x-matroska” is not supported. Load of media resource blob:https://fiddle.jshell.net/0e434e1b-b3b4-41cd-9c8c-8a4f4a2890dc failed. _display
Cannot play media. No decoders for requested formats: video/x-matroska
-->
    <audio controls src="test_matroska_audio.mkv"></audio>
    <script>
      const input = document.querySelector('input[type=file]');
      const audio = document.querySelector('audio');
      audio.onloadedmetadata = (e) => console.log(audio.duration);
      input.onchange = async ({
        target: {
          files: [file],
        },
      }) => {
        let blob;
        if ('mozCaptureStream' in audio && 'CanvasCaptureMediaStream' in globalThis) {
          blob = new Blob([file], {type:'audio/webm;codecs=opus'});
        }
        console.log(blob || file);
        audio.src = URL.createObjectURL(blob || file);
      };
      fetch('test_matroska_audio.mkv')
      .then((response) => response.arrayBuffer())
      .then((ab) => audio.src = URL.createObjectURL(new Blob([ab], {type:'audio/webm;codecs=opus'})))
    </script>
  </body>
</html>

(In reply to nicomu.net from comment #52)

(In reply to guest271314 from comment #38)

Note, technically Firefox does support playback of Matroska files with VP8, VP9, Opus codecs at HTMLMediaElement, e.g.,
just not when navigated to at address bar and loaded as a media document or set directly; Chrome and Chromium also exhibit that behaviour; Chrome supports playback of video/x-matroska;codecs=h264 recording with MediaRecorder and playback at HTMLVideoElement, "tip-of-tree" Chromium does not, unless built by a distribution that affirmatively enables OpenH264 support.

Is the ultimate goal to playback H264/AVC1 in a Matroska container?

I couldn't make it work.

But yes, MKV support would be really nice. At least partial support with VP8, VP9 and h264 codecs.

Alternatively, for Opus, VP8, VP9. Theora, Vorbis; AVC/H.264/MPEG-4p10, MP3 in Matroska container does not decode.

  fetch('./video.mkv')
  .then((response) => {
    console.log(response.headers.get('Content-Type'));
    if (response.headers.get('Content-Type') === 'video/x-matroska') {
      return new Response(response.body, {headers:{'Content-Type':'video/webm;codecs=vp9'}}).blob()
    }
    return response.blob();
  })
  .then((blob) => {
     console.log(blob.type);
     video.src = URL.createObjectURL(blob)
  });
Webcompat Priority: ? → P2
See Also: 1606822

I was surprised today when Firefox 100.0.2 direct-played AV1 in MKV today when testing the AV1-transcoding-capabilities of my 10600K for Jellyfin. It appears there is some level of support for MKV now.

(In reply to Roov4.email from comment #59)

I was surprised today when Firefox 100.0.2 direct-played AV1 in MKV today when testing the AV1-transcoding-capabilities of my 10600K for Jellyfin. It appears there is some level of support for MKV now.

This is just because Firefox supports AV1. Remember that the WEBM file format spec is just a pure subset of MKV, so the parser is likely just parsing the AV1 video (+ compatible audio codec) in the MKV format as valid WEBM (which it is) and playing it.

Flags: needinfo?(jmathies)
Depends on: 1790164
Severity: normal → S3

The severity field for this bug is relatively low, S3. However, the bug has 4 duplicates, 25 votes, 62 CCs and 16 See Also bugs.
:jimm, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Flags: needinfo?(jmathies)

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.

Flags: needinfo?(jmathies)

Unfortunately, they took off https://connect.mozilla.org/t5/ideas/add-support-for-matroska-media-container-mkv-files/idi-p/15935 because they do not plan to add it in the future. I have been building my smart home systems to use HTML webpages to play all sorts of videos on every TV in my house. I am now switching from Firefox to Chrome because many videos cannot be played in Firefox (HEVC/H265, MKV, etc.), but all videos can be played in Chrome. For Firefox, it used to be able to play MKV, but not in recent versions.

In my opinion, there seems to be some problem with the company: it is extremely unwise to remove Matroska container support because a large proportion of online videos are still in this container including webm. Moreover, they are not buying the patent license for HEVC/H265 (like Google/Microsoft), given a significant proportion of online videos are in this codec format nowadays. That is why Chrome can play HEVC/H265 videos, they bought the license.

That is why Chrome can play HEVC/H265 videos, they bought the license.

No that is exactly not the reason. Chrome and Microsoft only play heavy when hardware acceleration is present, because then the hardware manufacturer payed for a license. If they would licence it themselves, the browsers would cost them billions literally, pricing is that bad.

Also no webm is still supported and that's what you'll often see get used for inline video, because it uses vp9, the usage of hevc for online videos is still pretty rare and given the license fees and limited client support it'll stay that way. The big reason MKV is so nice isn't hevc, but literally anything else it can support (that won't immediately bankrupt anyone who's trying to use it).

Maybe this should be closed as WontFix so developers in the field can rationally abandon all hope of this getting done. Then those same developers in the field can create user-defined solutions for this without any idea that browser maintainers are going to be involved with the process. For starters we can launch mpv from the browser using Native Messaging.

An alternative to using an external media player controlled from the browser or embedded in the browser is to use and extension to convert all URL's navigated to that end with .mkv to WebM containing VP9 and Opus on the fly.

Matroska is just a open source video container format and the superset of WebM. It should not require any video codec patents to be implemented.
Why will Firefox not support this format or even treat its MIME as an alias of WebM?

(In reply to guest271314 from comment #75)

An alternative to using an external media player controlled from the browser or embedded in the browser is to use and extension to convert all URL's navigated to that end with .mkv to WebM containing VP9 and Opus on the fly.

Alternative where the functionality is missing from the browser, yes. I wouldn't say it's an alternative solution from the browser development perspective. Is there an extension that does this?

Is there an extension that does this?

We can do this using Native Messaging and local ffmpeg without any issues. Here's how we can do this using a Deno script, where URL can be a file: URL or HTTP URL.

const url = new URL(Deno.args[0], import.meta.url).href;

const command = new Deno.Command("ffmpeg", {
  args: [
    "-i",
    "pipe:",
    "-nostats",
    "-hide_banner",
    "-loglevel",
    "error",
    "-c:v",
    "libvpx-vp9",
    "-c:a",
    "libopus",
    "-f",
    "webm",
    "pipe:1",
  ],
  stdout: "piped",
  stdin: "piped",
});

try {
  const process = command.spawn();
  console.log(process);
  const readable =  (await fetch(url)).body;
  console.log(readable);
  await readable.pipeTo(process.stdin);
  const blob = await new Response(process.stdout, {
    headers: {"Content-Type": 'video/webm;codecs="vp9,opus"'}
  }).blob();
  console.log(blob);
  console.log(await process.status);
} catch (e) {
  console.log(e);
}

I began work on using ffmpeg.wasm in the browser alone and ran into a consistency issue using the playground https://github.com/ffmpegwasm/ffmpeg.wasm/issues/722.

(In reply to guest271314 from comment #79)

Is there an extension that does this?

We can do this using Native Messaging and local ffmpeg without any issues. Here's how we can do this using a Deno script, where URL can be a file: URL or HTTP URL.

const url = new URL(Deno.args[0], import.meta.url).href;

const command = new Deno.Command("ffmpeg", {
  args: [
    "-i",
    "pipe:",
    "-nostats",
    "-hide_banner",
    "-loglevel",
    "error",
    "-c:v",
    "libvpx-vp9",
    "-c:a",
    "libopus",
    "-f",
    "webm",
    "pipe:1",
  ],
  stdout: "piped",
  stdin: "piped",
});

try {
  const process = command.spawn();
  console.log(process);
  const readable =  (await fetch(url)).body;
  console.log(readable);
  await readable.pipeTo(process.stdin);
  const blob = await new Response(process.stdout, {
    headers: {"Content-Type": 'video/webm;codecs="vp9,opus"'}
  }).blob();
  console.log(blob);
  console.log(await process.status);
} catch (e) {
  console.log(e);
}

I began work on using ffmpeg.wasm in the browser alone and ran into a consistency issue using the playground https://github.com/ffmpegwasm/ffmpeg.wasm/issues/722.

In fact, I wonder why not use ffmpeg library instead? AFAIK Edge uses ffmpeg.dll.

Duplicate of this bug: 1913681

The severity field for this bug is set to S3. However, the following bug duplicate has higher severity:

:jimm, could you consider increasing the severity of this bug to S2?

For more information, please visit BugBot documentation.

Flags: needinfo?(jmathies)
Flags: needinfo?(jmathies)
No longer duplicate of this bug: 1913681

We have for a long time chosen not to support matroska due to breadth of formats the container can support. Some background discussion can be found here -
https://docs.google.com/document/d/1sWKlq0BqRAbJtzHnvNU_ZG50JvyFEDFrTVHhMydbFMI/edit?tab=t.0#heading=h.hyfeh8f9vggd

Needs to be revisited and discussed after some additional research into the size of the project. We would also need to allocate the resources to this, which could impact other projects.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: [Rethink] Support mkv|matroska|video/x-matroska in Firefox → Support mkv|matroska|video/x-matroska in Firefox

We have for a long time chosen not to support matroska due to breadth of formats the container can support

Please note that what people like me most desire is support for the codecs which Firefox already supports in other container formats.

I'm pretty sure you can stuff pretty much any codec in existence into Matroska but what we actually need is the popular formats such as h.264, VP9, AV1, AAC, Opus, Vorbis etc. Firefox already supports these when they're in MP4 or WebM containers but not when delivered in a Matroska container. That's the core of this issue I think.

We do not need more codecs to be supported here and probably shouldn't given the increase in attack surface.

https://docs.google.com/document/d/1sWKlq0BqRAbJtzHnvNU_ZG50JvyFEDFrTVHhMydbFMI/edit?tab=t.0#heading=h.hyfeh8f9vggd

Opening that link prompts me to log in; please ensure it's publicly accessible or mirror the relevant sections please.

You don't have to try to support all of the codecs supported by MKV. Just those that have already supported and are planned to be supported by Firefox.

(In reply to Tatsunori Uchino from comment #86)

You don't have to try to support all of the codecs supported by MKV. Just those that have already supported and are planned to be supported by Firefox.

have already supported → have already been supported

(In reply to Tatsunori Uchino from comment #86)

You don't have to try to support all of the codecs supported by MKV. Just those that have already supported and are planned to be supported by Firefox.

Why do you need a particular container? What are the advantages?

Flags: needinfo?(tats.u)

(In reply to [:Atemu] from comment #85)

https://docs.google.com/document/d/1sWKlq0BqRAbJtzHnvNU_ZG50JvyFEDFrTVHhMydbFMI/edit?tab=t.0#heading=h.hyfeh8f9vggd

Opening that link prompts me to log in; please ensure it's publicly accessible or mirror the relevant sections please.

Sorry about that. I'll open that doc up once we have a chance to clean it up of internal data. It's just a developer discussion of what's needed in Firefox to support mkv and was posted here for internal reference related to webcompat discussions.

(In reply to Jim Mathies [:jimm] from comment #88)

(In reply to Tatsunori Uchino from comment #86)

You don't have to try to support all of the codecs supported by MKV. Just those that have already supported and are planned to be supported by Firefox.

Why do you need a particular container? What are the advantages?

Advantages are being able to watch a video that you encounter on the Web instead of not being able to watch it.

Why do you need a particular container? What are the advantages?

It's just a developer discussion of what's needed in Firefox to support mkv and was posted here for internal reference related to webcompat discussions.

For the case of VP8 or VP9 and Opus in Matroska container I think all that is necessary is to check the underlying codecs in the container to determine if Firefox can play the media or not.

Firefox can already play VP8 or VP9 and Opus in a Matroska container - but won't play the media unless the MIME type is changed to e.g. video/webm;codecs=vp9,opus.

Seems to me that case can be fixed, today.

(In reply to Jim Mathies [:jimm] from comment #88)

(In reply to Tatsunori Uchino from comment #86)

You don't have to try to support all of the codecs supported by MKV. Just those that have already supported and are planned to be supported by Firefox.

Why do you need a particular container? What are the advantages?

It's very easy for media server software to transcode everything into MKV, source Files are often MKV and with MKV the server can then just transcode what needs to be transcoded into MKv without checking what Container is needed. With h264 you need mp4, but wait opus isn't working, so maybe webm, but wait that doesn't support h264... Some of those things are sometimes supported, but not by everyone... With MKV it's easy: if the codec is supported, you can pack it into MKV. That simple, no vendor shenanigans between MPEG and Google, everything that's supported by Firefox is supported by MKV, no need for special logic guessing for each client what combinations might be possible in which container, it just immediately works (as long as the codecs are supported of course).

The question nowadays should probably be: Why anything else than Matroska (Aside from it's support, especially in commercial hardware and software of course, those often only do mp4)? An in theory if Matroska is supported you could in theory throw out explicit webm decoding as webm is a subset of MKV and as such an MKV decoder should be able to decode webm as well.

User Story: (updated)

Why do you need a particular container? What are the advantages?

https://tests.caniuse.com/av1

The test cases in caniuse used to be only MKV. I submitted the other ones, which can be played by Firefox.
I wouldn't have to make such effort if Frefox had already been able to play MKV...

https://github.com/Fyrd/caniuse/issues/6693

Advantages are being able to watch a video that you encounter on the Web instead of not being able to watch it.

RIGHT

Flags: needinfo?(tats.u)

We are considering supporting this for the codecs we already have support for in the browser. We would not support the full range of mkv features though. Would that be sufficient for people requesting support here in the bug?

User Story: (updated)

(In reply to Jim Mathies [:jimm] from comment #94)

We are considering supporting this for the codecs we already have support for in the browser. We would not support the full range of mkv features though. Would that be sufficient for people requesting support here in the bug?

For sure, supporting every feature of MKV is practically impossible as it simply supports basically everything, that's why it so nice as a container. But what would be great is if FF just supported everything in MKV that it supports anyway so we can just have a look at what the browser supports (using the appropriate JS APIs) and mux that into MKV, that's everything I'm asking for. MKV after all is just a container like mp4 and you can put things like h265 and even h266 in there that have no support in most browsers, but that still doesn't prevent browsers from supporting MKV.

but that still doesn't prevent browsers from supporting MKV.

Sorry, that should be mp4 there not MKV.

Absolutely. I'd wager the vast majority of MKVs that are out there are of the type that contains only codecs that Firefox already supports.

As mentioned previously, I find it preferable to not support all the niche codecs that fit into Matroska due to an immense increase in attack surface and dubious benefit.

(In reply to [:Atemu] from comment #97)

As mentioned previously, I find it preferable to not support all the niche codecs that fit into Matroska due to an immense increase in attack surface and dubious benefit.

Can't firefox using RLBox? AFAIK firefox has been use it to isolate some third-party libraries.

User Story: (updated)

(In reply to Jim Mathies [:jimm] from comment #88)

Why do you need a particular container? What are the advantages?

For me, the main advantage is for home clouds/NAS systems, where you would want to play a video that is stored on the NAS. mkv is an extremely common video format, so it's very helpful to not have to re-pack the video just to be able to play it in the browser.

Some personal streaming tools like https://emby.media/do not play h265 media on Firefox because of this. Most new videos are encapsulated inside an .mkv. Using it with NAS is one type of usage, but not the only one.
Cheers!

(In reply to Marcin Gil from comment #100)

Some personal streaming tools like https://emby.media/ do not play h265 media on Firefox because of this. Most new videos are encapsulated inside an .mkv. Using it with NAS is one type of usage, but not the only one.
Cheers!

Here is the corrected link as I could not edit the message.
https://emby.media/

Please be aware that this is about MKV support, not HEVC. Only because MKV is is supported, don't expect all your files to magically play as some codecs like HEVC or subtitles the are not webvtt won't play. It just makes it a lot easier to craft files with codecs that already work in the browser.

HEVC itself works as MP4, but not as MKV.

Duplicate of this bug: 1957262

Note that MKV is now a standard: https://www.ietf.org/rfc/rfc9559.html

So, this isn't some "unofficial" container format anymore.

Flags: needinfo?(08xjcec48)
Blocks: 1981131
Duplicate of this bug: 1835393

I’ll be working on implementing this. Once I’ve sorted out the implementation details, I’ll share my plan.

Alias: matroska
Assignee: nobody → alwu
Keywords: meta
Summary: Support mkv|matroska|video/x-matroska in Firefox → [meta] Support mkv|matroska|video/x-matroska in Firefox
Depends on: 1985336
Depends on: 1985384
Type: defect → task
Depends on: 1985419
User Story: (updated)
Depends on: 1986058
No longer blocks: 1835393
User Story: (updated)
Blocks: 1938052

Once this is behind a pref in Nightly, it should probably be documented at https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Experimental_features.

Sebastian

Keywords: dev-doc-needed
Depends on: 1429986
See Also: 1429986
Duplicate of this bug: 476727
See Also: 476727
Depends on: 1827913
Blocks: 1986414
Depends on: 1987783

A quick update: you can now watch AVC (H.264) and AAC in MKV on the latest Nightly. The preference is media.mkv.enabled (currently enabled by default only on Nightly). Feel free to try it out and report any issues if you encounter AVC or AAC in MKV failing to play.

Support for other codecs will be added gradually.

(In reply to Alastor Wu [:alwu] from comment #115)

A quick update: you can now watch AVC (H.264) and AAC in MKV on the latest Nightly. The preference is media.mkv.enabled (currently enabled by default only on Nightly). Feel free to try it out and report any issues if you encounter AVC or AAC in MKV failing to play.

Support for other codecs will be added gradually.

Thank you very much, very happy to see that firefox can now play MKV.
I confirm that I can play the "dom_media_test_output_avc.mkv" file on latest nightly.

Looking forward for more codecs support

Depends on: 1718468
See Also: 1718468

Thanks for supporting it, but I still have problem with the nightly.

Just playing a matroska video using a video element works great.
However MediaSource does not support it. E.g. MediaSource.isTypeSupported() returns false for any matroska mimetype I throw at it. Even those that work with the video tag.

Re: #117 #118
Please file a new bug and block this bug about you problem instead of commenting here. This is a meta bug and does not treat individual problems.

Depends on: 1988447
Depends on: 1988450

This should be re-opened https://bugzilla.mozilla.org/show_bug.cgi?id=1563140 as "Depends on".

Depends on: 1988676
Depends on: 1563140
Depends on: 1988848
Depends on: 1989491

A quick update: now Opus, Vorbis, VP8, VP9, AV1 and HEVC support have been added after bug 1988848. FLAC and MP3 are not implemented yet, as they require additional plumbing and have very limited usage.

Hello Alastor Wu [:alwu]
I have some files with :

Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
Stream #0:1(fre): Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s (default)

Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv), 1920x1080, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn (default)
Stream #0:1(eng): Audio: aac (LC), 48000 Hz, 5.1, fltp (default)

Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv, progressive), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 1k tbn
Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s (default)

I got the "No video supported" message.
Aren't they supposed to be played in latest nightly ?

Thanks!

Flags: needinfo?(alwu)

@GreyXor Please read comment #119.

Summary: [meta] Support mkv|matroska|video/x-matroska in Firefox → [meta] (please file a new bug about you problem instead of commenting here) Support mkv|matroska|video/x-matroska in Firefox
Summary: [meta] (please file a new bug about you problem instead of commenting here) Support mkv|matroska|video/x-matroska in Firefox → [meta] (please file a new bug about your problem instead of commenting here) Support mkv|matroska|video/x-matroska in Firefox
Depends on: 1989759
No longer depends on: 1718468
Depends on: 1718468
User Story: (updated)
User Story: (updated)
Flags: needinfo?(alwu)
Depends on: 1990076
User Story: (updated)
Restrict Comments: true
No longer depends on: 1988450, 1988676
Depends on: 1991746
Depends on: matroska-mp3
Depends on: 1991748
Depends on: 1991752
No longer duplicate of this bug: 1606822
Depends on: 2000420
Depends on: 2006226
See Also: → 2025302
Depends on: 2025302
See Also: 2025302
Depends on: 2027522
Depends on: 2041969
User Story: (updated)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: