Last Comment Bug 435339 - DirectShow backend for HTML5 video element
: DirectShow backend for HTML5 video element
Status: VERIFIED WONTFIX
[wontfix? - see comment 53]
:
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: Trunk
: x86 Windows Vista
: -- enhancement with 21 votes (vote)
: ---
Assigned To: Chris Pearce (:cpearce)
:
: Maire Reavy [:mreavy]
Mentors:
Depends on: video
Blocks:
  Show dependency treegraph
 
Reported: 2008-05-22 17:38 PDT by Matthew Gregan [:kinetik]
Modified: 2016-05-13 17:22 PDT (History)
56 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Patch v1 (338.39 KB, patch)
2008-10-23 17:09 PDT, Chris Pearce (:cpearce)
no flags Details | Diff | Splinter Review

Description Matthew Gregan [:kinetik] 2008-05-22 17:38:39 PDT
Bug 382267 covers the main work on the video element.  This bug is for the
DirectShow backend implementation.
Comment 1 Damon Sicore (:damons) 2008-06-04 16:46:26 PDT
marking wanted1.9.1? to get this in the triage queue.  If this needs to be blocking1.9.1?, please mark it as so.
Comment 2 Jonas Sicking (:sicking) No longer reading bugmail consistently 2008-06-10 17:26:31 PDT
Chris: Need your input here, should this be marked wanted, and at what priority?
Comment 3 Chris Pearce (:cpearce) 2008-06-10 18:25:16 PDT
I have made good progress on this. I've got the DirectShow backend playing videos in <video> in the browser, but there's still a lot to do. Provided all the <video> element stuff gets through ok, this should be ready by 1.9.1. P1?
Comment 4 cajbir (:cajbir) 2008-06-10 20:06:52 PDT
Yes, the plan is for 1.9.1. Wanted and P1 please, as it's needed for Windows implementation of <video>
Comment 5 Jonas Sicking (:sicking) No longer reading bugmail consistently 2008-06-11 00:48:48 PDT
So for what it's worth we don't set priority based on how much work is remaining to implement, but rather on how badly we want it.

But we badly want <video> on windows, so P1 sounds right :)
Comment 6 Chris Pearce (:cpearce) 2008-10-23 17:09:20 PDT
Created attachment 344562 [details] [diff] [review]
Patch v1

DirectShow decoder backend patch v1. W00t.

Build requirements:
* Windows Media SDK
* DirectX SDK
* Appropriate include/lib path changes to MozillaBuild.

I'll post a build and build instructions shortly.
Comment 7 Chris Pearce (:cpearce) 2008-10-23 20:05:25 PDT
I have a build for people to play with here: 

http://pearce.org.nz/firefox-directshow-video-2008_10_24.win32.zip

You can test it here:

http://pearce.org.nz/player/
Comment 8 Robert O'Callahan (:roc) (email my personal email if necessary) 2008-10-23 20:34:46 PDT
A few quick comments:

+NS_IMPL_THREADSAFE_ISUPPORTS2(nsVariant, nsIVariant, nsIWritableVariant)

This seems bad. Why do we have to do this?

+    case NS_DIRECTSHOW_MEDIA_EVENT:

Aren't user messages supposed to have IDs above 0xC000 or something? Or isn't there an API for registering them?

All your compiler directives like #ifdef should not be indented at all.

Does it make sense for Chris Double to review this? I'd still review it too.
Comment 9 Chris Pearce (:cpearce) 2008-10-23 21:53:00 PDT
(In reply to comment #8)
> +NS_IMPL_THREADSAFE_ISUPPORTS2(nsVariant, nsIVariant, nsIWritableVariant)
> 
> This seems bad. Why do we have to do this?

IIRC this fixed a bunch of assertion failures saying unthreadsafe release, and a few crashes... I'll get back to you with more details on this.

> 
> +    case NS_DIRECTSHOW_MEDIA_EVENT:
> 
> Aren't user messages supposed to have IDs above 0xC000 or something? Or isn't
> there an API for registering them?

They can be between 0x0400 and 0x7fff.
See: http://msdn.microsoft.com/en-us/library/ms644931(VS.85).aspx


> All your compiler directives like #ifdef should not be indented at all.

I have a lot of them. It's very jarring when reading code if they're not indented. Most of them are guarding dbglog() calls, I think I can see how to do some #define magic so that we don't need all those guards.
 
> Does it make sense for Chris Double to review this? I'd still review it too.

Yeah, poor Chris D. ;)
Comment 10 Chris Pearce (:cpearce) 2008-10-23 21:58:50 PDT
To build you need to install the DirectX SDK, you don't actually need the Windows Media SDK it turns out, it's included in the Windows SDK now. You can download the DirectX SDK here: (458MB) http://www.microsoft.com/downloads/details.aspx?FamilyId=519AAE99-B701-4CA1-8495-39DDDE9D7030&displaylang=en

You must set the MOZCONFIG option --enable-directshow and you must change your MozillaBuild start-msvcX.bat to add the DirectX lib and include paths to %LIB% and %INCLUDE%, i.e. to start-msvc8.bat, before the call to "%MOZILLABUILD%\msys\bin\bash --login -i" add:

set LIB=%LIB%;%DXSDK_DIR%\Lib\x86
set INCLUDE=%INCLUDE%;%DXSDK_DIR%\Include
Comment 11 pd 2008-10-31 22:56:42 PDT
I've given the test build a run on your test page and generally everything seemed to work well.

The test page contains a lot of controls and options compared to the OGG test pages I've used. However there do not appear to be any particular narrowly defined tasks I could run to test finitely what is behaving and what is not behaving.

Am I missing something? If not, how can I give more specific feedback if indeed you want more specific feedback?
Comment 12 Chris Pearce (:cpearce) 2008-11-02 01:01:40 PDT
Thanks, at the moment I don't have any specific test cases. I was generally planning to just port the Ogg test cases and ensure the behaviour matches.
Comment 13 Anand Kumria 2008-11-08 15:28:59 PST
Perhaps this has been discussed before but is this using the GStreamer wrapper around DirectShow (like in Songbird) http://blog.songbirdnest.com/2008/01/30/gstreamer-for-all-all-for-gstreamer/ ?
Comment 14 Chris Pearce (:cpearce) 2008-11-08 22:47:26 PST
No, we don't use the Songbird DirectShow source/renderer filters, we do all the decoding and rendering with DirectShow.
Comment 15 Robert Millan 2008-12-07 12:42:36 PST
This looks really promising.  If FLV video support were easy to install, this could become a viable alternative for big sites like youtube.  Though, since FLV is patent encumbered I understand you're probably reluctant to distribute it as a backend.

So, how does this DirectShow backend react when file types aren't supported by the system?  I'm thinking that a dialog prompting the user to download missing codecs (e.g. http://www.playflv.com/ for FLV) would solve that problem, similar to the one displayed for Flash or Java plugins.

If this happened, I don't see why wouldn't youtube et al want to use <video> tags to play their FLVs with fallback to the Adobe Flash baggage.
Comment 16 Chris Pearce (:cpearce) 2008-12-07 13:42:37 PST
(In reply to comment #15)
> So, how does this DirectShow backend react when file types aren't supported by
> the system?

Currently if a required codec isn't installed, or the codec isn't registered to accept a byte-stream input, DirectShow won't be able to construct filter graph which it uses to decode the video, and the video won't play.

> If this happened, I don't see why wouldn't youtube et al want to use <video>
> tags to play their FLVs with fallback to the Adobe Flash baggage.

It would be simpler to just use Ogg/theora/vorbis, as support/behaviour would be the same on all platforms...
Comment 17 WL 2009-04-22 20:02:31 PDT
Any result for the reviewing?
Is it possible to build it into Firefox 3.6 nightly?
Comment 18 Chris Pearce (:cpearce) 2009-04-26 18:35:03 PDT
Comment on attachment 344562 [details] [diff] [review]
Patch v1

(In reply to comment #17)
> Any result for the reviewing?

We've decided to focus on the Ogg backend, as it provides a cross platform solution. Thus I'm clearing review requests.

> Is it possible to build it into Firefox 3.6 nightly?

No. The video backend has changed considerably since the latest patch was posted. It would need some work to make it compile and run.
Comment 19 pd 2009-04-27 05:57:21 PDT
This is an enormous disappointment for video on the web. Expect to see no change from content providers at all due not just to the lack of support for ogg in IE but this poor decision. Cross platform is not everything, market share pressuring Microsloth into more standards support is. Browser features that have limited chance of being supported in the real world are just a waste of precious developer resources.
Comment 20 Ben Bucksch (:BenB) 2009-04-27 07:21:49 PDT
Yes, please try to get this in. At least MPEG2 and MPEG4 support are very important, as they are standards and a tons of video are in that format, and they are cross-platform (if you ignore software patents): There's probably hardly any system used for video which cannot play MPEG2.
I need this MPEG2+4 support for a xulrunner app I'm writing right now (think Joost or MythTV).
Comment 21 pd 2009-05-05 00:26:38 PDT
FFMPEG is cross platform, why not use that as a core library? It's good enough for Xbox Media Centre (XBMC) why can't it work for Firefox?

http://xbmc.org/about/thankyou/

Then there is GStreamer which is not just Linux-based, it's cross platform:

http://en.wikipedia.org/wiki/GStreamer

Does someone really have to hand out a truckload of venture capital funding to build a Songbird for video out of the mozilla trunk?

Why does Mozilla insist on re-inventing the wheel by coding a cross-platform application to use platform-specific subsystems? Is it performance? Is the difference really that dramatic? If not, why? Chris Double could you shed some light on this situation?

Re Chris Pearce's statement about the video backend having changed considerably, does this mean that Firefox does not currently compile with the DirectShow code enabled?
Comment 22 Mike Kristoffersen (:MikeK) 2009-05-05 01:21:08 PDT
(In reply to comment #21)
> Then there is GStreamer which is not just Linux-based, it's cross platform:

FYI: We are/I am working on integrating and using GStreamer for Fennec on the Linux based devices, it might end up on more platforms...
Comment 23 cajbir (:cajbir) 2009-05-05 01:39:05 PDT
(In reply to comment #21)
> FFMPEG is cross platform, why not use that as a core library? It's good enough
> for Xbox Media Centre (XBMC) why can't it work for Firefox?

The problem is not the availability of source code, the problem is the licensing of patents to use that code. If we use ffmpeg, or any other implementation of video codec code, we'd need to pay the license fees to distribute Firefox with that code.

We could do this of course since the cost of an h.264 license is in the vicinity of a few million dollars a year. But anyone that builds or distributes modified versions of Firefox would not be covered by this. They would have to remove that code before building or pay for a license themselves. This goes against one of the good things about the Mozilla project. Being able to take our code, change it to have the features they want, and redistribute it with the same features as the binary they can download.

I do not know how the xbmc works around this issue.

> Then there is GStreamer which is not just Linux-based, it's cross platform:

See bug 422540 for a GStreamer backend. It won't be in Firefox 3.5. It is likely to be in Fennec, the mobile build. The latter is because the mobile vendors have paid for the required codec licenses for their device. My understanding is that the only codecs that will work, even though GStreamer is included, is those the mobile vendors has licenses for and chooses to distribute.

> Why does Mozilla insist on re-inventing the wheel by coding a cross-platform
> application to use platform-specific subsystems? Is it performance? Is the
> difference really that dramatic? If not, why? Chris Double could you shed some
> light on this situation?

It is the licensing requirements for the codecs, it's that simple. Not only do you have to pay to encode, decode and distribute a product with some codecs but you even have to pay to stream the video from your website. It's a legal minefield and we're treading carefully. Firefox 3.5 is the first release with built in video and we'll see what happens from there.

> Re Chris Pearce's statement about the video backend having changed
> considerably, does this mean that Firefox does not currently compile with the
> DirectShow code enabled?

Correct.
Comment 24 pd 2009-05-05 02:48:05 PDT
Thanks for the background everyone. I do wonder how several projects seem to overcome the licensing issue WRT libraries like FFMPEG. Perhaps it's the patent troll scenario? No point getting the law involved if there's nobody big enough to sue? I guess Mozilla is relatively big so you need to be somewhat defensive legally. Still I wonder how the FFMPEG kids do it. Surely they do not pay license fees and then hand over the code for free? I'd like to know how that business model works!

Good news that GStreamer might be progressing though sounds like only if Nokia and the like are willing to pay the licensing fees, which they would have no reason to do for the desktop Firefox. 

So is DirectShow support in Firefox just going to become bit rot? I think if the open web has much of a chance, it clearly needs a more flexible a/v backend than Flash and Silverlight offer.
Comment 25 Mike Kristoffersen (:MikeK) 2009-05-05 03:34:23 PDT
(In reply to comment #24)
> Good news that GStreamer might be progressing though sounds like only if Nokia
> and the like are willing to pay the licensing fees, which they would have no
> reason to do for the desktop Firefox. 
I'm not into the legal stuff, but there are different parts of GStreamer, there is the generic framework, and then the codecs packed in a number of packages depending on the quality and legal aspects of each individual codec.  

As long as we don't ship our code with the GStreamer codecs, I can't see how we can violate any patents... (but remember I don't know the law in detail for any country) 

The Nokia device has GStreamer and some codecs installed from the factory, it might be a little tricky for some users to fetch the remaining codecs manually, but it could be done, or it might be possible to imagine a 3-rd party plugin that did this automatically (for the free codecs or the after-the-patents-expire-they-are-free codecs if people handle the legal aspects on an individual basis)
Comment 26 Chris Pearce (:cpearce) 2009-05-05 03:49:22 PDT
(In reply to comment #24)
> So is DirectShow support in Firefox just going to become bit rot?

Yes.

> I think if the open web has much of a chance, it clearly needs a more 
> flexible a/v backend than Flash and Silverlight offer.

DirectShow won't really help the open web... Suppose you want to put a movie on your website. Unless the movie is encoded in WMV or MPEG1, you can't guarantee that the DirectShow backend will be able to decode it on any given computer. You can't guarantee that a web user has any particular codec that doesn't ship with Windows installed (like divx, xvid, etc...), so you have to use either WMV or MPEG1. But if your video is encoded in WMV, it will play on Windows machines, but Mac users, which make up approximately 10% of all internet users, can't view it. Linux users also won't be able to view it. The QuickTime backend has the same problem - videos encoded in MOV can't be viewed by the DirectShow backend. Using gstreamer would be nice, but as Chris Double pointed out, the codec licensing is still a problem.

(In reply to comment #25)
> As long as we don't ship our code with the GStreamer codecs, I can't see how we
> can violate any patents... (but remember I don't know the law in detail for any
> country) 

Right, but again then web developers can't guarantee what codecs are on a users system right?

The best solution for the open web is to encode your video in Ogg Theora/Vorbis. You can then be sure it will play in Firefox on all platforms. You can also add fallback to the Cortado Java applet to play Ogg videos in browsers that don't support <video> or Ogg. http://www.scvi.net/cortado.htm

See http://tinyvid.tv for an example of this fallback working.
Comment 27 Ben Bucksch (:BenB) 2009-05-05 04:20:49 PDT
> As long as we don't ship our code with the GStreamer codecs

Exactly.

It's the same situation as on Windows: Any application can use the generic DirectShow APIs, without violating patents. The user may get the codec from anybody legally, and any DirectShow application can legally play the videos without being in violation of the patents. There are tons of DirectShow applications which intentionally don't ship with MPEG codec, but will pay MPEG2/4 once you install the right codecs. We'd be in exactly the same situation.

We're not implementing MPEG2, so we're not in violation of the MPEG patents. We're merely using generic OS or library functions.

----

As for the open web: I agree with Chris. Allowing arbitrary codecs shouldn't be the goal, because it doesn't help the open web. However, I think that a <video> tag that plays *no* usable codec, or only a codec that nobody has encoded video in, does not help the open web either.
I really like the idea of Theora, but unfortunately, it is neither efficient (IIRC not even in comparison to MPEG2, much less MPEG4) nor an official standard. MPEG2/4 are unfortunately patent encumbered, but are a standard and very widely used, in DVDs, DVB (almost all TV in Europe) etc..
I have a few TB of MPEG videos myself. I have 1 or 2 Theora videos.
Not many Windows users have DirectShow codecs for it, which means they can't save and play the video.
Hardly any website wanting to show video to their users will consider Theora to be a tempting alternative. It's more likely to be the object of ridicule of those people who don't like open source very much, which doesn't make it easier for our open web friends to push for <video> support on their website. The provider would have to keep all videos in another format.
If <video> would allow MPEG2 and MPEG4, they could stream their existing video files, which would make support for <video> a *lot* easier to implement, just some HTML+JS changes versus changing their content management, encoders and adding tons of storage.
I could create a XULrunner application which can play video as background, preview or a video player. I am writing just such an app right now and need MPEG2/4 support. Reencoding 1 TB is out of question.

So:
- Please enable MPEG2 and 4, and not the other proprietary codecs. This will not make the video situation worse: an open system needs to deal with MPEG anyways due to DVB and DVD.
- It will help <video> tremendously, because it reduces the effort to use it from very high (change the whole video library) to small (change one HTML page).
- I don't think patents are an issue, if you don't ship MPEG codes, but merely use generic APIs.
Comment 28 Ben Bucksch (:BenB) 2009-05-05 04:28:07 PDT
> I have a few TB of MPEG videos myself

FYI, this is mostly TV recordings. Not that anybody gets a wrong impression :).
The XULrunner app I mentioned would be a personal TV app similar to Joost.
Comment 29 Mike Kristoffersen (:MikeK) 2009-05-05 04:31:34 PDT
(In reply to comment #26)
> (In reply to comment #25)
> > As long as we don't ship our code with the GStreamer codecs, I can't see how we
> > can violate any patents... (but remember I don't know the law in detail for any
> > country) 
> 
> Right, but again then web developers can't guarantee what codecs are on a users
> system right?
> 
Right :) but we have two aspects to take care for here - one is the
promotion/support of an open web and the free codec (Ogg) and the other is to
give the users of Firefox the best access to the content on the web.

So yes, I agree that we should support Ogg and recommend it to content
providers, but at the same time I believe that we should also do our best to
support as many other formats as possible as it is highly likely that there are
content providers out there, who can't/won't use Ogg for one reason or the
other - and why shouldn't we allow our users to easily download and use any
codecs they might want to?
Comment 30 cajbir (:cajbir) 2009-05-05 04:51:08 PDT
From comment 27:
> Please enable MPEG2 and 4, and not the other proprietary codecs

What audio format would you use? And in what container?

From comment 29:
> So yes, I agree that we should support Ogg and recommend it to content
> providers, but at the same time I believe that we should also do our best to
> support as many other formats as possible as it is highly likely that there are
> content providers out there, who can't/won't use Ogg for one reason or the
> other - and why shouldn't we allow our users to easily download and use any
> codecs they might want to?

The problem with this approach is content providers won't know what formats users can play. There are no common codecs across all operating systems if we rely on OS installed codecs. Users are back to the situation of having to download a codec to play content for a particular site. DirectShow codecs are often a cause of malware and spyware - installed when people are told to download it so they can play videos they've obtained. One advantage of <video> was that it avoids this situation.
Comment 31 Anand Kumria 2009-05-05 05:00:16 PDT
From comment 27:

>However, I think that a <video>
>tag that plays *no* usable codec, or only a codec that nobody has encoded video
>in, does not help the open web either.

I love this old chestnut.

Presumably wikipedia's content counts for nothing, then?

Yes, it is sad that there is no censensus about the native video format - but the correct place to put pressure for minimal baseline support is Google (chrome), Microsoft and Apple.

With both Firefox and Opera supporting Ogg natively, and users of those other browsers having to utilise Cortado to see wikipedia content - I think you will find that the next releases include support.
Comment 32 Ben Bucksch (:BenB) 2009-05-05 05:13:51 PDT
> > Please enable MPEG2 and 4, and not the other proprietary codecs

> What audio format would you use? And in what container?

Audio: This is defined by MPEG2 and 4. They define the audio codec. That one needs to be in every stream, additional streams like AC3 are optional. The fact that hardware needs to play DVD and DVB limits even the additional streams.

The container is defined by MPEG2 and 4 as well. There are two types: TS (Transport Stream) and PS (Program Stream). TS is what you find in DVB / TV, PS is what you find on DVD (.VOB) and many .mpg video files. Given that this is just the container, it's easy to support both, and most applications support both. The content is the same.

> There are no common codecs across all operating systems if we
> rely on OS installed codecs.

I already mentioned that above: Given that MPEG2 is so widespread, almost any user interested in video has MPEG codecs. Esp. on Linux.

> minimal baseline support

The point is that we need more than "minimal baseline".
Comment 33 cajbir (:cajbir) 2009-05-05 05:26:36 PDT
> Please enable MPEG2 and 4, and not the other proprietary codecs

It might be worthwhile opening a new bug about MPEG2/4 support rather than asking for it in this one. This one is for the DirectShow backend which is Windows specific. I'm assuming you want MPEG2/4 across all platforms. This is the approach taken by those wanting other codec support (Speex in bug 476752 and kate in bug 481529) with <video>.
Comment 34 Radek 'sysKin' Czyz 2009-05-05 05:34:35 PDT
(In reply to comment #32)
> I already mentioned that above: Given that MPEG2 is so widespread, almost any
> user interested in video has MPEG codecs. Esp. on Linux.

I don't think mentioning mpeg-2 makes much sense here, given how inefficient it is. For some legacy systems it might make some sense, but if a website ever wastes my download quota on an mpeg-2 stream ripped off DVD or DVB, I will rage :)

There are pretty much two efficient video codecs out there: mpeg's AVC and (slightly less so) Microsoft's VC-1. Both open/free standards but both require license for implementation.

> Presumably wikipedia's content counts for nothing, then?

Hmm I'd say it counts for pretty much nothing, really. Very low quality videos at quite a high bitrate, and nowhere near as easy to access as youtube. Not a single advantage for any user.

I said it before and I'll say it again: there are two problems with Theora. One is that the standard is not as efficient as the best ones (there's only this much you can do without b-frames). Second is that there's just no useful implementation. The only implementation which exists is reference encoder - which has its own goals, by necessity much different than being "useful" for any kind of "consumer".
Comment 35 Ben Bucksch (:BenB) 2009-05-05 06:02:12 PDT
Chris, MPEG support, I presume, would be implemented by using a generic API, due to the mentioned patent problems preventing distribution of the MPEG decoders. Sure, I guess one could use gstreamer on Windows, if that can use DirectShow codecs, but you'd have to ship gstreamer. Therefore, I think the best approach is to implement the platform-specific video API and use the already-installed codecs.
If you want to prevent the use of proprietary codecs like WMV, you can prevent their use specifically and only allow MPEG2/4.
Comment 36 pd 2009-05-05 06:26:49 PDT
>> So is DirectShow support in Firefox just going to become bit rot?

>Yes.

I feel very sorry for whoever worked hours upon hours on this, let alone the many users who could finally have enjoyed the sort of video freedom they have on the desktop, inside an open web browser.

>> I think if the open web has much of a chance, it clearly needs a more 
>> flexible a/v backend than Flash and Silverlight offer.

>DirectShow won't really help the open web... Suppose you want to put a movie on
>your website. Unless the movie is encoded in WMV or MPEG1, you can't guarantee
>that the DirectShow backend will be able to decode it on any given computer.

I think that logic is really missing the point. Right now if you want to author video on the web you do it in Flash, a proprietary close-source, more impressive and reliable version of the web. Flash is so far ahead of the open web Mozilla is forced to bundle it with Firefox in order to be competitive as a usable browser! Microsoft is trying to emulate Flash with SilverLight. What is the open web's answer? OGG? This is a codec that for all it's purity in terms of patents and royalties *nobody* uses! ... </rant>

Ok, I'll try to cut myself short. Here's the future:

Open web: OGG, 35% market share, less than great quality
Closed web: H.264, VP6, DivX/Xvid, 90+% market share

Which web do you think is going to succeed?

Don't know why I put DivX/XviD in there? Windows 7 will be bundling it and anybody who has ever downloaded a TV or movie off a P2P service already has XviD installed.

Conclusion: you're banking on a solution that unless the EU comes through, is highly unlikely to be supported by a large market player (IE) and therefore will be ignored by developers who cannot be botherd double encoding content to provide a OGG version along with an H.264 or XviD version when the latter can target 90+% market share.

>You can't guarantee that a web user has any particular codec that doesn't ship
>with Windows installed (like divx, xvid, etc...)

Windows 7 is already shipping with an H.264/MPEG4-AVC codec:

http://www.digitalwerks.org/wp-content/uploads/windows7_AVCa.png

That will eventually cover up to (assuming Windows 7 achieves the sort of popularity that XP has, which most indications thus far suggest it will) over 80% of users. Given that x264 has been around for quite a while as an encoding and decoding solution and is now embedded in a plethora or front ends:

http://en.wikipedia.org/wiki/X264#x264_frontends

across at least Linux and Windows, surely DirectShow / GStreamer (Linux) and QuickTime support in Firefox for H(x).264 is the most appropriate video solution? This will allow content authors who are already encoding in this higher-definition codec for use in Flash to just keep on doing so but easily transition to HTML5 instead of a nasty and insecure Flash plugin.

>(In reply to comment #25)
>> As long as we don't ship our code with the GStreamer codecs, I can't see how we
>> can violate any patents... (but remember I don't know the law in detail for any
>> country) 

Shipping without any codec(s) IMHO is a big no-no *unless* a clear and simple download wizard - similar to the Plugin Finder - can be used to reliably download codecs at will. For reasons not completely clear, almost every video player I've tried fails to do this well except GOM Player:

http://www.gomlab.com/eng/

>Right, but again then web developers can't guarantee what codecs are on a users
>system right?
>
>The best solution for the open web is to encode your video in Ogg
>Theora/Vorbis. You can then be sure it will play in Firefox on all platforms.
>You can also add fallback to the Cortado Java applet to play Ogg videos in
>browsers that don't support <video> or Ogg. http://www.scvi.net/cortado.htm
>
>See http://tinyvid.tv for an example of this fallback working.

Installing a Java Applet? That's madness. Java is one of the most clunky nagging plugins out there. Why force users to add a runtime that slows down their system performance and presents them with a constant upgrade hassle?

Really, that is not a solution that is competitive with the current dominant solution: Flash. 

Why should this video support be any different than fonts? Give content authors and users full access to whatever is installed on the user's system and allow content authors access to specify codec download like the new @font-face ?

The best way to grow the open web is to provide a content author and user experience that is the most compelling available without sacrificing FOSS ethics. OGG has very few encoding applications for content authors and the OGG/<video> + Java + Java Applet solution is not something any reasonable author would present to their users, let alone something users should have to put up with. You might as well just say "we like OGG and we know it's only going to be supported by 35% of the market but we don't care".
Comment 37 pd 2009-05-05 06:37:19 PDT
>> As long as we don't ship our code with the GStreamer codecs
>
>Exactly.
>
>It's the same situation as on Windows: Any application can use the generic
>DirectShow APIs, without violating patents. The user may get the codec from
>anybody legally, and any DirectShow application can legally play the videos
>without being in violation of the patents. There are tons of DirectShow
>applications which intentionally don't ship with MPEG codec, but will pay
>MPEG2/4 once you install the right codecs. We'd be in exactly the same
>situation.
>
>We're not implementing MPEG2, so we're not in violation of the MPEG patents.
>We're merely using generic OS or library functions.

IF Firefox supports DirectShow yeah, agreed. It's the hand responsibility to the user approach. Same approach Microsoft has employed with Windows and they are supplementing in Windows 7 by paying for codecs to be installed. Same approach taken by Ubuntu. The different is that Firefox without native OS APIs is saying, in effect "you better hope content authors support OGG or you're stuffed" when really Firefox should be saying "hey we support OGG to the hilt but we also support any codec you've installed on your system".
Comment 38 Mike Kristoffersen (:MikeK) 2009-05-05 07:01:41 PDT
(In reply to comment #30)
> The problem with this approach is content providers won't know what formats
> users can play. There are no common codecs across all operating systems if we
> rely on OS installed codecs. Users are back to the situation of having to
> download a codec to play content for a particular site. DirectShow codecs are
> often a cause of malware and spyware - installed when people are told to
> download it so they can play videos they've obtained. One advantage of <video>
> was that it avoids this situation.

How can it be that supporting Ogg + codecs installed by user is seen as making it harder for the content provider than only supporting Ogg ? 

No, there might not be common codecs installed on all OS's but why do we have to make that a problem for the individual user? - If the users favorite web-site uses format A, what is it going to help him if we state that "no, we think format A that your computer can play back is evil because it is not Ogg so you better use another un-pure browser if you want to see that content"  

I suppose that if a website uses a non-free-font then we would not refuse to show the page?
Comment 39 pd 2009-05-05 07:08:39 PDT
>> Please enable MPEG2 and 4, and not the other proprietary codecs
>
>What audio format would you use? And in what container?

Good question. I'm not so clear on audio. Vorbis (FOSS), mp3 (actually in use) would cover both current reality (mp3) and ideal (vorbis). Container is tricky for sure.

>From comment 29:
>> So yes, I agree that we should support Ogg and recommend it to content
>> providers, but at the same time I believe that we should also do our best to
>> support as many other formats as possible as it is highly likely that there are
>> content providers out there, who can't/won't use Ogg for one reason or the
>> other - and why shouldn't we allow our users to easily download and use any
>> codecs they might want to?
>
>The problem with this approach is content providers won't know what formats
>users can play. 

This is not the case if the ideal+reality solution is implemented, along with cascading support like font-family has used for years. Ideal = vorbis, Reality = MPG2+4 (covered through native APIs - DS, GStreamer, QT - by any P2P user, all Windows 7 users, x264 codec for non MS platforms).

With the current Firefox OGG solution, content providers still cannot guarantee what formats users can play either due to IE. 

>There are no common codecs across all operating systems if we
>rely on OS installed codecs. Users are back to the situation of having to
>download a codec to play content for a particular site. 

There are no common codecs across all browsers either, except H.264 and VP6 in the Flash plugin. Really, we need to realise we are competing with Flash, nothing more, nothing less. Just ask GoogleTube and any other video web site. There is no guarantee of how to compete with Flash but there is an approach that is legal, is relatively user and content provider friendly and is already significantly coded in the trunk: OGG plus native (DirectShow, QT, GStreamer) API support.

If necessary cut down support in 3.5 from all natively supported codecs to just MPG2/4 for reliability and testing purposes. If that goes well, further support could always be adopted in the future.

>DirectShow codecs are
>often a cause of malware and spyware - installed when people are told to
>download it so they can play videos they've obtained. One advantage of <video>
>was that it avoids this situation.

That's not correct. As suggested by Chris Pearce: *IF* content authors decide to support the OGG + Java, Java applet for IE users, they are going to have to download a plugin/applet anyway. 

Also don't forget the enemy is Flash, arguably a security/malware plugin itself given how many security holes are often found in it and how the update cycle is not clear.

Furthermore the best way to avoid a executable-download scenario is to support native APIs because they are the most likely to have support already installed. Is it more likely that authors will choose OGG + Java + Java applet which will force at least 50% of their users into an executable-download scenario, or choose MPG2/4 which in a Windows 7 / P2P world (which is what we are likely to be living with given the likely long gestation of Fx 4) will be natively installed on around 70% of machines, if not more like 80% ?

I realise people in Mozilla land nearly all seem to run around with MacBooks running MacOSX and perhaps this clouds their thinking but for the foreseeable future we are living in a Windows dominated world, for better or worse. All those enterprises who gave Vista the cold shoulder appear likely at this stage to upgrade to Windows 7. XP and Vista have a limited lifespan so Windows 7, wit it's native H.264/MPEG-AVC support via DirectShow is really the most sensible target for competing with Flash. If this means enabling DirectShow then this is what should surely happen? This is not excluding anyone because H.264 is already installed with QuickTime 7:

http://www.apple.com/quicktime/technologies/h264/

and x264 is available free for Linux.

How much more evidence is needed?!!
Comment 40 pd 2009-05-05 07:11:27 PDT
>>However, I think that a <video>
>>tag that plays *no* usable codec, or only a codec that nobody has encoded video
>>in, does not help the open web either.
>
>I love this old chestnut.
>
>Presumably wikipedia's content counts for nothing, then?

No, it accounts for a lot to support my argument: there is bugger-all video on WikiPedia which is exactly the point.

Anyway this point is moot because I don't think anybody is arguing against supporting OGG, just that OGG alone is not enough.
Comment 41 Shawn Wilsher :sdwilsh 2009-05-05 07:13:45 PDT
Discussions on which codes we will be shipping surely doesn't belong in this bug.  The more appropriate place would be the mozilla.dev.apps.firefox or mozilla.dev.platform newsgroups.  See also https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
Comment 42 pd 2009-05-05 07:23:41 PDT
(In reply to comment #41)
> Discussions on which codes we will be shipping surely doesn't belong in this
> bug.  The more appropriate place would be the mozilla.dev.apps.firefox or
> mozilla.dev.platform newsgroups.  See also
> https://bugzilla.mozilla.org/page.cgi?id=etiquette.html

I disagree. This is the perfect place. Newsgroups are not as accessible as bugzilla is and nor are they are effective in archiving the views of those who wish to comment.

This is simply a very hot topic that should be heavily debated but like most such topics is buried eventually by someone who doesn't want to churn through all the mail generated by such topics.
Comment 43 Anand Kumria 2009-05-05 07:27:03 PDT
nothing new to add in this bug, as per the guidelines related to politeness and etiquette.
Comment 44 Martijn Wargers [:mwargers] (not working for Mozilla) 2009-05-05 08:04:43 PDT
(In reply to comment #42)
> (In reply to comment #41)
> > Discussions on which codes we will be shipping surely doesn't belong in this
> > bug.  The more appropriate place would be the mozilla.dev.apps.firefox or
> > mozilla.dev.platform newsgroups.  See also
> > https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
> 
> I disagree. This is the perfect place. Newsgroups are not as accessible as
> bugzilla is and nor are they are effective in archiving the views of those who
> wish to comment.

No, this is not the place. Bugzilla is meant for technical issues (what is the bug, how can it be reproduced, how can it be fixed). It is never meant to be used as a discussion system. You might be right that newsgroups/mailing lists are not as accessible, but that doesn't make it right to use bugzilla.

> This is simply a very hot topic that should be heavily debated but like most
> such topics is buried eventually by someone who doesn't want to churn through
> all the mail generated by such topics.

I agree this is a hot topic and it should be debated, but please don't use bugzilla for this.
Comment 45 Martijn Wargers [:mwargers] (not working for Mozilla) 2009-05-05 08:42:09 PDT
Pd responded to me personally, and I'm sorry that I sounded so harsh on him. It's nothing personal, bugzilla isn't meant as a discussion forum and this bug is getting out of hand.

Btw, I've just asked on IRC where this discussion should take place/continue and it's in the dev.platform mailing list/newsgroup:
https://lists.mozilla.org/listinfo/dev-platform
Comment 46 Shawn Wilsher :sdwilsh 2009-05-05 08:44:53 PDT
And it also has a web front end via google groups, so I'm really not sure how anyone could say it's less accessible than bugzilla.:
http://groups.google.com/group/mozilla.dev.platform/topics
Comment 47 cajbir (:cajbir) 2009-05-05 16:32:58 PDT
I agree about the newsgroup being a better avenue for the discussion. I'll bow out of the codec issue here until someone raises the issue on the newsgroup.
Comment 48 pd 2009-05-29 06:43:08 PDT
Nothing like a bit of vindication to cure my nausea:

"Chrome will support H.264 video and AAC audio as well as Ogg Theora video and Ogg Vorbis audio format."

http://www.builderau.com.au/news/soa/Google-Chrome-gets-HTML-video-support/0,339028227,339296704,00.htm
Comment 49 Wesley Johnston (:wesj) 2009-06-21 07:04:27 PDT
Just to be clear, this hasn't been rejected or WONTFX has it? If someone (maybe me if I get off my butt and learn a bit about DirectShow) wants to work on this for FF.Next, it could be accepted? Or have the drivers rejected this idea entirely? Is working on this a waste of time?
Comment 50 Jonas Sicking (:sicking) No longer reading bugmail consistently 2009-06-21 09:44:22 PDT
As I understand it there are people of differing opinions. So while there's a chance a patch wouldbe accepted, there's also a chance it would not.
Comment 51 Biju 2009-06-21 10:58:11 PDT
dont we have a patch already? ie, attachment 344562 [details] [diff] [review]
and a test build

(In reply to comment #7)
> I have a build for people to play with here: 
> http://pearce.org.nz/firefox-directshow-video-2008_10_24.win32.zip
> You can test it here:
> http://pearce.org.nz/player/

I thought we are not going to do this because we want a technology which is open and works on multi platform.
Comment 52 Wesley Johnston (:wesj) 2009-06-21 14:07:41 PDT
Yeah. I just assumed that the video codec changes that have happened since then had bitrotted that patch.

I posted a comment on moz.dev.platform for people to argue in if they want to. I don't think the argument has much to do with platforms. Each platform has their own built in way to deal with video from what I know, and each allows codecs to be plugged in in order to support other formats. Gecko has platform specific code all over the place to support stuff like that, and there are three different bugs here to implement the platform backend for each major OS (mobile I know nothing about).

But if someone qualified to make a decision wants to make one, feel free to tell everyone here.
Comment 53 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-21 16:01:01 PDT
Even if a volunteer produces a patch, I would not want to ship it in Firefox in the near future, for the following reasons:
-- We want to focus our energy on promoting open unencumbered codecs at this time.
-- Only a very small fraction of Windows users have a DirectShow codec for the most important encumbered codec, H.264.
-- DirectShow is underspecified and codecs are of highly variable quality. Many codecs probably will not work with Web sites that use all the rich APIs of <video>, and those bugs will be filed against us. We may not even be able to fix them. (Note that the problem is bad enough that in Windows 7, Microsoft isn't even going to allow unknown third parties to install DirectShow codecs.) We could avoid some of this problem by white-listing codecs, but then a lot of the people who want DirectShow support wouldn't be satisfied anyway.
-- Many DirectShow codecs are actually malware.
-- DirectShow codecs are quite likely to have security holes. As those holes are uncovered, we will have to track the issues and almost certainly the only possible response will be to blacklist insecure codecs (since we can't fix them ourselves). If we blacklist enough codecs, DirectShow support becomes worthless. And each rushed-out Firefox update creates a ton of work, of course.
-- Each new video backend creates additional maintenance headaches as we evolve our internal video code.

So even if we didn't care about promoting unencumbered codecs and someone gave us a working patch, shipping DirectShow support in Firefox is of limited value and creates tons of maintenance work for us.

Because of the last issue above, I wouldn't want to land a patch in mozilla-central either. But thanks to the wonders of distributed version control, someone could have their own Mercurial repository containing their own DirectShow branch, release their own browser builds, and do the maintenance work themselves.
Comment 54 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-21 16:05:13 PDT
I think the above reasons explain why so far, no browser or browser plugin vendor is using codec implementations they don't control.
Comment 55 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-21 21:15:14 PDT
Jonas corrects me: Apple ships Safari, which can use third party codecs people may have installed themselves and which Apple doesn't control. I have no idea how Apple reacts to security problems in those codecs. But at least Apple controls the Quicktime framework and the all codecs most users will have installed.
Comment 56 Christopher Blizzard (:blizzard) 2009-06-21 21:21:52 PDT
I spent some time talking with one of the Apple engineers who has been working on the video stack.  They are doing things like disabling parts of QuickTime when video playback is coming from the web.  (In this case the QT href track, which I didn't even know existed!)  So they are doing some medium-to-low level stuff in there to get ready for supporting QT on the web.  So even Apple is making choices about what to expose and what not to expose.
Comment 57 pd 2009-06-21 21:55:32 PDT
Thanks for finally giving this issue some genuine consideration Robert. It's
good to see that the reasonable issues of maintenance and security concerns are
finally being offered instead of the malarky about insisting upon 'open'
codecs.

However I think it's possible you've got it wrong re H.264

In Windows 7 the H.264 codec is installed by Microsoft. Although Vista failed,
is it that unreasonable to assume that Windows 7 will succeed? Certainly most
reviews have been very positive.

Now, assuming that Windows 7 takes substantial market share, there is a good
reason to support the H.264 codec Microsoft has kindly paid for. 

As for Mac, AFAIK H.264 has been installed with QuickTime since version 7. 

In terms of Linux there is the x264 codec, an 'open' implementation of H.264
Whilst this may not be installed by default, what is installed by default on
Linux? Not very much! Considering every other multimedia aspect of Linux
requires a separate download, I don't see why Firefox supporting x264 via
GStreamer is any different.

Regarding security, can you elaborate on how supporting a specific codec
installed and (hopefully) maintained by the OS's manufacturer, is a security
risk? How does it differ, if at all, from allowing Flash and other plugins to
run in the browser? 

Couldn't the Plugin Finder in Firefox be engineered to only ever download a
codec from X trusted location?
Comment 58 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-21 22:02:28 PDT
Flash is a gigantic problem. It's a leading exploit vector, and a leading cause of browser crashes. We don't want to expose ourselves to more of the same.

> is it that unreasonable to assume that Windows 7 will succeed?

No, Windows 7 may succeed. In 5-10 years it may be true that most Windows users have an H.264 codec. At that point, we can reopen this discussion.

As for "malarky", perhaps you missed my first point, which is that independent of these other issues, our priority is unencumbered codecs.

> Regarding security, can you elaborate on how supporting a specific codec
> installed and (hopefully) maintained by the OS's manufacturer, is a security
> risk?

Because many OS vendors are slow to do security updates. And if we're the only ones exposing their codecs to Web content, their motivation for doing quick updates is low.
Comment 59 pd 2009-06-22 01:40:18 PDT
(In reply to comment #58)
> Flash is a gigantic problem. It's a leading exploit vector, and a leading cause of browser crashes. We don't want to expose ourselves to more of the same.

Wouldn't one of the best ways to discourage Flash use be to provide a realistic alternative? Of the codecs that Flash supports (AFAIK OnVP6 and H.264) is it reasonable to suggest the majority of content will become H.264 given it's  good quality/bandwidth ratio? 

Let's say I'm a content developer pumping video out to the web. If I encode in H.264 for Flash, I could abandon using Flash in Firefox if <video/> supported H.264 and thus reduce the hazard of Flash to Firefox. Meanwhile I still present H.264 content to IE within a Flash player. Aren't content developers more likely to do that as opposed to double-encoding content both in Theora and then in H.264? 

> > is it that unreasonable to assume that Windows 7 will succeed?
> 
> No, Windows 7 may succeed. In 5-10 years it may be true that most Windows users have an H.264 codec. At that point, we can reopen this discussion.

That's good to know. Shame Chrome will steal Firefox's thunder though.

> As for "malarky", perhaps you missed my first point, which is that independent
> of these other issues, our priority is unencumbered codecs.

I used a derisory word because there is more than one definition of 'open'. 'Open' could be opening up the browser to whatever codecs the user has installed and / or can install. Your 'open' in this case could easily been seen as 'closed' to perfectly good non-politically-correct codecs. That said, I understand any concern about patent trolls. "Unencumbered" explains that. Sorry if I offended. My frustration with ideology before practicality got the better of me.
 
> > Regarding security, can you elaborate on how supporting a specific codec
> > installed and (hopefully) maintained by the OS's manufacturer, is a security
> > risk?
> 
> Because many OS vendors are slow to do security updates. And if we're the only
> ones exposing their codecs to Web content, their motivation for doing quick
> updates is low.

How can you be sure Xiphos will respond with patches quickly? Does the Mozilla 'relationship' (can't remember the term) with Xiphos guarantee that? 

How does a widely-used library like ffmpeg address this problem? I wonder how many thousands of xvid video files are downloaded every day via P2P and played in players like Media Player Classic that run a version of ffmpeg (ffdshow).

Obviously Firefox's reputation for fast security patches is critical however is it perhaps possible that the threat from malformed video files is perhaps exaggerated? Chris Blizzard's last comment on this post seems to indicate there may be room for research and solutions based on limiting exposure of certain aspects of codecs.

Usually at this point I get told to shut up and / or continue this discussion elsewhere. I think it's a real shame that bugzilla is where decisions are documented yet admins would prefer such issues are not discussed here.
Comment 60 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-22 02:19:04 PDT
(In reply to comment #59)
> Wouldn't one of the best ways to discourage Flash use be to provide a
> realistic alternative? Of the codecs that Flash supports (AFAIK OnVP6 and
> H.264) is it reasonable to suggest the majority of content will become H.264
> given it's good quality/bandwidth ratio?

It's complicated. It takes a lot more to replace Flash than just H.264 playback, there's the problem of H.264 being encumbered, and we wouldn't want to replace one stability problem with another. Note that if Flash has a security hole then the attacker will use Flash on their attack page; nothing is gained unless you disable Flash completely.

> My frustration with ideology before practicality got the better of me.

Mozilla is a non-profit organization. We balance practicality with ideology, but without ideology we'd have no reason to exist.

> How can you be sure Xiphos will respond with patches quickly? Does the Mozilla
> 'relationship' (can't remember the term) with Xiphos guarantee that?

It helps. Plus, we have the source code and can fix issues if we have to, or hire someone to fix them. Plus the codecs are bundled into Firefox and so are easy for us to update.

> How does a widely-used library like ffmpeg address this problem?

Shipping ffmpeg (like Chrome does) would address some of the issues, but raise some new ones. That's not what this bug is about though.

> Obviously Firefox's reputation for fast security patches is critical however
> is it perhaps possible that the threat from malformed video files is perhaps
> exaggerated?

It's not exaggerated. Apple fixed 10 Quicktime vulnerabilities last month.
http://www.betanews.com/article/Apples-vulnerability-patch-count-10-QuickTime-1-iTunes-0-Java/1243893833

> Chris Blizzard's last comment on this post seems to indicate
> there may be room for research and solutions based on limiting exposure of
> certain aspects of codecs.

Apple can do that because they can modify Quicktime. We can't modify DirectShow.

> Usually at this point I get told to shut up and / or continue this discussion
> elsewhere. I think it's a real shame that bugzilla is where decisions are
> documented yet admins would prefer such issues are not discussed here.

mozilla.dev.platform would be a better place.
Comment 61 Ben Bucksch (:BenB) 2009-06-22 04:31:44 PDT
> Shame Chrome will steal Firefox's thunder though.

Thunderbird did that already.

> mozilla.dev.platform would be a better place.

msg-id caf41166-2196-4f11-90ba-20c5abde1c55@g20g2000vba.googlegroups.com
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/6c1708a1c8477942/d06239cfaaa04d0c
Comment 62 Neal Gompa 2010-01-05 01:08:36 PST
Couldn't GStreamer be used on Windows, with a bridge to DirectShow to use additional codecs from there if needed? That way, Mozilla can ship a quality engine with "core" codecs that can be switched around as needed.

Opera is including GStreamer with their Opera browser for their HTML5 implementation, though there are no plans to add a bridge to DirectShow.
http://my.opera.com/core/blog/2009/12/31/re-introducing-video
Comment 63 Matthew Gregan [:kinetik] 2010-01-10 18:04:30 PST
We're pretty happy with our current approach.  Seeking works, and the implementation works well (and behaves the same way) on all three major platforms.
Comment 64 Neal Gompa 2010-01-10 18:15:18 PST
Yeah, but it seems like it would be rather painful to add other open codecs into Firefox's HTML5 support. For example, what if you wanted to add Matroska container support? Would that be easily possible with the current libtheora/libvorbis backend? I doubt it. What about FLAC, Dirac, and other open codecs? What about adding in subtitle support, from Ogg Kate or something else? An engine like GStreamer would make adding support for this a lot easier, and certainly more flexible.
Comment 65 Matthew Gregan [:kinetik] 2010-01-10 18:26:09 PST
We're not looking to add support for other codecs or container formats right now.  There's work underway for subtitle support, but it's still in development and it's certainly not clear that Gstreamer would making supporting the result of that development any easier.
Comment 66 Neal Gompa 2010-01-10 18:55:01 PST
Well, with GStreamer, you wouldn't have to write integration for each new codec and container. There is already a patch available that allows Firefox to use GStreamer for the HTML 5 <video> and <audio> tags. However, I doubt the patch includes subtitle support. It makes codec support a bit more flexible. And GStreamer already does support subtitles, albeit not a very wide range of subtitle formats.
Comment 67 Stuart Parmenter 2010-01-10 20:13:07 PST
This bug is about DirectShow.  Work is ongoing in bug 422540 for a GStreamer backend
Comment 68 Ben Bucksch (:BenB) 2010-01-21 18:56:05 PST
<http://youtube-global.blogspot.com/2010/01/introducing-youtube-html5-supported.html>

I filed bug 541286 specifically about MPEG2/4, according to comment 27. That may implemented via DirectShow or not, I don't care.
Comment 69 Paul [pwd] 2010-01-23 23:39:18 PST
Reading this as an end-user, it's highly disappointing, it comes across as if the developers are withholding functionality to make a point.  Surely that's just as bad if not worse than this whole encumbered/unencumbered codec ****.

I just want a browser that is on the cutting edge, that can play every page I throw at it and is fast. This has little or nothing to do with the politics that you guys are bogging down end-users with. If there's a way to make sure Firefox can play h.264, make it happen, reward the millions that have installed the browser for their support, don't force them to fight in your war. Allow an about:config option so we can use external decoders at the very least.
Comment 70 Chris Pearce (:cpearce) 2010-01-24 00:39:45 PST
Paul: http://weblogs.mozillazine.org/roc/archives/2009/06/directshow_and.html
Comment 71 Paul [pwd] 2010-01-25 11:17:22 PST
Chris Pearce: I'm most certainly not saying that it should be DirectShow, as an end-user, I've got to be honest and tell you that along with the fact that no matter how I get it, I just want it now. I know that Mozilla wants to push Theora/Vorbis and that's fine, however when I chose Firefox, I chose it because it was the best browser, not because I was subscribing to it's ambitions, as such, while I'm actually a huge supporter of the idea of an open web and thus Theora/Vorbis, I'm not to the point that I'm willing to sacrifice my internet experience and with YouTube and Vimeo now rolling out HTML5, I'm in a position where by as an end-user, the browser I've chosen has now intentionally denied me parts of the internet and that's a problem.

If Firefox want the support of their users on this matter, roll-out 3.6.1 and on the thanks page, ask users to opt-in to a petition for Theora/Vorbis as the base codecs. Then it's up to W3C to decide based on the vested and tangible interest of the common public. But denying content and upsetting users in the name of unencumbered patent evangelism is exactly what Chrome will want, because it's things like this which will promote the move away from Firefox.
Comment 72 Robert O'Callahan (:roc) (email my personal email if necessary) 2010-01-25 12:06:15 PST
Paul: We've always done this:
http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html

Also, is <video>+H.264 really providing a better user experience than Flash, today?
Comment 73 Ben Bucksch (:BenB) 2010-01-25 13:25:53 PST
> is <video>+H.264 really providing a better user experience than Flash, today?

Yes. Flash is a security risk, running highly annoying ads, crashing my browser often (Linux), and proprietary, therefore not an option for me, therefore I can't see YouTube videos *at all* at the moment (unless I resort to uncomfortable and timeconsuming workarounds like keepvid.com). <video> with H.264 would allow me to use YouTube properly, finally. Given how popular YouTube is, I think you can see the value. I think YouTube is the main reason why most normal users have Flash installed and enabled.

The YouTube Flash also doesn't easily allow me to store the video, and has other deficits, e.g. when seeking/skipping.
Comment 74 RNicoletto 2010-01-27 07:46:03 PST
(In reply to comment #72)
> Paul: We've always done this:
> http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html
> 
> Also, is <video>+H.264 really providing a better user experience than Flash,
> today?

Wrong question. H.264 is a standard (set of specifications) for video compression while Flash is a technology that can play videos based on H.264 standard.
Comment 75 Paul [pwd] 2010-01-27 08:27:11 PST
(In reply to comment #74)
> (In reply to comment #72)
> > Paul: We've always done this:
> > http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html
> > 
> > Also, is <video>+H.264 really providing a better user experience than Flash,
> > today?
> 
> Wrong question. H.264 is a standard (set of specifications) for video
> compression while Flash is a technology that can play videos based on H.264
> standard.

I've read pretty much everything I can find about this whole codec debate and though I initially started out feeling that Mozilla were in the wrong for not supporting the current standard, I now feel as though Mozilla are wrong for supporting the best alternative in a **** manner.

It's clear that Theora is better than h.263, but ultimately it's target is HQ video which is h.264. From what I can gather, people are hoping that Dirac will be the competitor for HD. That's a little confusing but it is what it is.

Anyway, bringing Theora up to the standard is all fine and dandy, but the fact of the matter is, the fight here is to convince the world it's worth encoding twice; H.264 is the standard for pretty much everything in the world at this point.

Thus I worry if the fight in the name of Theora is a tad bigger than everyone has expected. Yes the more stuff that is hosted on video sites the better because everything will encode to Theora on upload but what self-hosted projects? Are the encoding tools are widely hosted as everyone seems to be assuming? There is no focus here. On top of that you're thinking about all the products out there that are made to deal with this standard, i.e. phones, pda's, etc. Is everyone pretty much willing to push the end of support of most handheld technology out?

The fact as it stands is that Apple and Microsoft need convincing in order to convince W3C and until the W3C is convinced the whole progression of the internet is at a standstill.

My current hope now actually lies in this On2 acquisition (rumoured) by Google. I'm hoping that they've seen something here that can help everyone, perhaps they feel that with On2 they can blow everything else out of the water, make it open and still make it viable for them as a business. After all, Youtube is now trying to move into movie/episode rentals, so being able to offer that minus a $5,000,000 a year bill will serve them, us and everyone. But that's only if they plan on pushing something major through.

All in all though, Mozilla is pushing this enough for me, they're making this stand but how does it involve the end-user other than frustrating them by denying them content? Even Daily Motion who were once touted as a Mozilla partner and 'pioneered' the Theora video-site movement are still serving content in Flash.

Lets hope for vast improvements from the pro-Theora community, lets hopefully stop seeing "Sorry you can't play this HTML5/h.264" but rather "you're now watching content served in HTML5/Theora", there really is no buzz for Theora right now and that's not good.
Comment 76 Shawn Wilsher :sdwilsh 2010-01-27 10:38:09 PST
This bug is NOT the place to discuss which codes Mozilla is/should/must ship with Firefox.  To participate in that discussion, please go here:
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/b89b996d581325dc#
Comment 77 Siddharth Agarwal [:sid0] (inactive) 2010-02-22 07:00:52 PST
For the record: the entire discussion in this bug about H.264 on Windows 7 is moot, since Windows 7 does not actually make its H.264 decoder available through DirectShow. The decoder is only accessible through Media Foundation.
Comment 78 Masatoshi Kimura [:emk] 2010-03-17 21:48:18 PDT
Actually "Microsoft DTV-DVD Video decoder" DirectShow filter can decode H.264 on Windows 7.
http://msdn.microsoft.com/en-us/library/dd390679%28VS.85%29.aspx
Comment 79 pd 2010-03-18 16:41:56 PDT
Great find Masatoshi.

As was very predictable, IE9 will support H.264 decoding using HTML5 <video>.

Regardless of politics, Firefox and Opera are now the only browsers not supporting H.264 through HTML5 <video>.

It is time Mozilla re-thought this issue however interestingly I've not seen one post about this topic on Planet Mozilla in the couple of days since the IE9/H.264 news was released.

Even if Mozilla has to pay for a license for once, they should. Mozilla makes tens of millions of dollars in profit yearly. Where does it go? Why not spend some more on enriching the product directly rather than less directly through developer resources?
Comment 80 Neal Gompa 2010-03-18 17:05:19 PDT
The problem with that is most of Firefox's re-distributors cannot afford the licenses. And ideologically, it doesn't really sit all that well with Mozilla.

That's why we want the DShow/GST/QT backends...
Comment 81 Siddharth Agarwal [:sid0] (inactive) 2010-03-18 21:59:40 PDT
(In reply to comment #78)
> Actually "Microsoft DTV-DVD Video decoder" DirectShow filter can decode H.264
> on Windows 7.
> http://msdn.microsoft.com/en-us/library/dd390679%28VS.85%29.aspx

That's strange, AFAICT it seems to directly contradict numerous reports of DirectShow not being able to decode H.264: <http://stackoverflow.com/questions/2140120/hardware-accelerated-video-in-directshow-winforms/2142677#2142677>, or <http://social.msdn.microsoft.com/Forums/en/windowsdirectshowdevelopment/thread/fba76906-4e64-403e-8b75-15b956100670>.
Comment 82 Biju 2010-06-14 00:19:12 PDT
If issue is only viewing HTML5 video at youtube, then Minefield support WebM
* Download nightly from http://nightly.mozilla.org/
* follow instruction at http://www.webmproject.org/users/

A sample WebM video
http://www.youtube.com/watch?v=4ERrvGIpffk&feature=related
Comment 83 bb10 2010-06-14 16:09:56 PDT
Using Media Foundation would be better.
Comment 84 bb10 2010-08-26 16:37:16 PDT
Beta release of the Platform Update Supplement for Windows Vista.

http://support.microsoft.com/kb/2117917

Vista now has H.264 and AAC support in Media Foundation.
Comment 85 bb10 2010-08-26 23:08:32 PDT
(In reply to comment #53)
> Even if a volunteer produces a patch, I would not want to ship it in Firefox in
> the near future, for the following reasons:
> -- We want to focus our energy on promoting open unencumbered codecs at this
> time.
Why is there work being done on a GStreamer backend then?

> -- Only a very small fraction of Windows users have a DirectShow codec for the
> most important encumbered codec, H.264.
Media Foundation is a better solution IMO. So it's actually a very large fraction (50+%) if you count 7 and Vista's Media Foundation support. (I realize your post is rather old)

> -- DirectShow is underspecified and codecs are of highly variable quality. Many
> codecs probably will not work with Web sites that use all the rich APIs of
> <video>, and those bugs will be filed against us. We may not even be able to
> fix them. (Note that the problem is bad enough that in Windows 7, Microsoft
> isn't even going to allow unknown third parties to install DirectShow codecs.)
> We could avoid some of this problem by white-listing codecs, but then a lot of
> the people who want DirectShow support wouldn't be satisfied anyway.
MF doesn't have these problems, the needed codecs are already on the system (7) or installed with a platform update (Vista).
Microsoft is going to use it in IE9, so they will fix anything that needs to be fixed. Also, what you put between brackets is not true.

> -- Many DirectShow codecs are actually malware.
See above, use MF instead.

> -- DirectShow codecs are quite likely to have security holes. As those holes
> are uncovered, we will have to track the issues and almost certainly the only
> possible response will be to blacklist insecure codecs (since we can't fix them
> ourselves). If we blacklist enough codecs, DirectShow support becomes
> worthless. And each rushed-out Firefox update creates a ton of work, of course.
Same as above, MF.

> -- Each new video backend creates additional maintenance headaches as we evolve
> our internal video code.
> 
> So even if we didn't care about promoting unencumbered codecs and someone gave
> us a working patch, shipping DirectShow support in Firefox is of limited value
> and creates tons of maintenance work for us.
> 
> Because of the last issue above, I wouldn't want to land a patch in
> mozilla-central either. But thanks to the wonders of distributed version
> control, someone could have their own Mercurial repository containing their own
> DirectShow branch, release their own browser builds, and do the maintenance
> work themselves.
True, it does create some more work. However, it's either that or having users moving on to different browsers. People don't want to be involved in format/patent wars. They'll just choose the path of least resistance. Chrome, Safari and IE9 (will) have H.264 support and I do not see the majority of users using some weird unsupported fork of Firefox.

The only other downside is that it doesn't work on XP. But seeing as XP is brain-dead (not getting new features, no IE9), on life support and has a continuously shrinking user base, I don't think that will matter much.

With this we can support all HTML5 video and audio codecs including H.264, MP3 and AAC.
Comment 87 123 2013-05-19 06:17:12 PDT
Status: VERIFIED WONTFIX  Whiteboard:[wontfix? - see comment 53]
Okay, I'm reading comment 53. Among other things that it says, it contains following lines:

-- Only a very small fraction of Windows users have a DirectShow codec for the most important encumbered codec, H.264
-- Note that the problem is bad enough that in Windows 7, Microsoft isn't even going to allow unknown third parties to install DirectShow codecs.
-- Many DirectShow codecs are actually malware.

This is so bad, it can't even be called misleading. It's a blatant lie. If I was your poss and to my question "is is worth to add directshow support?" you answered that, you'd be fired right away.
Comment 88 forkest 2016-05-13 15:55:40 PDT
Can anything be done for this to be reconsidered? Windows XP now, I believe, is the only platform where H.264 support in Firefox can't be enabled by any way. Since the closing of this bug, there was DirectShow backend for MP3 decoding added (Bug 861693), so the foundations were laid.
Comment 89 Masatoshi Kimura [:emk] 2016-05-13 17:22:10 PDT
We will enable unencrypted H.264 playback on Windows XP once we dropped the support for non-SSE processors (see bug 1266195).

Note You need to log in before you can comment on or make changes to this bug.