778617 - (MSE) [meta] Tracking bug for Media Source Extensions API implementation

Status: NEW

(Core :: Audio/Video: Playback, defect, P2)

Description

[Matthew Gregan [:kinetik]]

This proposal extends HTMLMediaElement to allow JavaScript to generate media streams for playback. Allowing JavaScript to generate streams facilitates a variety of use cases like adaptive streaming and time shifting live streams.

Comment 1

[Matthew Gregan [:kinetik]]

Attachment: WebIDL/DOM stubs - patch v0

I plan to work on this in the near future, but I've got a couple of things to finish first.  In the mean time, I stubbed out the DOM side of this to give myself an excuse to play with the new WebIDL bindings.

Comment 2

[David Dahl :ddahl]

is there a bug filed for handling mediaStream decoding/decryption?

Comment 3

[Ralph Giles (:rillian)]

(In reply to David Dahl :ddahl from comment #2)
> is there a bug filed for handling mediaStream decoding/decryption?

You mean like the webm encryption stuff? Not that I'm aware of. Of course, the MediaSource API makes it possible to implement many scrambling and encryption methods in javascript.

Comment 4

[Matthew Gregan [:kinetik]]

And if you're referring to the Encrypted Media Extensions (http://dvcs.w3.org/hg/html-media/raw-file/default/encrypted-media/encrypted-media.html), there's no bug that I'm aware of, and I don't think there is a plan to implement it at this stage.

Comment 5

[whadar]

Any progress? Is there any estimation on when this API will be implemented?

Comment 6

[Feross Aboukhadijeh]

(In reply to Matthew Gregan [:kinetik] from comment #1)
> I plan to work on this in the near future

Matthew - are you still planning to work on this? If not, can this be reassigned? With WebRTC support around the corner, I think the MediaSource API will allow the creation of some pretty amazing applications.

Please let me know the current status of this bug. Thanks!

Comment 7

[Matthew Gregan [:kinetik]]

Yes, I picked it up again recently.  There are a first set of patches coming shortly.

Comment 8

[Feross Aboukhadijeh]

Excellent!

Comment 9

[guy paskar]

Anything new? when should we expect first patch?

Comment 10

[Feross Aboukhadijeh]

Hey Matthew, what's the latest on the Media Source Extensions API implementation? You mentioned that a first set of patches would be out around 1 month ago. Any progress?

Comment 11

[Shachar]

http://techcrunch.com/2013/04/15/netflix-html5-chromebooks/
http://www.webmonkey.com/2013/04/netflix-plans-to-ditch-silverlight-for-html5/
http://techblog.netflix.com/2013/04/html5-video-at-netflix.html

Comment 12

[u459114]

List things that we may need to do after Bug 855130 landing.

1. Relation between MediaSource & SourceBuffer should be 1-to-N
2. MediaSource::addSourceBuffer: param(type) MIME type checking.
3. Depend on Bug 744896: to enable/ disable audio track/ to select a video.
4. HTMLMediaElement behavior change 
https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#htmlmediaelement-extensions

Comment 13

[Matthew Gregan [:kinetik]]

Items 2 and 4 are partially done in the latest patch attached to bug 855130.  I'll file bugs for the others (and a number of other things) that block this bug over the next day or so.

Comment 14

[me]

Hi, 

How is this going? Are you working on it or waiting for some of the dependencies to clear? Can you give a hint of when it will be milestone?

Thank you

Dennis

Comment 15

[Matthew Gregan [:kinetik]]

It's being worked on, most of the action is in the dependent bugs.  The proposed schedule is here: https://wiki.mozilla.org/Platform/MediaSourceExtensions

Comment 16

[vulcain]

This API is needed if you want to see youtube video on 1080p:
http://googlesystem.blogspot.fr/2013/10/youtube-drops-support-for-1080p-streams.html

" YouTube removed the non-DASH streams for 480p and 1080p. This means that you'll no longer be able to watch 1080p videos in YouTube's HTML5 player if you use Firefox or Safari. "

Comment 17

[anvar.sosnitski]

The proposed schedule is from the last year. Is the schedule still actual or not? What is the timeline when Firefox would be able to play videos using DASH reference implementation at dashif.org?

Comment 18

[cajbir (:cajbir)]

(In reply to anvar.sosnitski from comment #17)
> The proposed schedule is from the last year. Is the schedule still actual or
> not? What is the timeline when Firefox would be able to play videos using
> DASH reference implementation at dashif.org?

Some DASH implementations are working in nightly builds - those that use WebM/VP9. See for example:

http://cd.pn/mse/webm/dash-player.html

Support for YouTube's DASH player is ongoing. Nightly build plus bug 989888 allows playback of VP9 Dash streams on YouTube. More work on that is needed for seeking and stream switching however.

I have not yet investigated dashif.org support due to it requiring (AFAIK) H.264 support and we've had muxer issues to deal with in that regard. I would hop ewe can get on to testing and getting it working in the near future.

Comment 19

[Tantek Çelik]

Shipping Chrome supports the MSE API:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/RG1dEbQEu8E
As does IE 11:
http://msdn.microsoft.com/en-us/library/ie/dn594470%28v=vs.85%29.aspx

Comment 20

[Will Law]

(In reply to cajbir (:cajbir) from comment #18)
> (In reply to anvar.sosnitski from comment #17)
> > The proposed schedule is from the last year. Is the schedule still actual or
> > not? What is the timeline when Firefox would be able to play videos using
> > DASH reference implementation at dashif.org?
> 
> Some DASH implementations are working in nightly builds - those that use
> WebM/VP9. See for example:
> 
> http://cd.pn/mse/webm/dash-player.html
> 
> Support for YouTube's DASH player is ongoing. Nightly build plus bug 989888
> allows playback of VP9 Dash streams on YouTube. More work on that is needed
> for seeking and stream switching however.
> 
> I have not yet investigated dashif.org support due to it requiring (AFAIK)
> H.264 support and we've had muxer issues to deal with in that regard. I
> would hop ewe can get on to testing and getting it working in the near
> future.

@cajbir - I help manage the dash.js project https://github.com/Dash-Industry-Forum/dash.js which is the reference client for the DASH Industry Forum and also the preferred MSE implementation for many members including Microsoft and Akamai. We have good implementations under Chrome and IE11 and would love to add Firefox to that mix, not just for our companies but for the MSE ecosystem in general. If there is anything we can do from our end to help accelerate testing and/or code changes for H.264 ISOBMFF support please don't hesitate to reach out. 

Cheers

Will Law

Comment 21

[Aymeric Vitte]

Is there a first FF stable release where MSE can be used or is it still in Nightly?

Comment 22

[Nazar Mokrynskyi]

Dash on youtube doesn't work even in Nightly 32. I think we have to wait for all issues on which this one depends until get it working in nightly. It is too early to talk about stable.

Comment 23

[avada]

(In reply to Aymeric Vitte from comment #21)
> Is there a first FF stable release where MSE can be used or is it still in
> Nightly?

(In reply to Nazar Mokrynskyi from comment #22)
> Dash on youtube doesn't work even in Nightly 32. I think we have to wait for
> all issues on which this one depends until get it working in nightly. It is
> too early to talk about stable.

It works with VP9/WEBM video streams on youtube. But videos that have them are by far the minority. Also they have around half the bitrate and are more resource demanding to decode.
For whatever (irrational) reason h.264 - which is overwhelmingly the dominant format - is ignored until everything else is done.

Comment 24

[cajbir (:cajbir)]

(In reply to avada from comment #23)
> For whatever (irrational) reason h.264 - which is overwhelmingly the
> dominant format - is ignored until everything else is done.

This is an inappropriate comment. Please don't pass judgement (using words like irrational) on why things are done, assume good intent from the people doing the work. H.264 support relies on other work which is being done in parallel.

Comment 25

[Aymeric Vitte]

Do you have any idea when this could be available?

Comment 26

[Stefan Neufeind]

Afaik h264 in general meanwhile works. Does that mean support for dash/h264 as well (or maybe soon down the road)?

Comment 27

[Matthew Gregan [:kinetik]]

Yes, see bug 1027875.

Comment 28

[Leman Bennett [Omega]]

As of today, Google has locked Nightly users out of being able to switch back an forth between HTML5 and Flash. 

HTML5 is now potentially the permanent default on YouTube for Fx33+. This should probably become a higher priority if not already YouTube's policy sticks.

Comment 29

[Bill Walker [:bwalker] [@wfwalker]]

Sandip, how can we nominate this as a b2g 2.2 feature?

Comment 30

[info]

Now that there is support for H264 in FirefoxNightly when I'm trying to use H264 with MSE I'm getting a 'NotSupportedError: Operation is not supported' error when I'm calling var sourceBuffer = ms.addSourceBuffer('video/mp4;codecs="avc1.42c00d"');

Comment 31

[bugzilla]

Of course, have you ever look to this site (youtube.com/html5)?
MSE with H264 is not implemented enough today. Just stay tuned, when this bug are resolved, it should work:)

Comment 32

[Jet Villegas (inactive)]

> feature-b2g: --- → 2.2?

We are currently reviewing the bug backlog for this feature, and the resulting schedule changes. Please don't set this flag until after that review. Thanks!

Comment 33

[John]

(In reply to info from comment #30)
> Now that there is support for H264 in FirefoxNightly when I'm trying to use
> H264 with MSE I'm getting a 'NotSupportedError: Operation is not supported'
> error when I'm calling var sourceBuffer =
> ms.addSourceBuffer('video/mp4;codecs="avc1.42c00d"');

I have the same error. 
Does Anyone know how to fix it?

Comment 34

[John]

(In reply to John from comment #33)
> (In reply to info from comment #30)
> > Now that there is support for H264 in FirefoxNightly when I'm trying to use
> > H264 with MSE I'm getting a 'NotSupportedError: Operation is not supported'
> > error when I'm calling var sourceBuffer =
> > ms.addSourceBuffer('video/mp4;codecs="avc1.42c00d"');
> 
> I have the same error. 
> Does Anyone know how to fix it?

Oh, don't support yet, video/webm is OK.

Comment 35

[Jean-Yves Avenard [:jya]]

(In reply to John from comment #33)
> (In reply to info from comment #30)
> > Now that there is support for H264 in FirefoxNightly when I'm trying to use
> > H264 with MSE I'm getting a 'NotSupportedError: Operation is not supported'
> > error when I'm calling var sourceBuffer =
> > ms.addSourceBuffer('video/mp4;codecs="avc1.42c00d"');
> 
> I have the same error. 
> Does Anyone know how to fix it?

set media.mediasource.ignore_codecs to true

Comment 36

[Chris]

guys need to ramp up the pace.

2 days ago flash got deprecated on youtube now is max 720only.

Html5 now supports 60fps and super high res videos.

90% of high quality html5 videos are mp4.

All these work in chrome and win8/10 version of IE11.

Comment 37

[Daryl Wise]

(In reply to Chris from comment #36)
> guys need to ramp up the pace.
> 
> 2 days ago flash got deprecated on youtube now is max 720only.
> 
> Html5 now supports 60fps and super high res videos.
> 
> 90% of high quality html5 videos are mp4.
> 
> All these work in chrome and win8/10 version of IE11.

Agreed! I hope soon Firefox can enjoy lag free HTML5 60fps video content! Get the MSE's working!

Comment 38

[Sylvestre Ledru [:Sylvestre]]

Release Note Request (optional, but appreciated)
[Why is this notable]: A new area for videos
[Suggested wording]: Media Source Extensions API implemented
[Links (documentation, blog post, etc)]: Any link?

AFAIK, this will ship with 36.

Comment 39

[Ralph Giles (:rillian)]

Specification is https://w3c.github.io/media-source/

Comment 40

[Kai-Chih Hu [:khu]]

It doesn't have to be in 2.2. But, need to continue working on it. 
Bobby, please follow up this topic.

Comment 41

[Sylvestre Ledru [:Sylvestre]]

(In reply to Ralph Giles (:rillian) from comment #39)
> Specification is https://w3c.github.io/media-source/
Thanks but I was hoping for something more general and less technical.

Comment 42

[Bobby Chien]

continue work and mark as tracking-b2g:+ for future release.

Comment 43

[Karl Tomlinson (:karlt)]

(In reply to Sylvestre Ledru [:sylvestre] from comment #38)
> Release Note Request (optional, but appreciated)
> [Why is this notable]: A new area for videos
> [Suggested wording]: Media Source Extensions API implemented
> [Links (documentation, blog post, etc)]: Any link?
> 
> AFAIK, this will ship with 36.

See bug 1112761.  API use in 36 will be limited.

Comment 44

[Sylvestre Ledru [:Sylvestre]]

Added to the release notes with "Media Source Extensions API implemented (Youtube only)" as wording.
Let me know if someone has an issue with this.

Comment 45

[Sylvestre Ledru [:Sylvestre]]

Update to "Implemented a subset of the Media Source Extensions (MSE) API to allow native HTML5 playback on YouTube. Full support is on the way". Thanks to Bobby for the new version!