Closed Bug 881512 Opened 11 years ago Closed 10 years ago

Implement enough of MSE to run the YouTube MSE player

Categories

(Core :: Audio/Video, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla31

People

(Reporter: kinetik, Assigned: kinetik)

References

(Depends on 1 open bug, Blocks 1 open bug, )

Details

Attachments

(17 files, 5 obsolete files)

1.86 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
11.72 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
11.18 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
4.78 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
20.50 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
2.31 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
8.55 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
1.23 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
1.58 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
4.23 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
2.03 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
10.64 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
2.29 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
9.31 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
2.68 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
2.05 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
5.54 KB, patch
cajbir
: review+
Details | Diff | Splinter Review
At a minimum this requires support for multiple decoders and separate audio and video tracks.
Depends on: 886196
Depends on: 896888
Depends on: 905513
Depends on: 833023
Depends on: 911059
Now that all depending bugs are fixed is this far off?
How come https://www.youtube.com/html5 shows all green except for "MSE & H.264"? Isn't the same system h.264 codec available with MSE?
Can we get an update on MediaSource in Firefox? ETA for release?
Depends on: 976037
(In reply to Feross Aboukhadijeh from comment #3)
> Can we get an update on MediaSource in Firefox? ETA for release?

From https://wiki.mozilla.org/Platform/MediaSourceExtensions :

"Timeline wise we are hoping to support enough of the spec to support the YouTube production player by the end of the year but will need to collaborate with YouTube to ensure that we're focussing on the right things. We also expect that dash.js compatibility will require a similar feature set."
That's what it said last year already... So actually youtube should work by now. (It doesn't)
(In reply to Feross Aboukhadijeh from comment #3)
> Can we get an update on MediaSource in Firefox? ETA for release?

It's still a work in progress. Bug 976037 comment 1 gives an update on what is playable and what isn't. Basically some 'DASH in JavaScript' implementations are working but the YouTube demo player is not yet.
Attached patch bug881512_p1_v0.patch (obsolete) — Splinter Review
First patch in a series, more coming soon.
Depends on: 238041
Attachment #8401715 - Attachment is obsolete: true
Attachment #8406652 - Flags: review?(cajbir.bugzilla)
Attachment #8406657 - Flags: feedback?(cajbir.bugzilla)
Attached patch p6: Switch video readers on EOF. (obsolete) — Splinter Review
Attachment #8406658 - Flags: feedback?(cajbir.bugzilla)
Attachment #8406660 - Flags: feedback?(cajbir.bugzilla)
The r? patches are ready, pending any review comments.  The f? patches have at least one known bug, so I expect to post updated versions soon.  There's probably at least one more patch to come, too.
Attachment #8406651 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8406652 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8406654 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8406658 - Flags: feedback?(cajbir.bugzilla) → feedback+
Attachment #8406653 - Flags: review?(cajbir.bugzilla) → review+
Comment on attachment 8406660 [details] [diff] [review]
p7: Crude attempt at segment parser for WebM.

Review of attachment 8406660 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/mediasource/SourceBuffer.cpp
@@ +111,5 @@
>    int64_t offset = (aTime / (double(mMediaDuration) / USECS_PER_S)) * length;
>    return offset;
>  }
>  
> +class ContainerParser {

Does this need a virtual destructor?
Attachment #8406660 - Flags: feedback?(cajbir.bugzilla) → feedback+
Attachment #8406657 - Flags: feedback?(cajbir.bugzilla) → feedback+
Attachment #8410095 - Flags: review?(cajbir.bugzilla)
Attachment #8406657 - Attachment is obsolete: true
Attachment #8406658 - Attachment is obsolete: true
Attachment #8410097 - Flags: review?(cajbir.bugzilla)
Attachment #8406660 - Attachment is obsolete: true
Attachment #8410098 - Flags: review?(cajbir.bugzilla)
Attachment #8410099 - Flags: review?(cajbir.bugzilla)
Attachment #8410097 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410098 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410099 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410101 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410102 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410103 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410095 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410793 - Flags: review?(cajbir.bugzilla)
Attachment #8410793 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410794 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410795 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410796 - Flags: review?(cajbir.bugzilla) → review+
Attachment #8410797 - Flags: review?(cajbir.bugzilla) → review+
Depends on: 1000686
Comment on attachment 8410099 [details] [diff] [review]
p8: Unpref Media Source Extensions (for WebM) in non-release builds.

Obsoleting pref change patch (p8), it has been moved to bug 1000686.
Attachment #8410099 - Attachment is obsolete: true
Try push (also includes seeking patches from bug 1000608): https://tbpl.mozilla.org/?tree=Try&rev=533fd846f2b7
Attachment #8411625 - Flags: review?(cajbir.bugzilla) → review+
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=d75260ca4ed2

	44ad5896ff61	Matthew Gregan — Bug 881512 - Make sure any released decoders are destroyed on the main thread. r=cajbir
	15fe0db2ff8d	Matthew Gregan — Bug 881512 - Explicitly initialize ReadMetadata's aTags parameter to null. Also add some null deref tests. r=cajbir
	81403176036e	Matthew Gregan — Bug 881512 - Fix bogus indexing of audio/video readers by ditching audio/video reader arrays and using the parent decoder directly. r=cajbir
	6c69283f7713	Matthew Gregan — Bug 881512 - Delay decoder initialization by indicating that we're waiting for subdecoders to be created. r=cajbir
	1eddd13262de	Matthew Gregan — Bug 881512 - Don't mix refcounting with auto ptr use. Rather than try to hang on to the reader, punch the appropriate holes through via the state machine class. r=cajbir
	5fbc68a34f62	Matthew Gregan — Bug 881512 - Remove useless static_casts. r=cajbir
	199c040d09f0	Matthew Gregan — Bug 881512 - Use the base video and audio queues. Currently works by shifting media data from the sub-reader queues on each decode call. Future work could allow the sub-readers to use the base queues directly. r=cajbir
	c116a4499d82	Matthew Gregan — Bug 881512 - Don't pretend the entire resource is always completely cached. r=cajbir
	5ea033583b20	Matthew Gregan — Bug 881512 - Temp fix: don't call mParentDecoder functions while SBD runs off decode thread. Revert when SBD runs on state machine's task queue. r=cajbir
	a14f9763bf8f	Matthew Gregan — Bug 881512 - Crude attempt at segment parser for WebM. r=cajbir
	cf3e31d25944	Matthew Gregan — Bug 881512 - Switch video readers on EOF. r=cajbir
	59ffec98e96f	Matthew Gregan — Bug 881512 - Start processing multiple decoders. r=cajbir
	18754facd070	Matthew Gregan — Bug 881512 - Add hidden pref to ignore MIME type in AddSourceBuffer and IsTypeSupported. For experimental/debugging use only. r=cajbir
	f8ece490919d	Matthew Gregan — Bug 881512 - Hide MediaSourceResource implementation details from HTMLMediaElement. r=cajbir
	d978fe16633a	Matthew Gregan — Bug 881512 - Allow a SourceBuffer to own multiple subdecoders. Switch decoders on abort(). r=cajbir
	c570bcfd29b4	Matthew Gregan — Bug 881512 - Add start (and rename end) time helper to TimeRanges. r=cajbir
Summary: Implement enough of MSE to run the YT demo player → Implement enough of MSE to run the YouTube MSE player
Attachment #8413148 - Flags: review?(cajbir.bugzilla) → review+
Comment on attachment 8413148 [details] [diff] [review]
p18: Use MediaTaskQueue to run MSE tasks rather than a per-MSE thread.

p18 was planned for a near future cleanup in a different bug, but it turned out to be needed to fix an orange on TBPL debug builds:

3587 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/mediasource/test/test_MediaSource.html | Assertion count 1 is greater than expected range 0-0 assertions.

...which didn't show up on try because the same test failed for a different (innocuous) reason.
Attachment #8413148 - Attachment description: p18: Use MediaTaskQueue to run MSE tasks rather than a per-MSE thread. r=cajbir → p18: Use MediaTaskQueue to run MSE tasks rather than a per-MSE thread.
YouTube still says that the browser does not support Media Source Extensions even with the patched build. Is it YouTube's issue?
The functionality is hidden behind a pref for now; opening that up (for WebM only) is bug 1000686.
With both media.mediasource.enabled and media.mediasource.ignore_codecs set to true, youtube videos just play for a few seconds and skip to the end, not sure if this is normal.
With media.mediasource.enabled set to true and media.mediasource.ignore_codecs set to false, they seem to play normally but only 360p/720p is available, which I guess is normal.

I'm using the latest nightly 31.0a1 (2014-04-27) on Arch Linux.
(In reply to AnAkkk from comment #39)
> With both media.mediasource.enabled and media.mediasource.ignore_codecs set
> to true, youtube videos just play for a few seconds and skip to the end, not
> sure if this is normal.
> With media.mediasource.enabled set to true and
> media.mediasource.ignore_codecs set to false, they seem to play normally but
> only 360p/720p is available, which I guess is normal.

If you right click on the video, choose "stats for nerds", do you see it using the Dash player and WebM?
With media.mediasource.ignore_codecs = true, DASH:yes, video/mp4 and it plays for exactly 3 seconds before skipping to the end.
With media.mediasource.ignore_codecs = false, DASH:no, video/mp4 and it works fine.
Sorry, it is only supposed to work on WebM videos?
Nevermind. On some videos it does show DASH: yes, video/webm and I have the exact same issue.
Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1002019 because setting media.mediasource.enabled to true and enabling YouTube's HTML5 player causes Nightly to freeze/hang/quit responding.
(In reply to AnAkkk from comment #42)
> Sorry, it is only supposed to work on WebM videos?

The current implementation was built and tested with WebM. It is planned to improve support to MP4. It may work already on some platforms or combinations of decoder but I wouldn't expect it.
(In reply to AnAkkk from comment #41)
> With media.mediasource.ignore_codecs = true, DASH:yes, video/mp4 and it
> plays for exactly 3 seconds before skipping to the end.

media.mediasource.ignore_codecs is for developer-only testing, please don't set it unless you have a good reason, and expect problems (but don't report them) when you do.

(In reply to AnAkkk from comment #43)
> Nevermind. On some videos it does show DASH: yes, video/webm and I have the
> exact same issue.

Please file a new bug and CC me.  Reporting new issues in a closed bug has a high probability of the issue being overlooked.
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #47)
> https://hg.mozilla.org/mozilla-central/rev/bc6a9d65b84f

Oops, that's actually for bug 1000686, but I forgot to fix the commit message when splitting the bugs.
Depends on: 1002404
Depends on: 1001952
(In reply to Matthew Gregan [:kinetik] from comment #46)
> Please file a new bug and CC me.  Reporting new issues in a closed bug has a
> high probability of the issue being overlooked.

This won't be necessary, my issue seem to be fixed with the latest nightly.
Depends on: 1005366
In my case, (Nightly 32.0a1, BuildID=20140505030202 )
before setting media.mediasource.enabled to true, 3/6 are green in https://www.youtube.com/html5 :
"HTMLVideoElement", "H.264" and "WebM VP8".

After setting media.mediasource.enabled to true, 5/6 are green :
all except for "MSE & H.264".
Is this expected?
Shouldn't the "MSE & H.264" be green now too?
(In reply to Kostas from comment #50)
> Shouldn't the "MSE & H.264" be green now too?

MP4 (H.264, etc.) aren't supported in MSE yet.  We're concentrating on WebM on YouTube right now, other functionality will follow.
(In reply to Matthew Gregan [:kinetik] from comment #51)
> (In reply to Kostas from comment #50)
> > Shouldn't the "MSE & H.264" be green now too?
> 
> MP4 (H.264, etc.) aren't supported in MSE yet.  We're concentrating on WebM
> on YouTube right now, other functionality will follow.
Thanks for the quick reply.
So, the target milestone for this bug - Firefox 31 has been released, but MSE on YouTube still doesn't work properly. Seeking sometimes takes a lot of time, seeking back doesn't work at all and repeat also doesn't work.
I tried the latest Nightly, but it seems that it only got worse - sometimes seeking just hangs.
Is bug 1024858 the only reason why it's still broken or there're some other YouTube related bugs?
Flags: needinfo?(kinetik)
Follow bug 1000686, which is tracking enabling this functionality by default in nightly builds.  This bug covered the initial implementation, but there are a number of bugs remaining which are being worked on now.
Flags: needinfo?(kinetik)
2 days ago youtube deprecated flash (now max 720p) and upped html5 functionality significantly, adding 60fps and super high resolutions to html5.

I think the slow approach to this needs changing I consider it now urgent to have fully working mp4 support on html5, the internet is plastered with "chrome only for 6ofps youtube"

It also works on IE11 in windows 8 although the internet is not making that clear its just saying it works on IE11 full stop but it does require windows 8.
I just switched from Firefox to Nightly and I manually enabled MSE & h264 so I can use YouTube.

The two biggest problems I have with it right now are:

- Seeking doesn't work. Seeking within the 'buffered range' is slow and takes ~10 seconds. Seeking beyond the buffered range doesn't work at all.

- Videos with inVideo branding (example: https://srv.tuxplace.nl/hosted/screenshots/2014-11-03-13-36-00.png , icon in the top right corner) will freeze at the exact moment the icon will appear.
I wanted to provide some help to anyone frustrated by this. Here is an article about a very helpful plugin that forces flash in HD to run in nightly instead of html5, but you can select between the two. http://www.ghacks.net/2014/10/20/force-flash-video-on-youtube-in-firefox/ and a link to the plugin http://barisderin.com/?p=2225 . For the time being, this makes everything run smoothly.
(In reply to DrDumbass from comment #57)
> I wanted to provide some help to anyone frustrated by this. Here is an
> article about a very helpful plugin that forces flash in HD to run in
> nightly instead of html5, but you can select between the two.
> http://www.ghacks.net/2014/10/20/force-flash-video-on-youtube-in-firefox/
> and a link to the plugin http://barisderin.com/?p=2225 . For the time being,
> this makes everything run smoothly.

This comment is not quite related but I want to say using Flash was obsoleted because of slowness and lag. I'd rather see 320p than using Flash on my slow computer.
(In reply to daniel.2345 from comment #58)
> (In reply to DrDumbass from comment #57)
> > I wanted to provide some help to anyone frustrated by this. Here is an
> > article about a very helpful plugin that forces flash in HD to run in
> > nightly instead of html5, but you can select between the two.
> > http://www.ghacks.net/2014/10/20/force-flash-video-on-youtube-in-firefox/
> > and a link to the plugin http://barisderin.com/?p=2225 . For the time being,
> > this makes everything run smoothly.
> 
> This comment is not quite related but I want to say using Flash was
> obsoleted because of slowness and lag. I'd rather see 320p than using Flash
> on my slow computer.

I'm on the opposite side. I have a fast laptop and I'm fed up being forced webm which doesn't use hardware acceleration. I'd much rather have Flash and h.264 with HW acceleration over webm eating battery life and burning my lap. Anyway, bugzilla isn't an Internet forum so let's end this.
Depends on: 1113776
Hi all, in bug #833023 Jan Gerber said that:

>> - http://www.youtube.com/html5 states that the latest Aurora does not
>> support "MSE & WebM VP9".
>For MSE support, check Bug 881512

And I came here, saw this bug is marked as "fixed", but in https://www.youtube.com/html5 it still shows "MSE & WebM VP9" not supported. (I'm using latest Nightly) 

How could I fix this or it has not been implemented into Firefox yet?
It is temporarily disabled (see bug 1097436 and bug 1108059).
On win7 youtubne reports all codecs available (without ignore_codecs) but when youtube tries to start playing it spins for ages and eventually falls back to non 60fps codec.

beta testers are reporting its been working fine and are astonished its not in the release version

so whast the issue with ff36?

code removed?
not working yet on windows?
You need to log in before you can comment on or make changes to this bug.