794171 - Brightcove HTML5/H.264 video playback support not detected

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P1)

Description

[Aaron Train [:aaronmt]]

Test-page: http://admin.brightcove.com/html5support/index.html
Player test-page: http://support.brightcove.com/en/docs/video-test-html-5

Brightcove serves video to thousands of websites; we should be compatible.

--
Nightly (09/25)
Samsung Galaxy Nexus (Android 4.1.1)

Comment 1

[Lawrence Mandel [:lmandel] (use needinfo)]

Here's the Brightcove test for H.264 support from the test page that Aaronmt linked to above:

var v = document.createElement("video");
...
var h264Support = !!(v.canPlayType && v.canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));

Does Fennec support the canPlayType function on the video element?
Does Fennec support the video/mp4 type?
Does Fennec support the avc1.42E01E or mp4a.40.2 codecs?

Comment 2

[Lawrence Mandel [:lmandel] (use needinfo)]

I just tried the player test page on Fennec Nightly (2012-10-05). The video does play, albeit very choppy.

Comment 3

[Aaron Train [:aaronmt]]

(In reply to Lawrence Mandel [:lmandel] from comment #2)
> I just tried the player test page on Fennec Nightly (2012-10-05). The video
> does play, albeit very choppy.

Uninstall Flash if it is installed.

Comment 4

[Chris Peterson [:cpeterson]]

(In reply to Lawrence Mandel [:lmandel] from comment #1)
> Here's the Brightcove test for H.264 support from the test page that Aaronmt
> linked to above:
> 
> var v = document.createElement("video");
> ...
> var h264Support = !!(v.canPlayType && v.canPlayType('video/mp4;
> codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));
> 
> Does Fennec support the canPlayType function on the video element?
> Does Fennec support the video/mp4 type?
> Does Fennec support the avc1.42E01E or mp4a.40.2 codecs?

That's very interesting. I just tested Fennec's video.canPlayType() and got the following results:

 * canPlayType('video/webm') -> "probably"
 * canPlayType('video/mp4') -> "maybe"
 * canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"') -> "" (i.e. NO)

Comment 5

[Chris Peterson [:cpeterson]]

Attachment: part-1-add-android-codecs.patch

Part 1: Add H.264 codecs that are commonly supported on Android.

Henri, I CC'd you because you had some insightful comments on GStreamer bug 760140.

This patch fixes Android's codec list to use the proper codec names from the HTML5 spec [1]. To be conservative, I've only included the Baseline and Main profiles and levels commonly recommended for mobile video.

Google's recommends Baseline Profile [2] when encoding low, high, or even HD quality videos for Android devices.

Zencoder recommends Main Profile [3] for high quality videos. The iPad 1 and iPhone 4 support Main Profile Level 3.1. The iPad 2 and iPhone 4S support High Profile Level 4.1. The iPhone 3GS and earlier only support Baseline Profile Level 3.0. 

[1] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type
[2] https://developer.android.com/guide/appendix/media-formats.html#recommendations
[3] http://blog.zencoder.com/2012/01/24/encoding-settings-for-perfect-ipadiphone-video

Comment 6

[Chris Peterson [:cpeterson]]

Attachment: part-2-add-codec-comments.patch

Part 2: Add comments to GStreamer and Gonk's list of H.264 codecs.

Comment 7

[Chris Peterson [:cpeterson]]

btw:

1. I suspect "avc1.4D001F" is misspelled and should be "avc1.4D401F".

2. Should canPlayType()'s codec name check be case-insensitive? Our current implementation is case-sensitive [1].

The HTML5 spec does not specify case-sensitivity, but its examples go out of their way to specify mixed-case names like "avc1.42E01E" [2]. On the other hand, Apple's streaming media FAQ [3] lists lowercase codec names like "avc1.42e01e".

Chrome and Safari are case-sensitive when comparing the codec name prefix like "avc1.", but the case of the codec name suffix is moot because they claim to "probably" support any "avc1.*" wildcard name.

[1] https://hg.mozilla.org/mozilla-central/annotate/99898ec9976a/content/html/content/src/nsHTMLMediaElement.cpp#l2363
[2] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type
[3] https://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html#//apple_ref/doc/uid/TP40008332-CH103-SW1

Comment 8

[cajbir (:cajbir)]

Let's be case insensitive. I can imagine that there will be a lot of mis-casings in web code. These codec names are so similar there will probably be a ton of mistypings too unfortunately.

Comment 9

[Henri Sivonen (:hsivonen) (not reading bugmail until 2023-12-11)]

Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

(In reply to Chris Peterson (:cpeterson) from comment #7)
> 2. Should canPlayType()'s codec name check be case-insensitive? Our current
> implementation is case-sensitive [1].

The RFC says the ISO BMF namespace is case-sensitive: http://tools.ietf.org/html/rfc6381#section-3.3

The canPlayType() spec doesn’t say anything to override the RFC: http://www.whatwg.org/specs/web-apps/current-work/#dom-navigator-canplaytype

That said, I wouldn’t oppose to making it ASCII case-insensitive if you file a WHATWG spec bug to request ASCII case-insensitivity and citing the outright wildcarding in WebKit.

> On the other
> hand, Apple's streaming media FAQ [3] lists lowercase codec names like
> "avc1.42e01e".
>
> Chrome and Safari are case-sensitive when comparing the codec name prefix
> like "avc1.", but the case of the codec name suffix is moot because they
> claim to "probably" support any "avc1.*" wildcard name.

Ouch. Those are both good reasons to be ASCII case-insensitive.

I’m marking this feedback+, but I am worried about enumerating a few AVC profile/level combinations when the competitions just gives up and wildcards them. If we don’t end up wildcarding e.g. because we really don’t support the High profile or something, I think it would be prudent to parse out the stuff after “avc1.” and make sure we support everything in some ranges of values that make sense. I don’t want to suggest slowing down this fix, though, as this is a definite improvent over the current code.

I notice that you are removing mp4v. Good. I hope we have code elsewhere that bans MPEG-4 Visual from being decoded. It would be sad to end up supporting it accidentally on mobile and having to pay royalties for yet another encumbered codec (that’s already obsolete even) for compatibility on desktop.

> [2] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type

It’s generally safer to read the WHATWG spec. In this case, WHATWG and W3C don’t disagree, but the W3C HTML 5.0 release branch will get less and less precise over time as things that don’t meet the CR exit criteria get removed. (See also http://w3cmemes.tumblr.com/post/31865121758/the-joker-shares-his-approach-on-css2-1-issues )

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to Henri Sivonen (:hsivonen) from comment #9)
> I’m marking this feedback+, but I am worried about enumerating a few AVC
> profile/level combinations when the competitions just gives up and wildcards
> them. If we don’t end up wildcarding e.g. because we really don’t support
> the High profile or something, I think it would be prudent to parse out the
> stuff after “avc1.” and make sure we support everything in some ranges of
> values that make sense. I don’t want to suggest slowing down this fix,
> though, as this is a definite improvent over the current code.

I can file a follow-up bug to investigate codec name sensitivity and/or "avc1.*" wildcard support. That work is beyond the scope of this bug because it would affect other platforms and possibly change the MPAPI for platform decoders.

Comment 11

[Chris Peterson [:cpeterson]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e0fff9d29111
https://hg.mozilla.org/integration/mozilla-inbound/rev/7474711c6ebd

I plan to uplift patch 1 to Aurora 18 and Beta 17 after it lands on mozilla-central.

Comment 12

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/e0fff9d29111
https://hg.mozilla.org/mozilla-central/rev/7474711c6ebd

Comment 13

[Aaron Train [:aaronmt]]

Can one confirm if this does not fix the video player in the URL in comment #0, as with these changes I can still not get the video to play. Tapping the play button does nothing in their test player.

I can verify that the tests pass in the the test-page URL on mozilla-inbound though.

Comment 14

[Chris Peterson [:cpeterson]]

Aaron, I think this fix was merged to mozilla-central too late for last night's build.

Comment 15

[Aaron Train [:aaronmt]]

(In reply to Chris Peterson (:cpeterson) from comment #14)
> Aaron, I think this fix was merged to mozilla-central too late for last
> night's build.

This was tested on an inbound build from a couple hours ago.

Comment 16

[Chris Peterson [:cpeterson]]

Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

!!! This bug is a high priority because it is blocking a Mozilla/Wired.com marketing campaign (bug 794100).

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Not a regression
User impact if declined: Brightcove video sites (like wired.com and nytimes.com) won't play HTML5 video
Testing completed (on m-c, etc.): QA verified on m-i
Risk to taking this patch (and alternatives if risky): Surprising low risk. This patch does not change any video decoding code; it just changes the list of supported video formats returned from the video.canPlayType() JS API.
String or UUID changes made by this patch: N/A

Comment 17

[Alex Keybl [:akeybl]]

Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

[Triage Comment]
We can land now, but this won't be built for beta until Tuesday and released later in the week.

Comment 18

[Chris Peterson [:cpeterson]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/8ee98cea0f22

Comment 19

[Chris Peterson [:cpeterson]]

https://hg.mozilla.org/releases/mozilla-beta/rev/31675d03cc9b

Comment 20

[Chris Peterson [:cpeterson]]

   doublec, can you please take a look at this bug? My change in bug 794171 fixed Brightcove's compatibility test page, but did not fix Brightcove's video player test or Wired.com's videos.

 * Compatibility test: http://admin.brightcove.com/html5support/index.html
 * Player test: http://support.brightcove.com/en/docs/video-test-html-5

Comment 21

[Chris Pearce [:cpearce (Not reading bugmail)]]

It seems to me that the Player Test page is at fault here. I get Flash served to me with Chrome (which is the poster child for H.264 support) on Linux and Win7 when visiting the Player Test page.

I also tried with the GStreamer backend on Linux desktop Firefox, and it tells me that the page is using canPlayType() to check whether 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"' is supported ("probably" is). The "Player Support Test" page is checking the same string, and on my android phone the "Player Support Test" page is claiming that H.264 is supported, so I assume we're returning "probably" for 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"' there too.

I think this is a bug in BrightCove's code, rather than ours.

Comment 22

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Chris Pearce (:cpearce) from comment #21)
> I think this is a bug in BrightCove's code, rather than ours.

Belay that. I hooked up our media logging to logcat on Android and I can see that a H.264 decoder is being instantiated and is playing. Will investigate further tomorrow...

Comment 23

[Chris Pearce [:cpearce (Not reading bugmail)]]

I'm pretty sure Fennec's continued inability to play H.264 on Brightcove is due to Brightcove's player sniffing the user agent and not handling Fennec. See the results of my investigation in bug 794100 comment 17.

I'm going to resolve this fixed again, and we'll continue tracking the evangelism efforts to get Brightcove to fix their player to on their side in Bug 794100.

Comment 24

[Ron Piovesan]

Hi team,

This issue with Brightcove is holding up one of our apps partners. has it been resolved yet?

Thanks
Ron

Comment 25

[Chris Peterson [:cpeterson]]

Ron, does the partner have a test case or web page that reproduces their problem? What version(s) of Firefox are they testing?

Comment 26

[Ron Piovesan]

It is the Wired app in the Marketplace. We can't properly play the video https://marketplace.firefox.com/app/wiredcom/?src=mkt-search

Comment 27

[Kevin Brosnan [Ex-Mozilla]]

Looks like http://www.wired.com/underwire/2013/01/the-monitor-tries-to-warm-up-your-winter/ is an example.