Last Comment Bug 794171 - Brightcove HTML5/H.264 video playback support not detected
: Brightcove HTML5/H.264 video playback support not detected
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: Trunk
: ARM Android
: P1 normal (vote)
: mozilla19
Assigned To: Chris Pearce (:cpearce)
:
Mentors:
Depends on: 800600
Blocks: brightcove
  Show dependency treegraph
 
Reported: 2012-09-25 11:09 PDT by Aaron Train [:aaronmt]
Modified: 2013-01-24 16:54 PST (History)
12 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
unaffected
fixed
fixed
fixed
+


Attachments
part-1-add-android-codecs.patch (1.21 KB, patch)
2012-10-10 22:49 PDT, Chris Peterson [:cpeterson]
cajbir.bugzilla: review+
hsivonen: feedback+
akeybl: approval‑mozilla‑aurora+
akeybl: approval‑mozilla‑beta+
Details | Diff | Review
part-2-add-codec-comments.patch (1.18 KB, patch)
2012-10-10 22:50 PDT, Chris Peterson [:cpeterson]
cajbir.bugzilla: review+
Details | Diff | Review

Description Aaron Train [:aaronmt] 2012-09-25 11:09:26 PDT
Test-page: http://admin.brightcove.com/html5support/index.html
Player test-page: http://support.brightcove.com/en/docs/video-test-html-5

Brightcove serves video to thousands of websites; we should be compatible.

--
Nightly (09/25)
Samsung Galaxy Nexus (Android 4.1.1)
Comment 1 Lawrence Mandel [:lmandel] (use needinfo) 2012-10-05 09:02:16 PDT
Here's the Brightcove test for H.264 support from the test page that Aaronmt linked to above:

var v = document.createElement("video");
...
var h264Support = !!(v.canPlayType && v.canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));

Does Fennec support the canPlayType function on the video element?
Does Fennec support the video/mp4 type?
Does Fennec support the avc1.42E01E or mp4a.40.2 codecs?
Comment 2 Lawrence Mandel [:lmandel] (use needinfo) 2012-10-05 09:06:47 PDT
I just tried the player test page on Fennec Nightly (2012-10-05). The video does play, albeit very choppy.
Comment 3 Aaron Train [:aaronmt] 2012-10-05 09:08:31 PDT
(In reply to Lawrence Mandel [:lmandel] from comment #2)
> I just tried the player test page on Fennec Nightly (2012-10-05). The video
> does play, albeit very choppy.

Uninstall Flash if it is installed.
Comment 4 Chris Peterson [:cpeterson] 2012-10-05 09:42:30 PDT
(In reply to Lawrence Mandel [:lmandel] from comment #1)
> Here's the Brightcove test for H.264 support from the test page that Aaronmt
> linked to above:
> 
> var v = document.createElement("video");
> ...
> var h264Support = !!(v.canPlayType && v.canPlayType('video/mp4;
> codecs="avc1.42E01E, mp4a.40.2"').replace(/no/, ''));
> 
> Does Fennec support the canPlayType function on the video element?
> Does Fennec support the video/mp4 type?
> Does Fennec support the avc1.42E01E or mp4a.40.2 codecs?

That's very interesting. I just tested Fennec's video.canPlayType() and got the following results:

 * canPlayType('video/webm') -> "probably"
 * canPlayType('video/mp4') -> "maybe"
 * canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"') -> "" (i.e. NO)
Comment 5 Chris Peterson [:cpeterson] 2012-10-10 22:49:17 PDT
Created attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

Part 1: Add H.264 codecs that are commonly supported on Android.

Henri, I CC'd you because you had some insightful comments on GStreamer bug 760140.

This patch fixes Android's codec list to use the proper codec names from the HTML5 spec [1]. To be conservative, I've only included the Baseline and Main profiles and levels commonly recommended for mobile video.

Google's recommends Baseline Profile [2] when encoding low, high, or even HD quality videos for Android devices.

Zencoder recommends Main Profile [3] for high quality videos. The iPad 1 and iPhone 4 support Main Profile Level 3.1. The iPad 2 and iPhone 4S support High Profile Level 4.1. The iPhone 3GS and earlier only support Baseline Profile Level 3.0. 

[1] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type
[2] https://developer.android.com/guide/appendix/media-formats.html#recommendations
[3] http://blog.zencoder.com/2012/01/24/encoding-settings-for-perfect-ipadiphone-video
Comment 6 Chris Peterson [:cpeterson] 2012-10-10 22:50:22 PDT
Created attachment 670261 [details] [diff] [review]
part-2-add-codec-comments.patch

Part 2: Add comments to GStreamer and Gonk's list of H.264 codecs.
Comment 7 Chris Peterson [:cpeterson] 2012-10-10 23:02:39 PDT
btw:

1. I suspect "avc1.4D001F" is misspelled and should be "avc1.4D401F".

2. Should canPlayType()'s codec name check be case-insensitive? Our current implementation is case-sensitive [1].

The HTML5 spec does not specify case-sensitivity, but its examples go out of their way to specify mixed-case names like "avc1.42E01E" [2]. On the other hand, Apple's streaming media FAQ [3] lists lowercase codec names like "avc1.42e01e".

Chrome and Safari are case-sensitive when comparing the codec name prefix like "avc1.", but the case of the codec name suffix is moot because they claim to "probably" support any "avc1.*" wildcard name.

[1] https://hg.mozilla.org/mozilla-central/annotate/99898ec9976a/content/html/content/src/nsHTMLMediaElement.cpp#l2363
[2] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type
[3] https://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html#//apple_ref/doc/uid/TP40008332-CH103-SW1
Comment 8 cajbir (:cajbir) 2012-10-11 00:13:25 PDT
Let's be case insensitive. I can imagine that there will be a lot of mis-casings in web code. These codec names are so similar there will probably be a ton of mistypings too unfortunately.
Comment 9 Henri Sivonen (:hsivonen) 2012-10-11 00:50:41 PDT
Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

(In reply to Chris Peterson (:cpeterson) from comment #7)
> 2. Should canPlayType()'s codec name check be case-insensitive? Our current
> implementation is case-sensitive [1].

The RFC says the ISO BMF namespace is case-sensitive: http://tools.ietf.org/html/rfc6381#section-3.3

The canPlayType() spec doesn’t say anything to override the RFC: http://www.whatwg.org/specs/web-apps/current-work/#dom-navigator-canplaytype

That said, I wouldn’t oppose to making it ASCII case-insensitive if you file a WHATWG spec bug to request ASCII case-insensitivity and citing the outright wildcarding in WebKit.

> On the other
> hand, Apple's streaming media FAQ [3] lists lowercase codec names like
> "avc1.42e01e".
>
> Chrome and Safari are case-sensitive when comparing the codec name prefix
> like "avc1.", but the case of the codec name suffix is moot because they
> claim to "probably" support any "avc1.*" wildcard name.

Ouch. Those are both good reasons to be ASCII case-insensitive.

I’m marking this feedback+, but I am worried about enumerating a few AVC profile/level combinations when the competitions just gives up and wildcards them. If we don’t end up wildcarding e.g. because we really don’t support the High profile or something, I think it would be prudent to parse out the stuff after “avc1.” and make sure we support everything in some ranges of values that make sense. I don’t want to suggest slowing down this fix, though, as this is a definite improvent over the current code.

I notice that you are removing mp4v. Good. I hope we have code elsewhere that bans MPEG-4 Visual from being decoded. It would be sad to end up supporting it accidentally on mobile and having to pay royalties for yet another encumbered codec (that’s already obsolete even) for compatibility on desktop.

> [2] http://dev.w3.org/html5/spec/the-source-element.html#attr-source-type

It’s generally safer to read the WHATWG spec. In this case, WHATWG and W3C don’t disagree, but the W3C HTML 5.0 release branch will get less and less precise over time as things that don’t meet the CR exit criteria get removed. (See also http://w3cmemes.tumblr.com/post/31865121758/the-joker-shares-his-approach-on-css2-1-issues )
Comment 10 Chris Peterson [:cpeterson] 2012-10-11 13:25:49 PDT
(In reply to Henri Sivonen (:hsivonen) from comment #9)
> I’m marking this feedback+, but I am worried about enumerating a few AVC
> profile/level combinations when the competitions just gives up and wildcards
> them. If we don’t end up wildcarding e.g. because we really don’t support
> the High profile or something, I think it would be prudent to parse out the
> stuff after “avc1.” and make sure we support everything in some ranges of
> values that make sense. I don’t want to suggest slowing down this fix,
> though, as this is a definite improvent over the current code.

I can file a follow-up bug to investigate codec name sensitivity and/or "avc1.*" wildcard support. That work is beyond the scope of this bug because it would affect other platforms and possibly change the MPAPI for platform decoders.
Comment 11 Chris Peterson [:cpeterson] 2012-10-11 13:40:58 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/e0fff9d29111
https://hg.mozilla.org/integration/mozilla-inbound/rev/7474711c6ebd

I plan to uplift patch 1 to Aurora 18 and Beta 17 after it lands on mozilla-central.
Comment 13 Aaron Train [:aaronmt] 2012-10-12 07:35:40 PDT
Can one confirm if this does not fix the video player in the URL in comment #0, as with these changes I can still not get the video to play. Tapping the play button does nothing in their test player.

I can verify that the tests pass in the the test-page URL on mozilla-inbound though.
Comment 14 Chris Peterson [:cpeterson] 2012-10-12 09:51:51 PDT
Aaron, I think this fix was merged to mozilla-central too late for last night's build.
Comment 15 Aaron Train [:aaronmt] 2012-10-12 09:56:43 PDT
(In reply to Chris Peterson (:cpeterson) from comment #14)
> Aaron, I think this fix was merged to mozilla-central too late for last
> night's build.

This was tested on an inbound build from a couple hours ago.
Comment 16 Chris Peterson [:cpeterson] 2012-10-12 11:06:43 PDT
Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

!!! This bug is a high priority because it is blocking a Mozilla/Wired.com marketing campaign (bug 794100).

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Not a regression
User impact if declined: Brightcove video sites (like wired.com and nytimes.com) won't play HTML5 video
Testing completed (on m-c, etc.): QA verified on m-i
Risk to taking this patch (and alternatives if risky): Surprising low risk. This patch does not change any video decoding code; it just changes the list of supported video formats returned from the video.canPlayType() JS API.
String or UUID changes made by this patch: N/A
Comment 17 Alex Keybl [:akeybl] 2012-10-12 13:28:56 PDT
Comment on attachment 670258 [details] [diff] [review]
part-1-add-android-codecs.patch

[Triage Comment]
We can land now, but this won't be built for beta until Tuesday and released later in the week.
Comment 18 Chris Peterson [:cpeterson] 2012-10-12 13:49:08 PDT
https://hg.mozilla.org/releases/mozilla-aurora/rev/8ee98cea0f22
Comment 19 Chris Peterson [:cpeterson] 2012-10-12 13:49:44 PDT
https://hg.mozilla.org/releases/mozilla-beta/rev/31675d03cc9b
Comment 20 Chris Peterson [:cpeterson] 2012-10-16 16:21:14 PDT
   doublec, can you please take a look at this bug? My change in bug 794171 fixed Brightcove's compatibility test page, but did not fix Brightcove's video player test or Wired.com's videos.

 * Compatibility test: http://admin.brightcove.com/html5support/index.html
 * Player test: http://support.brightcove.com/en/docs/video-test-html-5
Comment 21 Chris Pearce (:cpearce) 2012-10-22 16:21:56 PDT
It seems to me that the Player Test page is at fault here. I get Flash served to me with Chrome (which is the poster child for H.264 support) on Linux and Win7 when visiting the Player Test page.

I also tried with the GStreamer backend on Linux desktop Firefox, and it tells me that the page is using canPlayType() to check whether 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"' is supported ("probably" is). The "Player Support Test" page is checking the same string, and on my android phone the "Player Support Test" page is claiming that H.264 is supported, so I assume we're returning "probably" for 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"' there too.

I think this is a bug in BrightCove's code, rather than ours.
Comment 22 Chris Pearce (:cpearce) 2012-10-22 23:52:06 PDT
(In reply to Chris Pearce (:cpearce) from comment #21)
> I think this is a bug in BrightCove's code, rather than ours.

Belay that. I hooked up our media logging to logcat on Android and I can see that a H.264 decoder is being instantiated and is playing. Will investigate further tomorrow...
Comment 23 Chris Pearce (:cpearce) 2012-10-30 14:15:50 PDT
I'm pretty sure Fennec's continued inability to play H.264 on Brightcove is due to Brightcove's player sniffing the user agent and not handling Fennec. See the results of my investigation in bug 794100 comment 17.

I'm going to resolve this fixed again, and we'll continue tracking the evangelism efforts to get Brightcove to fix their player to on their side in Bug 794100.
Comment 24 Ron Piovesan 2013-01-24 08:07:14 PST
Hi team,

This issue with Brightcove is holding up one of our apps partners. has it been resolved yet?

Thanks
Ron
Comment 25 Chris Peterson [:cpeterson] 2013-01-24 11:20:22 PST
Ron, does the partner have a test case or web page that reproduces their problem? What version(s) of Firefox are they testing?
Comment 26 Ron Piovesan 2013-01-24 14:51:37 PST
It is the Wired app in the Marketplace. We can't properly play the video https://marketplace.firefox.com/app/wiredcom/?src=mkt-search
Comment 27 Kevin Brosnan [:kbrosnan] 2013-01-24 16:54:30 PST
Looks like http://www.wired.com/underwire/2013/01/the-monitor-tries-to-warm-up-your-winter/ is an example.

Note You need to log in before you can comment on or make changes to this bug.