Closed Bug 759945 Opened 12 years ago Closed 12 years ago

Android ICS/JB Software decoding ("software platform decoders") support for H.264/AAC/MP3 video/audio playback

Categories

(Core :: Audio/Video, defect)

ARM
Android
defect
Not set
normal

Tracking

()

VERIFIED FIXED
mozilla17
blocking-kilimanjaro +
Tracking Status
firefox18 --- verified

People

(Reporter: cajbir, Assigned: cajbir)

References

()

Details

Attachments

(6 files, 4 obsolete files)

Bug 714408 implements H.264/AAC/MP3 playback on B2G using Android libstagefright libraries. This bug is to make that work on consumer android devices.

Steps to reproduce:

1. Visit http://cd.pn/b2 in Firefox on Android
2. Press Play

What happens:

The controls disappear and no playback occurs.

What should happen:

Video should play.
blocking-kilimanjaro: --- → ?
Depends on: 714408, 755364
Blocks: 755364
No longer depends on: 755364
No longer depends on: 761762
No longer depends on: 761786
No longer depends on: 761814
blocking-kilimanjaro: ? → +
Attached patch Software decoding on ICS/JB (obsolete) — Splinter Review
This patch adds support for H.264 decoding using libstagefright software decoders on some ICS and Jellybean Android devices. Tested on:

Galaxy Note running ICS
Nexus S running ICS
Nexus S running Jellybean
HTC One X (International) running ICS

To build you'll need to have the Android OS source code (not just the SDK) for ICS. You'll also need the following shared libraries from an android device for linking:

libstagefright_omx.so
libstagefright.so
libutils.so

Add the following .mozconfig options:

--enable-media-plugins
--enable-omx-plugin
--with-android-source=/path/to/android/source
--with-stagefright-libs=/path/to/android/shared/libraries

The "with-stagefright-libs" should be a path to a directory containing the shared libraries mentioned above. These can be retrieved from a device with "adb pull /system/lib/libraryname.so".

A future iteration of the patch will hopefully remove the need for the binaries and the requirement for the android OS source.

Test videos:

http://cd.pn/b
http://cd.pn/b2

Patch includes a test video in content/media/test for running mochitests on a device.
Assignee: nobody → chris.double
Status: NEW → ASSIGNED
Attached patch Software decoding on ICS/JB (obsolete) — Splinter Review
Removes "--with-android-source" configure option and instead includes necessary Android OS headers directly. It's no longer needed to have the Android OS source around.
Attachment #645651 - Attachment is obsolete: true
We're morphing bug to follow software platform decoder support for Android.
Summary: Support H.264/AAC/MP3 video/audio playback on Android → Software decoding ("software platform decoders") support for H.264/AAC/MP3 video/audio playback on Android
Attached patch Software decoding on ICS/JB (obsolete) — Splinter Review
Removes need for binary files to link against. This patch builds stub versions of the libstagefright shared libraries we need and links against them. These stubs are not installed on the phone so at runtime we end up using the real libstagefright libraries.
Attachment #646020 - Attachment is obsolete: true
Configure and build changes for software decoding of H.264/AAC/MP3 content on Android ICS/JB using libstagefright.
Attachment #649532 - Attachment is obsolete: true
Attachment #650423 - Flags: review?(khuey)
Changes to mobile/android code to get the libomxplugin installed in the correct place and add the preference so that media plugins are disabled by default.
Attachment #650425 - Flags: review?(mark.finkle)
Imports Android OS headers needed for building. Adds code to build stub shared libraries to link against. These shared libraries aren't distributed - they are only used during the build to link against. On the device Fennec will link against the installed ICS/JB android libraries.

Media plugins are disabled via a pref by default as tests fail and playback is not yet performant.

Minor changes to the OmxPlugin code to get playback working.
Attachment #650426 - Flags: review?(cpearce)
Attachment #650425 - Flags: review?(mark.finkle) → review+
Adds an MP4 file for testing in the media mochitests that was missing from part 3.
Attachment #650434 - Flags: review?(cpearce)
Comment on attachment 650426 [details] [diff] [review]
Part 3: Changes needed to OmxPlugin

Review of attachment 650426 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/plugins/nsMediaPluginReader.cpp
@@ +142,5 @@
>      }
>      mVideoSeekTimeUs = -1;
>  
>      if (aKeyframeSkip) {
> +      // Disable keyframe skipping for now as

In Edwin's patches we're using some function he found that seeks the decoders to the next keyframe, maybe we can use that here too?

::: content/media/test/manifest.js
@@ +149,5 @@
>  
>    // Opus data in an ogg container
>    { name:"detodos.opus", type:"audio/ogg; codecs=opus", duration:2.9135 },
>  
> +  { name:"gizmo.mp4", type:"video/mp4", duration:5.0 } ,

nit: Remove space between } and ,

::: media/omx-plugin/OmxPlugin.cpp
@@ +216,1 @@
>      }

Indentation is off here.

@@ +523,5 @@
>    else if (err == INFO_FORMAT_CHANGED) {
>      // If the format changed, update our cached info.
> +    if (!SetVideoFormat())
> +      return false;
> +    else return ReadVideo(aFrame, aSeekTimeUs);

Break between else and return to be consistent please.

@@ +579,2 @@
>  
>    return true;

You can't reach this |return true|, so you don't need it now, right?
Attachment #650426 - Flags: review?(cpearce) → review+
Attachment #650434 - Flags: review?(cpearce) → review+
Building on B2G requires conditionals in the makefiles to stop building the stub shared libraries as the real versions are built as part of B2G already.
Attachment #650443 - Flags: review?(cpearce)
Attachment #650443 - Flags: review?(khuey)
Comment on attachment 650443 [details] [diff] [review]
Part 5: Fix build errors on B2G

Review of attachment 650443 [details] [diff] [review]:
-----------------------------------------------------------------

This would be best reviewed by someone familiar with the code.  The build changes are trivial.
Attachment #650443 - Flags: review?(khuey)
Attachment #650443 - Flags: review?(cpearce) → review+
media.plugins.enabled was being ignored. This patch fixes that.
Attachment #650786 - Flags: review?(cpearce)
Address review comments, carrying r+ forward. For this comment I'll address in a followup bug to fix playback issues that come up during testing:

>In Edwin's patches we're using some function he
> found that seeks the decoders to the next keyframe,
> maybe we can use that here too?
Attachment #650426 - Attachment is obsolete: true
Attachment #650787 - Flags: review+
Attachment #650786 - Flags: review?(cpearce) → review+
Using mozilla-inbound (17) (3bead0b0dd75), with the preference enabled there is still no available support (e.g, the test-case in comment #0) will not play (Video format or MIME type is not supported).
(In reply to Aaron Train [:aaronmt] from comment #15)
> Using mozilla-inbound (17) (3bead0b0dd75), with the preference enabled there
> is still no available support (e.g, the test-case in comment #0) will not
> play (Video format or MIME type is not supported).

Do any of the following URL's play for you:

http://cd.pn/a
http://cd.pn/b
http://cd.pn/b2

What device are you using and what Android version does it have? Have you enabled the pref media.plugins.enabled?
(In reply to Chris Double (:doublec) from comment #16)
> http://cd.pn/a

E/GeckoConsole( 5500): [JavaScript Warning: "HTTP "Content-Type" of "audio/mpeg" is not supported. Load of media resource http://cd.pn/a/joy.mp3 failed." {file: "http://cd.pn/a/" line: 0}]

> http://cd.pn/b

E/GeckoConsole( 5871): [JavaScript Warning: "HTTP "Content-Type" of "video/mp4" is not supported. Load of media resource http://cd.pn/b/bounty.mp4 failed." {file: "http://cd.pn/b/" line: 0}]

> http://cd.pn/b2

E/GeckoConsole( 5871): [JavaScript Warning: "HTTP "Content-Type" of "video/mp4" is not supported. Load of media resource http://cd.pn/b2/story.mp4 failed." {file: "http://cd.pn/b2/" line: 0}]
 
> What device are you using and what Android version does it have?

Samsung Galaxy Nexus (Android 4.1.1).

>Have you enabled the pref media.plugins.enabled?

Yes.
Did you build with "--enable-media-plugins" and "--enable-omx-plugin"? They're off by default.
(In reply to Chris Double (:doublec) from comment #18)
> "--enable-omx-plugin"?
Missed that one.
They should be the default on Android since it's pref'd off. I'll submit a bug/patch.
Blocks: 781831
Bug 781831 for enabling the configure options on android to build by default.
Comment 1 lists 4 devices on H.264 decoding is supported with this patch:

Galaxy Note running ICS
Nexus S running ICS
Nexus S running Jellybean
HTC One X (International) running ICS

1. Is there a complete list of devices for which this patch adds H.264 decoding? 
2. Does this patch require ICS or Jellybean? Does it work with previous versions of Android?
(In reply to Lawrence Mandel [:lmandel] from comment #23)
> 1. Is there a complete list of devices for which this patch adds H.264
> decoding? 

There isn't a complete list, only those I listed previously which I've tested on. It should work on any ICS or JB device though.

> 2. Does this patch require ICS or Jellybean? 

Either works.

> Does it work with previous versions of Android? 

I have not yet tested on previous versions but will do soon.
Blocks: 782508
Depends on: 783927
Depends on: 784329
Depends on: 786071
Depends on: 786112
Depends on: 786117
Depends on: 784019
Depends on: 786248
I'm changing the title to reflect that this is only for ICS/JB.  For tracking purposes I'm opening a new bug for GB support which will be added soon.
Summary: Software decoding ("software platform decoders") support for H.264/AAC/MP3 video/audio playback on Android → Android ICS/JB Software decoding ("software platform decoders") support for H.264/AAC/MP3 video/audio playback
Blocks: 787226
Moving dependencies on content-specific bugs against libstagefright to Bug 787226, which will be the meta bug tracking such issues.
No longer depends on: 784019, 784329, 786071, 786112, 786117, 786248
Changes were applied on the latest Nightly. However, the video content is not rendered, but this is a known issue already filled. Instead the audio is played correctly.
Closing bug as verified fixed.

--
Firefox 18.0a1 (2012-09-19)
Device: Galaxy Note
OS: Android 4.0.4
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.