766395 - Video decoding using libstagefright on otoro device does not use hardware decoder on B2G

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P1)

Description

[cajbir (:cajbir)]

When playing H.264 videos on an otoro device with B2G the software libstagefright decoder is used resulting in poor performance and basic profile support only.

Comment 1

[cajbir (:cajbir)]

Device is missing the following shared libraries to get hardware decoding working:

libOmxH264Dec.so
libOmxAacDec.so
libdivxdrmdecrypt.so
libmm-adspsvc.so

Comment 2

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Attachment: Patch 1 rev 1 - Add support for Qualcomm colour format

h.264 now working on otoro. Very slow, however, and some videos don't work. Will have to investigate further.

Comment 3

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Attachment: Patch 1 rev 2 - Add support for Qualcomm colour format

Had U and V channels backwards. Carry over r=doublec.

Comment 5

[cajbir (:cajbir)]

Can we land this so that when the issue in comment 1 is addressed hardware acceleration will work?

Comment 6

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Pull request for comment 1 at https://github.com/mozilla-b2g/android-device-otoro/pull/5

Comment 7

[[e-mail] [:onecyrenus]]

Just an update from my perspective. there are some other shared libraries that don't seem to be present. 

E/ExtendedExtractor(  453): Failed to open MM_PARSER_LIB, dlerror = Cannot load library: load_library[1091]: Library 'libmmparser.so' not found 
E/ExtendedExtractor(  453): Failed to open MM_PARSER_LITE_LIB, dlerror = Cannot load library: load_library[1091]: Library 'libmmparser_lite.so' not found 

Also the video I am trying to open
people.mozilla.com/~dclarke/a.html does not play.  

            //if(!strcmp(mComponentName, "OMX.google.h264.decoder")
            //    && (profile != kAVCProfileBaseline)) {
            //    LOGE("%s does not support profiles > kAVCProfileBaseline", mComponentName);
                // The profile is unsupported by the decoder
            //    return ERROR_UNSUPPORTED;
           // }
This seems to be where the decision is being made.  Uncommented, I get audio playback but the lack of the ".so" files listed above throw an error. 

Without uncommenting I get no audio / video . 

http://source-android.frandroid.com/frameworks/base/media/libstagefright/OMXCodec.cpp
Looks like there have been changes to libstagefright, that we don't have... 
as these lines don't show up.

Comment 8

[cajbir (:cajbir)]

OMX.google.h264.decoder is the software decoder. We want the *.qcom.* decoder which those .so files give us. What video doesn't play for you with those .so files included in /system/lib?

Comment 9

[[e-mail] [:onecyrenus]]

Which .so files ? are they checked in ?

Comment 10

[cajbir (:cajbir)]

See comment 1 or wait for comment 6 to be actioned. Even if comment 6 is done though, if you've already installed b2g you've lost those .so files and need to get them from the original android OS that was on the device.

Comment 11

[[e-mail] [:onecyrenus]]

Chris all otoro devices that went out to QA came pre-flashed to B2G. :( which means those libs are not on my machine.

Comment 12

[[e-mail] [:onecyrenus]]

Attachment: logcat output / hw decoding crash

Attached it the log i received when attempting to play content

people.mozilla.com/~dclarke/a.html || bit.ly/QencVk

Comment 13

[[e-mail] [:onecyrenus]]

Obtained original files, applied the pull from comment 6, and tested. 

@Edwin, what is the reference video that you are using. ?

Comment 14

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

This patch is now obsolete. We should have something proper going in the next couple of days. (sorry!)

Reference video I'm using is https://hg.mozilla.org/users/eflores_mozilla.com/video/raw-file/2aaf130aae3f/mp4/big_buck_bunny.mp4

Comment 15

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Actually, that's wrong. We'll have something going on maguro, but otoro will require a bit more work yet due to our GB/ICS munge.

Comment 16

[[e-mail] [:onecyrenus]]

So should we rename this bug for maguro, and create a new one for otoro ?

Comment 17

[cajbir (:cajbir)]

This should stay otoro because we don't have hardware decoding on otoro and that's what it's addressing.

Comment 18

[[e-mail] [:onecyrenus]]

Ok.? Is the above patch obsolete for otoro ? There should be a separate bug for maguro support then.

Comment 19

[[e-mail] [:onecyrenus]]

Edwin, can you give an eta on otoro, and is the b2g demo going to be on a maguro  or an otoro device?

Comment 20

[Joe Drew (not getting mail)]

We don't expect that this is device-specific, but rather chipset-specific, right? If it's otoro-only, then we might not need to block on this.

Comment 21

[[e-mail] [:onecyrenus]]

This is chipset specific, but for all intensive purposes it is device specific.
Otoro is the device we are shipping on.

Comment 22

[cajbir (:cajbir)]

This particular bug was for the otoro device specific problem of the hardware decoder shared libraries not being installed when doing an otoro config. It also requires the patch which is attached and this allows hardware decoding to work. But it is very very slow - slower than software. The fix for the slowness is being done in bug 759506.

The issue for the shared libraries not being copied was being addressed by Edwin in a pull request to the mozilla-b2g github repository. Once that's done this patch can be landed and the bug closed. 

Edwin, how'd that pull request go?

Comment 23

[Sotaro Ikeda [:sotaro]]

It is FYI. Platform specific hw codecs is provided by libstagefrighthw.so

it is added as OMX plugin in OMXMaster::addVendorPlugin() 
http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/omx/OMXMaster.cpp#46

Comment 24

[Andreas Gal :gal]

How are we doing here?

Comment 25

[cajbir (:cajbir)]

It's blocked on 759506 which currently has issues with hardware decoding.

Comment 26

[cajbir (:cajbir)]

Attachment: Enable use of hardware decoders

Turns on hardware decoding support on B2G. If the otoro device has the libOmx*.so files for hardware codecs mentioned in comment 1 this enables decoding videos using the hardware decoder.

Comment 27

[cajbir (:cajbir)]

Pull request to get the libraries copied over (updated version of comment 6):

https://github.com/mozilla-b2g/android-device-otoro/pull/12

Comment 28

[cajbir (:cajbir)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ce9f5c71a4aa

Comment 29

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/ce9f5c71a4aa