767849 - Choose more intelligently which video element uses the hardware decoder

Status: NEW

(Core :: Audio/Video: Playback, defect, P3)

Description

[cajbir (:cajbir)]

Bug 766395 enables hardware video decoding on otoro devices running B2G. This only works for one video decoder instance.

Steps to reproduce:

1) Load http://cd.pn/b2 in a browser window. Press play.
2) Open another browser window on http://cd.pn/b. Press play.

Expected Results:

Video in step 2 uses hardware decoder.

Actual Results:

Video in step 2 uses software decoder with following error in logcat:

E/OMXCodec(  848): Attempting to allocate OMX node 'OMX.qcom.video.decoder.avc'
E/QCvdec  (  848): Video decoder instance already exists.

Comment 1

[cajbir (:cajbir)]

This is also noticeable in the B2G video app. The app instantiates a decoder to get the thumbnail for the list of videos. When you then try to play the video it uses the software decoder as the hardware decoder has been used for the thumbnail.

Comment 2

[cajbir (:cajbir)]

A workaround for the Gaia video player issue mentioned in comment 1 was landed in Gaia:

https://github.com/mozilla-b2g/gaia/commit/18f404aeb2569ed6cb5035b76bf816054da94852

That commit discards the video element used to capture the thumbnail so that the hardware decoder is freed up for video playback.

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Is the bug here that
 - the hardware only supports one decoder instance at a time.  On pages with multiple videos, we sometimes choose the "wrong" video for HW decoding per comment 3

 - or the hardware supports multiple decoder instances, but we're not using them?

Comment 4

[cajbir (:cajbir)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #3)
> Is the bug here that
>  - the hardware only supports one decoder instance at a time.  On pages with
> multiple videos, we sometimes choose the "wrong" video for HW decoding per
> comment 3
> 
>  - or the hardware supports multiple decoder instances, but we're not using
> them?


The bug is that only one instance of the hardware decoder can be created and that instance can only decode one video. So it's the first item in your list.

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

The heuristics here can get really tricky, but from above maybe some simple things we can do are
 - videos that are never play()d always get software decoding (does this handle the poster case?)
 - videos that are below a size threshold always get software decoding

If an app tries to play two 1080p videos simultaneously, that's just going to suck, not much we can do.

Comment 6

[cajbir (:cajbir)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #5)
> If an app tries to play two 1080p videos simultaneously, that's just going
> to suck, not much we can do.

It affects multiple apps too. If I play a video in the browser, pause it, switch to another app, that app can't use the hardware decoder as the browser as it - assuming the browser hasn't been closed and is in the background.

Comment 7

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Yes, we should prefer foreground apps.

This may be somewhat of a silly question, but how hard is it to switch a video element from one decoder to another in-stream?

Comment 8

[cajbir (:cajbir)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #7)
> This may be somewhat of a silly question, but how hard is it to switch a
> video element from one decoder to another in-stream?

Not possible at all at the moment. It may be possible to refactor the OMX decoder so that it can switch from hardware to software - Edwin might be able to advise here as he's more familiar with that decoders implementation.

Comment 9

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Good idea.

Comment 10

[Andreas Gal :gal]

Renom if you think we can't ship a v1 without this.

Comment 11

[Jason Smith [:jsmith]]

Per IRC conversations with a few other folks, I think the best course of action if there is disagreement on whether this blocks or not is to do the following:

- Move the blocking-basecamp flag to ? for re-evaluation
- Indicate a rationale for why you disagree

Comment 12

[Jason Smith [:jsmith]]

Chris - We need input on how bad this is. Should we block on this?

Comment 13

[cajbir (:cajbir)]

I don't think we should block on this. The workaround for the video app in comment 2 makes sure that the video's played use the hardware decoder.

Comment 14

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Mass change P2 -> P3