1026312 - [MADAI][Multimedia][Audio] When I operate pause/resume repeatedly, OMXCodec(S/W decoder) is occurred time out.

Status: RESOLVED INVALID

(Firefox OS Graveyard :: Vendcom, defect, P3)

Description

[Jaemin Song]

User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36

Steps to reproduce:

1. Copy *audio file to device.
*audio file specification.
   - decoder(codec): Omx.google.*(S/W audio decoder) ex: amr-nb
   - mime type: audio/3gpp or audio/mpeg
   (Currently OMXCodec is using S/W audio decoder for amr(3gpp))
2. Play the audio file.
3. Press pause/resume repeatedly.
4. Audio does not come out.(Decoder is timed out)


Actual results:

When I operate pause/resume repeatedly using S/W audio decoder, 
OMXCodec is occurred time out.
As a result of analysis, I found difference like below.

[Android] In case of pause/resume operation using S/W audio decoder.
Player does not call OMXCodec::pause.
Player just call stop to audio track/sink.
And when I resume audio, player does not call OMXCodec::start.
player just call start to audio track/sink.

[FireFox OS v1.4] In case of pause/resume operation using S/W audio decoder.
Player call OMXCodec::pause, start.
If resume audio, OMXCodec call drainInputBuffer function.
When I operate pause/resume repeatedly, decoder's inqueue was getting smaller.
And When decoder's inqueue was 0, decoder was timed out.

In my opinion, when player uses S/W audio decoder,
player does not call OMXCodec::pause, start function like Android player.
Please check this issue.
Thanks.

Comment 1

[Preeti Raghunath(:Preeti)]

Moving this to backlog as it doesn't prevent phone from shipping. We're now in a stage where only critical blockers will be taken in the release.

Comment 2

[Sotaro Ikeda [:sotaro]]

Bug 904177 might fix the problem.

I think that one big intrinsic problem around media frame work is the following. Therefore user's activity, app's js code could block media playback work.
- use b2g main thread and app's main thread are used for data delivery.

This problem could also affect to web browsing performance in b2g.

Comment 3

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #2)

> I think that one big intrinsic problem around media frame work is the
> following. Therefore user's activity, app's js code could block media
> playback work.
> - use b2g main thread and app's main thread are used for data delivery.
> 
> This problem could also affect to web browsing performance in b2g.

The above problem is tracked by Bug 1013756.

Comment 4

[Sotaro Ikeda [:sotaro]]

Without analysis, it is not clear Comment 2 and comment 3 could apply to this bug.

Comment 5

[Jaemin Song]

(In reply to Sotaro Ikeda [:sotaro] from comment #2)
> Bug 904177 might fix the problem.
> 
> I think that one big intrinsic problem around media frame work is the
> following. Therefore user's activity, app's js code could block media
> playback work.
> - use b2g main thread and app's main thread are used for data delivery.
> 
> This problem could also affect to web browsing performance in b2g.

Is there another solution for this issue without Bug 904177's solution?
Since Bug 904177's patch will be added in 2.1 version, 
we should find another solution for 14.version.(MADAI)

Comment 6

[Jaemin Song]

(In reply to Jaemin Song from comment #5)
> (In reply to Sotaro Ikeda [:sotaro] from comment #2)
> > Bug 904177 might fix the problem.
> > 
> > I think that one big intrinsic problem around media frame work is the
> > following. Therefore user's activity, app's js code could block media
> > playback work.
> > - use b2g main thread and app's main thread are used for data delivery.
> > 
> > This problem could also affect to web browsing performance in b2g.
> 
> Is there another solution for this issue without Bug 904177's solution?
> Since Bug 904177's patch will be added in 2.1 version, 
> we should find another solution for 14.version.(MADAI)

Modified wrong version.
14.version. -> 1.4version

Comment 7

[Sotaro Ikeda [:sotaro]]

Before thinking about solutions, it seems better to analyze more about the problem.

Comment 8

[Jaemin Song]

Further information of debugging.
[FireFox OS V1.4]
If OMXCodec::pause() is called, OMXCodec::mPaused value is changed to "true".
Sometime, after OMXCodec::pause() command, OMXCodec::drainInputBuffer is called.
In this time, "OMXCodec::drainInputBuffer" function does not call "emptyBuffer".
If OMXCodec is pause state(OMXCodec::mPaused = true), "OMXCodec::drainInputBuffer" returns "false" and exit function.
So, "emptyBuffer" was not called and inqueue was getting smaller.
Please refer "OMXCodec::drainInputBuffer".

[Android]
Since android's player does not call OMXCodec::pause()(OMXCodec::mPaused = false),
this issue is not occurred.

Comment 9

[Wayne Chang [:wchang]]

Thomas, can someone investigate here?

Comment 10

[thomas tsai]

Vincent Liu will follow up after his vacation(~6/29).

Comment 11

[Vincent Liu[:vliu]]

Currently FFOS doesn't support OMXCodec::pause() function.

https://bug919590.bugzilla.mozilla.org/attachment.cgi?id=8340486

For more detail, please see bug 8340486.

Comment 12

[Sotaro Ikeda [:sotaro]]

(In reply to Vincent Liu[:vliu][PTO:6/22~6/27] from comment #11)
> Currently FFOS doesn't support OMXCodec::pause() function.
> 
> https://bug919590.bugzilla.mozilla.org/attachment.cgi?id=8340486
> 
> For more detail, please see bug 8340486.

FFOS support caf OMXCodec::pause(). FFOX does not support aosp OMXCodec::pause().

Comment 13

[Jaemin Song]

(In reply to Sotaro Ikeda [:sotaro] from comment #12)
> (In reply to Vincent Liu[:vliu][PTO:6/22~6/27] from comment #11)
> > Currently FFOS doesn't support OMXCodec::pause() function.
> > 
> > https://bug919590.bugzilla.mozilla.org/attachment.cgi?id=8340486
> > 
> > For more detail, please see bug 8340486.
> 
> FFOS support caf OMXCodec::pause(). FFOX does not support aosp
> OMXCodec::pause().

In case of S/W audio codec(google native codec),
is your meaning that we should apply 8340486 patch to OMXCodec for AOSP OMXCodec::pause()?

Currently, we are using QCOM H/W codec for video, S/W codec(google) for audio.

Comment 14

[Sotaro Ikeda [:sotaro]]

(In reply to Jaemin Song from comment #13)
> 
> In case of S/W audio codec(google native codec),
> is your meaning that we should apply 8340486 patch to OMXCodec for AOSP
> OMXCodec::pause()?
> 
> Currently, we are using QCOM H/W codec for video, S/W codec(google) for
> audio.

Sorry, I do not understand your situation well. If your device is b2g production device and uisng QCOM H/W, your device should use caf OMXCodec.

B2G calls OMXCodec::pause() only for QCOM device to reduce power consumption during pause. The meaning of OMXCodec::pause() is different between caf OMXCodec::pause() and aosp OMXCodec::pause(). If you use aosp OMXCodec, attachment 8340486 [details] [diff] [review] need to be applied. It does not matter, the codec is HW codec or SW codec.

This problem seems like a problem of OMXCodec. Therefore this problem is POVB problem.

Comment 15

[Jason Smith [:jsmith]]

Clearing nom & moving to vendcom because this is a vendor issue.

Comment 16

[Jaemin Song]

> Sorry, I do not understand your situation well. If your device is b2g
> production device and uisng QCOM H/W, your device should use caf OMXCodec.
> 
> B2G calls OMXCodec::pause() only for QCOM device to reduce power consumption
> during pause. The meaning of OMXCodec::pause() is different between caf
> OMXCodec::pause() and aosp OMXCodec::pause(). If you use aosp OMXCodec,
> attachment 8340486 [details] [diff] [review] need to be applied. It does not
> matter, the codec is HW codec or SW codec.
> 
> This problem seems like a problem of OMXCodec. Therefore this problem is
> POVB problem.

My device is using caf OMXCodec::pause() for video codec(QCOM's decoder)
and using AOSP OMXCodec::pause() for audio codec(Google's decoder).

According to decoder type, my device's OMXCodec::pause() code is divided.
In case of H/W codec(QCOM's decoder), 
since using caf OMXCodec::pause(), this issue was not occurred.
So, I will apply 8340486 patch for only audio codec's pause.(AOSP OMXCodec::pause)
When I apply this patch to check issue, issue was not occurred.
Thank you for your help.

Comment 17

[Wayne Chang [:wchang]]

Closing the bug as partner has resolved this, suggested in comment 16 and confirmed with partner.