Closed Bug 1156586 Opened 5 years ago Closed 4 years ago

[FFOS7715 v2.1][AudioChannel]we can not hear music when play a tone and then go back to music page

Categories

(Firefox OS Graveyard :: AudioChannel, defect, blocker)

x86_64
Linux
defect
Not set
blocker

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jingmei.zhang, Assigned: alwu)

References

(Depends on 1 open bug)

Details

(Whiteboard: sprd427808)

Attachments

(1 file)

STR:

1>plug in a headset.
2>play music
3>go to settings->sound->Manage Tones,play a tone
4>press home button to go back home screen
5>open music,we can see the progress bar is working ,but we can not hear song from headset.

Issue could be reproduce in flame
Assignee: nobody → alwu
Hi Alastor,

As I have checked,when we go back to the home screen,music is resumed normally in AudioChannel:

play ringtone:
I/* JINGMEI * ==>( 1359): AudioChannelAgent::NotifyAudioChannelStateChanged state 0

go back to homescreen:
I/* JINGMEI * ==>(  822): AudioChannelService::UnregisterType aChannel 5
I/* JINGMEI * ==>(  822): AudioChannelService::UnregisterTypeInternal aChannel 5
I/* JINGMEI * ==>(  822): AudioChannelService::GetStateInternal aChannel 1
I/* JINGMEI * ==>(  822): AudioChannelService::CheckTelephonyPolicy AudioChannel 1
I/* JINGMEI * ==>(  939): AudioChannelAgent::NotifyAudioChannelStateChanged state 0

But we can not hear sound,can you help to check?

Flame could also have such a issue.
Assignee: alwu → nobody
Flags: needinfo?(alwu)
Ok, I will check it.
Assignee: nobody → alwu
Flags: needinfo?(alwu)
This issue is the same as the Bug1151499. The root cause is that the audio context doesn't release the audio stream of Android audio back-end.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1151499
I wouldn't use the way which is to stop the audio backend stream to solve the bug1151499, so these bugs can't be regard as the same.

Because of we don't have AudioContext suspend() in v2.1, we need to find another way to fix this issue.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Correct the status.
Status: REOPENED → NEW
You could try to release all references to the AudioContext while the Settings app is in the background, so it is collected.
Hi Alastor,

Issue only occur when headset is plug in.

Does headset have a influence for the release of the audio stream of Android audio back-end?
Hi, JingMei,
In my observation, only when the headset is plugged, the AudioPolicyManager will mute all streams, except the STRATEGY SONIFICATION streams. I assume that the Android want to emphasize this type audio in order to ensure the user can really hear that important sound. 

The behavior of muting other streams is the root cause of this issue. I think we can't modify this behavior, so I want to find the way to close the audio backend stream.
Hi, Jim,
Sorry to bother you, 
According to the comment6, this issue might be solved by Gaia. 

The ringtone app used the ringtone AudioContext to play the sound. If the ringtone was playback when we connected the headset, the ringtone audio backend stream would auto mute other streams until the ringtone ended. This behavior is designed by Android, so the Gecko can't modify that.

Therefore, we need to release this backend stream when the ringtone app is on the backgroud. 
Could the ringtone app release its AudioContext when it is on the background?

Very appreciate :)
Flags: needinfo?(squibblyflabbetydoo)
(In reply to Alastor Wu [:alwu] from comment #9)
> Therefore, we need to release this backend stream when the ringtone app is
> on the backgroud. 
> Could the ringtone app release its AudioContext when it is on the background?

That's what the ringtones app currently does: https://github.com/mozilla-b2g/gaia/blob/master/apps/ringtones/js/tone_player.js#L150-L155
Flags: needinfo?(squibblyflabbetydoo)
Note that there's still bug 1120612, and apparently the AudioContext changed so that it doesn't do what we want anyway (I can't find the bug number for this issue, though).
According to [1], we can know the AudioContext can't be release even the app is on the background. It's the platform issue, but the fix of that would be risky [2]. Therefore, we should find a method to solve it in the Gaia.

If we create a new AudioElement when we want to play instead of using a singleton AudioElement, that can help to release the AudioContext. However, the audio backend stream still can't be release immediately.

I will try to find another way to free it immediately.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1152439#c38
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1152439#c39
Hi Alastor,

Issue will not exist when we press back button on the ManageTone Page,So could we reference Back button's way to close ringtone channel?
Flags: needinfo?(alwu)
Hi, JingMei,
I also observed that situation, but now I didn't fully know what happened in this step. I don't know why the audio backend stream can be release so fast.
I am still trying to find more details about this.
Flags: needinfo?(alwu)
Whiteboard: sprd427808
If we release the audio element when the ringtone app is on the background, the audio backend stream still needs lots time to be stop.

> Set app into background
> 05-05 19:45:14.963  1955  1955 I Gecko   : DD | HTMLMediaElement::Pause
> 05-05 19:45:14.973  1955  1955 I GeckoDump: DD | @@@ Gaia, set _player null
> 05-05 19:45:14.973  1955  1955 I GeckoDump: DD | @@@ Gaia, disconnect
> 
> Release the audio backend stream 
> 05-05 19:45:36.823  1955  1955 I Gecko   : DD | ~HTMLMediaElement
> 05-05 19:45:36.893  1955  1955 I Gecko   : DD | MediaStreamGraphShutDownRunnable
> 05-05 19:45:36.893  1955  2020 I Gecko   : DD | AudioCallbackDriver, stop
> 05-05 19:45:36.893  1955  2020 E Cubebc  : DD | cubeb_stream_stop
> 05-05 19:45:36.893  1955  2020 E Cubebc  : DD | cubeb_stream_destroy
> 05-05 19:45:36.903  1955  2020 E AudioTrack: DD | AudioTrack, Stop

---

About the way mentioned in comment13, I guess that is directly clear the whole memory address when we close the app.
Hi, Robert,
Sorry to bother you,
About you mentioned in [1], "it's quite risky to do modification in Gecko." 
What is the potential risk of the modification?

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1152439#c39

---

Here is my problem,
This issue is caused by the occupancy of the audio backend stream. (You can regard it as the output stream of the AudioConext) If I want to free it, I need to stop the AudioContext first. However, the HTMLMediaElement::mOutputStreams and MediaDecoder::mOutputStreams hold the reference of the output stream of the AudioContext so that we can't release the AudioContext.

The only way to release the AudioContext is also to release the AudioElement. It means that AudioContext can be GC because no any object references it. But, even I tried to free the AudioElement when the app was in the background, the AudioContext still can't be release immediately. (See log in comment15, and I forgot to print the destroy time of the AudioContext, it is close to the destroy time of AudioElement)

Therefore, I would like to know whether we can clear the reference (mOutputStreams) when we pause the AudioElement, and add it back when we resume the AudioElement?

If we can do so, the AudioConext should be release immediately when we stop the AudioElement.

How do you think?
Very appreciate for your help!
Flags: needinfo?(roc)
(In reply to Alastor Wu [:alwu] from comment #16)
> Therefore, I would like to know whether we can clear the reference
> (mOutputStreams) when we pause the AudioElement, and add it back when we
> resume the AudioElement?

No, I don't think we can do that.

Consider the case where an AudioContext has a MediaElementAudioSourceNode connected to its destination, all JS references to the AudioContext and MediaElementAudioSourceNode are dropped. The only thing keep the AudioContext alive is the reference chain HTMLAudioElement -> DOMMediaStream ->(via DOMMediaStream.mConsumersToKeepAlive) MediaElementAudioSourceNode -> AudioContext. If pausing the audio element breaks the first relationship, the AudioContext will die and after resuming the element, audio will not play as it should.

To fix this properly we need to handle the case where all JS references to a playing AudioNode are dropped while it's not connected to the AudioContext destination. In that case (plus some other conditions hold, depending on the AudioNode type) the AudioNode can be destroyed. When a MediaStreamAudioSourceNode is destroyed, it should remove its DOMMediaStream from the HTMLAudioNode's output list.

Detecting when we can destroy a playing AudioNode because all JS references have been dropped and it's not connected to the destination is bug 897796, which is a hard bug but would be great to fix.
Depends on: 897796
Flags: needinfo?(roc)
Hi Alastor,

Could we have any workaround solution for this issue?

This is a PTR Block issue in our part,so I am afraid we should have a quick solution for the current.

Sorry to trouble you ;(
Flags: needinfo?(alwu)
Severity: major → blocker
Sorry, JingMei,
Now I only figure out one way to solve this issue which is to remove the AudioContext in the ringtone app.
However, it would cause the regression error. [1] (conflicts with the feature of the ringtion app) 

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1107534#c18
Flags: needinfo?(alwu)
Hi Alastor,

I am afraid 'remove the AudioContext in the ringtone app' will cause another issue as:

Bug 958470 - [Ringtones][Music] Previewing the ringtones should not be mixed together with the background music.

;(
Hi, JingMei,
Sorry I can't get you a perfect solution, but here is the workaround patch which can solve the issue but results other regression error.
If you think the issue is the highest priority to solve, you can temporary use this patch in your local repo.
Thanks!
Flags: needinfo?(jingmei.zhang)
Hi Alastor, 

work around patch has been merged in sprd.

Thanks!
Flags: needinfo?(jingmei.zhang)
See Also: → 1194643
Close this bug because we wouldn't land the code for v2.1.
Status: NEW → RESOLVED
Closed: 5 years ago4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.