1179077 - [B2G] Sound should be paused when headphone is disconnect

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: AudioChannel, defect)

Description

[Alastor Wu [:alwu]]

In native app (music/video), the sound would be paused when the headphone is disconnected.
However, it would not take effect on the browser app, the audio would still be playback.

STR.
1. Plug the wired headphone.
2. Open the browser app.
3. Listen the music on browser app.
4. Disconnect the headphone.

Expected:
5. Paused sound

Actual:
5. Didn't pause sound

Comment 2

[Alastor Wu [:alwu]]

Hi, JR,
After discuss this issue with Dominic, I found that is not regression error.
Both FxOS and Android wouldn't pause the audio/video on the web browser.

---

In FxOS, the Gecko doesn't automatically pause the media when the headset disconnect. It's implemented by app itself.

The Gecko would send device available events to the system app, and then the system app checks these events to decide whether we need to pause the media.
If the system app decide to pause media, it would send the specific event. The app need to handle this event and pause its video/audio tags.

However, in this case, the browser app may have lots of tabs, it's impossible every tab(website) would handle this event.
Therefore, we can't use this method to pause the media of the browser app.

---

If we still need this feature, we need to move the pause logic from Gaia to Gecko.
Once we implement that, all apps would not need to handle the event, the media would be paused absolutely when the headset is disconnect.

It that your want? Have this feature already verified by UX?

If all the answers are yes, I think we can start to implement this :) 
Thanks!

Comment 3

[JR Conlin [:jrconlin,:jconlin]]

Would it be possible to provide an option to mute the media sound channel on headphone disconnect? That might suffice to solve the issue, since, say, a web page could continue to play media but would allow users the ability to not disturb others. 

If you see a way that makes this a more compelling user experience (or at least one that doesn't annoy users needlessly), then I'm all for it.

Thanks!

Comment 4

[Alastor Wu [:alwu]]

If we want to pause/mute the media sound and change UI icons at the same time, it need to directly control the MediaElement by those website. 

In native apps, Gaia would do it when receives the specific events. However, in browser app, we don't know whether these tabs handle this kind of event (the answer would probability be no), that is why we can't pause/mute these media content.

If we want to do that, we should directly control the whole iframe, to pause/mute all media contents of the specific iframe. 

---

There are two proposals,

(1) Use the mozBrowser API in system app [work in Gaia]
The mozBrowser API can control the specific iframe, we can use setMute/setVolume to control the media contents of the iframe. We can integrate this logic to system app so that system app can control all audio status.

However, I don't know whether it's good idea to integrate the audio competing logic with that, because this API is originally designed for handling audio interrupt happen audio competing.

(2) Use mozAudioChannelManager [work in Gecko]
We can implement new interface in mozAudioChannelManager API. The new interface may be like that 

> void Paused(Muted)WhenDeviceDisconnect(bool aEnable = true) 
Default is not doing anything when the device disconnect.

Apps can use this function to define whether its media content need to be paused/muted when the device is disconnect. 

# Notice
Both all two proposals are needed to change the UI when we pause/mute the audio at the same time. 
It need to send the "volume-change" event in |MediaElement::WindowVolumeChanged()|.

---

Hi, Baku,
Could you give me some suggestions?
I'll discuss this issue with Dominic/Evan again on next Monday.

Thanks!

Comment 5

[Andrea Marchesini [:baku]]

> If we want to pause/mute the media sound and change UI icons at the same
> time, it need to directly control the MediaElement by those website. 

Which UI is this?

> (2) Use mozAudioChannelManager [work in Gecko]
> We can implement new interface in mozAudioChannelManager API. The new
> interface may be like that 

I like this option. Because I don't know if this new logic should work for any app.

> Both all two proposals are needed to change the UI when we pause/mute the
> audio at the same time. 
> It need to send the "volume-change" event in
> |MediaElement::WindowVolumeChanged()|.

It's not just about MediaElements, but also webaudio and so on.
Can we use mozinterruptbegin/end? We already have these 2 events per mediaelement/webaudio.

Comment 6

[Alastor Wu [:alwu]]

Hi, all,
Sorry about previous comment, I didn't think clearly before.
I think that we CAN NOT use the AudioChannelService::SetAudioChannelMuted to achieve this feature.

---

I forgot an important point, the volume controlling functions from Bug1113086 are designed for audio INTERRUPT.
It means that the setAudioChannelMute/Volume are not actually corresponding to the status of the media element (or webaudio).

For example, 
The SetAudioChannelMuted implement the pause functionality, but it wouldn't send the pause event. Therefore, the playbar icon wouldn't be changed to pause icon.

---

The important thing is not about implementation in Gecko or Gaia. It is that we need the new API which can directly control media element status.

Therefore, here is my new proposal.
We can create the new API in mozBrowser level, and the system can use this API to control the media content (or webaudio) of the specific iframe. 

* Control flow.
[Gaia]   System app → control API 
                           ↓                  
[Gecko]           AudioChannelService → AudioChannelAgent → HTMLMediaElement(or webaudio) pause()/resume()


The code maybe like this,
1. Create control API in nsBrowserElement 
> void PlayMediaContent() and void PauseMediaContent()   /*can be a better name*/
> or   
> void SetMediaContentStatus(bool aPlayable)

2. Create the new callback function in nsIAudioChannelAgentCallback, this function would active the pause/resume of the media element(and webaudio).

> APIControlFuction() {
>   nsRefPtr<AudioChannelService> service = AudioChannelService::GetOrCreate();
>   service->PauseWindowAudio(mFrameWindow);
> }
> 
> void AudioChannelService::PauseWindowAudio(nsPIDOMWindow* aWindow) {
>   AudioChannelWindow* winData = GetWindowData(aWindow->WindowID());
>   nsTObserverArray<nsRefPtr<AudioChannelAgent>>::ForwardIterator iter(winData->mAgents);
>   while (iter.HasMore()) {
>     iter.GetNext()->NotifyOwnerToPause();
>   }
> }
> 
> void AudioChannelAgent::NotifyOwnerToPause() {
>  callback->OwnerPause();
> }
> 
> void HTMLMediaElement::OwnerPause() {
>   this->Pause();
> }

---

Hi, Baku,
I have already discuss that with Dominic/Evan, and they think this kind of way might be acceptable.
Could you give me some suggestions?
Very appreciate :)

Comment 7

[Andrea Marchesini [:baku]]

> I think that we CAN NOT use the AudioChannelService::SetAudioChannelMuted to
> achieve this feature.

Why we cannot use AudioChannelService? The app can listen for mozinterruptbegin to know when the UI has to be updated.
But maybe I miss something.

Comment 8

[Dominic Kuo [:dkuo]]

(In reply to Andrea Marchesini (:baku) from comment #7)
> > I think that we CAN NOT use the AudioChannelService::SetAudioChannelMuted to
> > achieve this feature.
> 
> Why we cannot use AudioChannelService? The app can listen for
> mozinterruptbegin to know when the UI has to be updated.
> But maybe I miss something.

I think for the cases like Music/Video is okay for the apps to listen to mozinterruptbegin then pause the media elements or web audio, but for the regular web pages in the Browser app, unless the web page(like youtube) listens to mozinterruptbegin or the audio won't pause by itself.

Though the System app could use mute the web pages but think of if the Browser app is in foreground, and the user unplugs the headphones(the audio routing changed), the media element's state will be interrupted and no corresponding ui(it should still be the playing state ui), obviously it's a strange ux because the audio is muted but the state displays "playing".

So to actually pause the media elements or web audio from the System app should be an alternative way to pause the audio and the ui will display the correct state, if the pause function will trigger pause() on the media elements then cause the ui changes.

Comment 9

[Andrea Marchesini [:baku]]

Ok! Got it! Thanks.
The problem I see is that we can pause only MediaElements. WebAudio cannot be paused.
Would be nice if we can find a generic solutions.

In general I agree with the second option of Alastor. but I have another question:

why we cannot 'stop' all the MediaElements when the headset is removed for normal/content audioChannel?
Can we do it generically for those audioChannel? If yes we don't need an additional API.

A similar concept is introduced by the MediaSession API.

Comment 10

[BMO Automation]

Firefox OS is not being worked on