946556 - Progress bar moves when music is not playing

Status: RESOLVED FIXED

(Firefox OS Graveyard :: AudioChannel, defect)

Description

[bhargavg1]

Steps to reproduce:
1.Connect to BT headset.
2.Make MT call. Attend it & keep it in background.
3.Start playback from Music app.

Expected behavior:Voice call+Playback should be audible through BT headset.

Observed that,Voice call is audible but playback is not audible.

Progress bar moves ahead in the music app but Music is not heard.

Comment 1

[Ben Tian (inactive)]

ni? UX for expected behavior and Star for audio mixer capability.

Neo, what should be audible in BT headset when user plays music during voice call?

Star, do we have voice call + music playback audio mixing capability?

Comment 2

[Star Cheng [:scheng]]

I will investigate it and test.

Comment 3

[Neo Hsieh [:neo]]

Hi Ben, Normally we should pause the music and let user only hear the call voice. I think if we want music play in background, that is a new feature.

Comment 4

[Star Cheng [:scheng]]

According to current audio policy, the music stream can not be routed to SCO path.

Comment 5

[bhargavg1]

(In reply to Star Cheng [:scheng] from comment #4)
> According to current audio policy, the music stream can not be routed to SCO
> path.

Thanks for the info. Will update the component to Music, to read the status correctly and not not update the scroll bar

Comment 6

[Marco Chen [:mchen]]

Hi,

In the current audio competing policy, all foreground audio/video can be playable (Bug 828200) so it is under design of music app is on playing. Then the issue is that the audio HAL didn't route music stream type into SCO connection.

Comment 7

[Dominic Kuo [:dkuo]]

(In reply to bhargavg1 from comment #5)
> (In reply to Star Cheng [:scheng] from comment #4)
> > According to current audio policy, the music stream can not be routed to SCO
> > path.
> 
> Thanks for the info. Will update the component to Music, to read the status
> correctly and not not update the scroll bar

I wouldn't say this is an issue for music according to comment 6, the current audio competing policy was intended not to let the apps handle the extra logic for audio competing, so for music app, it just play the song once the user bring it to the foreground and tap play button. Music app does play the song but the sound will not route to SCO connection, that's why the progress bar is still going without sound.

We are still lacking a complete plan for the audio competing policy, I am planning to start a thread to discuss about this between UX and devs, let's have this issue in the component of Gaia::Music first then see if we should change it after the whole picture has come out.

Comment 8

[Hema Koka [:hema]]

Just going by the bug desc and the latest comments by Dominic, music continues to progress without audio (i.e does not route to SCO channel) when a call is in the background.  And it is how this was designed as per current audio policy 

Sri/Marvin, would be great if we can come with the audio routing policy since there seem to be multiple bugs around the same issue. It will be great if we can document the various use cases and define the audio competing policy requirements.

Comment 9

[Sri Kasetti]

I don't own the audio policy. Adding Marvin to the bug

Comment 10

[Marco Chen [:mchen]]

Hi Michael,

May I know your suggestion from the view of HAL layers?

In case that our UX spec defined music app should be allowed to play in the foreground no matter any other conditions.
Could Audio (HW/Policy) HAL be modified to output audio via output devices (speaker/receiver/headset/sco)?

Thanks.

Comment 11

[Michael Vines [:m1] [:evilmachines]]

(In reply to Marco Chen [:mchen] from comment #10) 
> In case that our UX spec defined music app should be allowed to play in the
> foreground no matter any other conditions.

Why would we want such a UX?

Comment 12

[Marco Chen [:mchen]]

(In reply to Michael Vines [:m1] [:evilmachines] from comment #11)
> Why would we want such a UX?

Actually the initial request of this is fired by you. :)
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c0

Solution and feedback from UX is on the same bug.
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c5
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c6

Comment 13

[Michael Vines [:m1] [:evilmachines]]

Does this issue also manifest on v1.2?

Comment 14

[bhargavg1]

(In reply to Michael Vines [:m1] [:evilmachines] from comment #13)
> Does this issue also manifest on v1.2?

Yes its the same behavior as in, no Music is heard when in a call over BT

Comment 15

[Bruce Huang [:bhuang] (inactive)]

Clearing my ni as it seems unrelated to BT specifically

Comment 16

[bhargavg1]

blocking for 1.3, as there needs to be a fix still in the UI

Comment 17

[Marvin Khoo [:Marvin_Khoo]]

Hi Macro, Michael,
I think this quite make sense to stop music playback once call is activated.
let's do the fix and Pause progress bar in Notification tray. When call is ended, resume the music playback.

doable?

Comment 18

[Marco Chen [:mchen]]

(In reply to Marvin Khoo [:Marvin_Khoo] from comment #17)
> Hi Macro, Michael,
> I think this quite make sense to stop music playback once call is activated.
> let's do the fix and Pause progress bar in Notification tray. When call is
> ended, resume the music playback.
> 
> doable?

I think it is doable but we need UX's input to release and confirm the spec.
By the way, Dominic fired a mail to dev-b2g/gaia and ux's mail loop for collecting and coming out the story for whole audio competing policy we knew it currently.

Comment 19

[Hema Koka [:hema]]

NI'ing jacqueline (assuming you are the right contact for this) based on comment 18. Please provide Ux input.

I agree that we need to pause the progress bar especially when there is *no* audio/music is being heard during voice call in the background. Dominic or Jim, can one of you take care of that part for 1.3 and look at the audio policy requirements (assuming that will fall under marco's team) for future release. Marvin/Sri, if there are any objections, please let us know. 

Thanks
Hema

Comment 20

[Jim Porter (:squib)]

This sounds like an audio competing policy bug and a regression from bug 828200. We'd need input on what the proper UX is here, since I'm thinking some stuff *should* play muted while you're on a call (video), and some stuff should pause (audio).

Comment 21

[Dominic Kuo [:dkuo]]

For 1.3 we can probably pause the progress bar while the music is playing but the sound is not routed to the bluetooth headset(SCO connection), I think there is one approach might be achievable:

1. Since music app already has the bluetooth permission for AVRCP, we should be able to know the states of SCO connection as well, so by checking the states, music app can pause/resume the progress bar.

I haven't try this in a real patch so don't know it work or not, but still we better have ux inputs that consider the audio competing policy for 1.3 and future release.

Comment 22

[Hema Koka [:hema]]

Jacqueline, 

Could you please provide your input. 

Thanks
Hema

Comment 23

[Jacqueline Savory [:jsavory] UX]

Yes, I agree with comment 18, regarding music playback. Once a call is initiated, the music and progress bar should pause and automatically continue after the call has ended.

Comment 24

[Hema Koka [:hema]]

Jim, 

Please sync with dkuo and address comment 17 for the scope of this bug

Thanks
Hema

Comment 25

[Hema Koka [:hema]]

Marking this a blocker since this seems to be a basic case and bad experience when music seems to progress but user cannot hear anything

Comment 26

[Dominic Kuo [:dkuo]]

Jim, since the patch needs bluetooth headset that supports hands-free profile, I remember you don't have one so I will take this first. I will first try the approach I mentioned in comment 21 to see if my assumption is workable, update later.

Comment 27

[Hema Koka [:hema]]

Dominic - please provide an update on where we are on this fix? Can this be wrapped up end of the week 1/17?

Thanks
Hema

Comment 28

[Dominic Kuo [:dkuo]]

Sorry about the late reply, after the offline meeting of audio competing, ux and devs decided we still let the music play while users are in a call and uses a bluetooth headset, only if they try to bring the music app to the foreground.

And once users bring the music app to the foreground, music should play and the sound should come out from the speaker, so the progress bar still goes on and users will not be confused.

We should be able to route the sound to the speaker, Star, since your are investigating this issue, would you please provide the information after you checked the code? thanks.

Comment 29

[Star Cheng [:scheng]]

I will try to set forceuse(media, speaker) function in the scenario of this case. The forceuse() function is an android AudioPolicy API, it can force to set the audio stream type (media) to the specific output device (speaker). After trying it, I will provide the result.

Comment 30

[Peter Dolanjski [:pdol]]

PM retriaged this and it is definitely a blocker.

Comment 31

[Dylan Oliver [:doliver]]

Star, have you been able to make progress on this issue?

Comment 32

[Star Cheng [:scheng]]

I set forceuse(media, speaker) in the scenario of this case, but the audio stream is routed to BT headset instead of speaker. I need chip vendor to help me to check the Audio Policy HAL.

Comment 33

[Preeti Raghunath(:Preeti)]

Hema

Adding you for music

Comment 34

[Star Cheng [:scheng]]

Attachment: Bug946556.patch

This is WIP patch - forceuse (media, speaker) in this case scenario.

Comment 35

[Hema Koka [:hema]]

From the comments, Star Cheng has been working on this (changing owner to the right person)

Star,

Are you blocked on getting confirmation on Audio HAL? Or are you asking for feedback on the patch. Please clarify on the next step for this bug. This is a CS blocker and hoping to get it fixed as early as possible. 

The issue is surfacing on music but is a audio competing policy/routing bug. 

Thanks
Hema

Comment 36

[Dominic Kuo [:dkuo]]

Hi all,

Let me explain this again to clarify the question, there are three approaches that devs can provide to ux for this scenario, they are:

1. Route the background music to the phone speaker. (This is the latest decision from ux team in comment 28, but Star mentioned in comment 32 that he needs to check the audio HAL, so it takes time and won't be able to make it in 1.3.)
2. Route the background music to the bluetooth headset. (Star has the WIP in attachment 8363424 [details] [diff] [review])
3. Pause/Disable the background music while the user is in call via bluetooth headset. (Jacqueline mentioned in comment 23 and I have implemented it with the approach in comment 21, attaching WIP later)

So for 1.3 I think we probably only have two choices, they are 2 and 3, and 3 is more likely the one gives better ux.

Comment 37

[Dominic Kuo [:dkuo]]

Attachment: Disable the play controls when using a bluetooth headset in a call

Comment 38

[Dominic Kuo [:dkuo]]

I will bring these two patches to ux team and let them decide tomorrow, update later.

Comment 39

[bhargavg1]

(In reply to Star Cheng [:scheng] from comment #34)
> Created attachment 8363424 [details] [diff] [review]
> Bug946556.patch
> 
> This is WIP patch - forceuse (media, speaker) in this case scenario.

Hi checked this, doesnt seem to help. Attached are the adb logs

Comment 40

[bhargavg1]

Attachment: log_bhargavg_bt2.log

Comment 41

[Star Cheng [:scheng]]

(In reply to bhargavg1 from comment #40)
> Created attachment 8364611 [details]
> log_bhargavg_bt2.log


--
AudioPolicyManagerALSA: clearComboDevice - No Combo device use case with ULL for device with two differnet backends
--

After checking the log, it seems to clear ComboDevice (media to Speaker, SCO to BT device).

Comment 42

[Star Cheng [:scheng]]

Hi, Michael

According to UX team design, UX team prefer to route music stream to speaker while SCO link is connected. I try to setforceuse(media, speaker) while SCO is connected (see Attachment 8363424 [details] [diff]), but the music stream is routed to BT SCO devices. Could you provide any information about AudioPolicy HAL and audio backend. 

device : Hamachi  / OS : 1.3 (ICS)

Comment 43

[Dominic Kuo [:dkuo]]

(In reply to Dominic Kuo [:dkuo] from comment #38)
> I will bring these two patches to ux team and let them decide tomorrow,
> update later.

After discussed with Harly(who owns the hardware related ux), we decide to go for approach 3: Pause/Disable the background music and controls while the user is in a call via bluetooth headset, we all agreed it has better ux than approach 2 for 1.3, so I will finish the WIP but since it became like a feature, I need more time to complete it and hopefully the ETA will be tomorrow.

Comment 44

[Preeti Raghunath(:Preeti)]

Thanks Dominic,

Appreciate it being on your radar.

Comment 45

[Dominic Kuo [:dkuo]]

Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

Okay, I think I almost finished the patch, I am still checking if there is any scenario the patch didn't handle well, but actually the patch is good to go. Jim, would you please review this? thanks. I guess you don't have a bluetooth headset so I am recording a demo video and attach it later, hope it helps on the review.

Comment 46

[Dominic Kuo [:dkuo]]

Attachment: Video - disable the music player when SCO is connected

Jacqueline and Harly,

The patch modified the music player and the hardware(bluetooth headset) is also involved, so I think it needs both of you to do the ui-review. Basically the idea is the music player will be disabled when the user is using the bluetooth headset in a call(SCO is connected), after the call is ended the player will be re-enabled and the music will resume automatically. I implemented this base on my understanding of comment 23 and feel free to give any feedback, thanks.

Jim,

I think this video can help you to review my patch, since you probably don't have a bluetooth headset that supports hands-free profile, let me know if you have any question, thanks.

Comment 47

[Jim Porter (:squib)]

I'm guessing we're going to need to do something with the lockscreen/utility tray controls, right? I'll take a look at that, but any UX input would be helpful.

Comment 48

[Dominic Kuo [:dkuo]]

(In reply to Jim Porter (:squib) from comment #47)
> I'm guessing we're going to need to do something with the lockscreen/utility
> tray controls, right? I'll take a look at that, but any UX input would be
> helpful.

Yes, you are correct, probably we should also disable the controls on lockscreen/utility to sync with music app, or just ignore all the remote controls when SCO is connected, I will see if we can just modify music instead of changing too many code on lockscreen/utility.

Comment 49

[Jim Porter (:squib)]

Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

I've commented over on GitHub about this. I *think* it's ok, but that's mostly because I assume it's temporary code. If this were going to stick around forever, I think I'd want to have something a little more general-purpose. That said, this looks like it should resolve the issue, and then we can figure out something better to do for future versions.

In the long term, we'll probably have to do a lot of thinking about how the audio competing policy works so that we have all of these cases addressed cleanly...

We might want a different string for the notification that we can't play music, but I'll leave that up to jsavory. :)

Comment 50

[Jim Porter (:squib)]

(In reply to Dominic Kuo [:dkuo] from comment #48)
> Yes, you are correct, probably we should also disable the controls on
> lockscreen/utility to sync with music app, or just ignore all the remote
> controls when SCO is connected, I will see if we can just modify music
> instead of changing too many code on lockscreen/utility.

I think we'll need to modify apps/system/js/media_playback.js to know when playing music is disabled. Currently, if we're paused and the user taps the pause button, we immediately show the play icon so that things seem to respond faster. We'll need to disable that when you can't start playing music.

Here's the code I'm talking about: https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/media_playback.js#L121

Comment 51

[Marco Chen [:mchen]]

Hi Jim and Dominic,

Thanks to follow up this issue.
And it seems that the WIP from Star for gecko patch is not the solution right now.

I unassigned Star and maybe one of you would take it.

Comment 52

[Dominic Kuo [:dkuo]]

Thanks Jim, I agreed with this patch should only for 1.3, after the long-term solution for audio competing has come out, we should remove this patch. I will address the github issues first and take a look on apps/system/js/media_playback.js, that's the scenario I didn't consider of, so I need to add more code to handle it, after that I should ask review from you again, also let's wait Jacqueline and see if she had better string for the notification.

I am working on this so assigning to myself.

Comment 53

[Preeti Raghunath(:Preeti)]

Dominic,

Please provide an ETA for the patch to land.

Comment 54

[Jacqueline Savory [:jsavory] UX]

Regarding the string, I think it would be best if the string suggested that the music is paused for the current activity. I would suggest something like "Music paused during call" for this example. 

However, this situation may pop up for other audio cases, in which case we could display "Music paused for current activity" 

Let me know if this makes sense to everyone :)

Comment 55

[Jim Porter (:squib)]

(In reply to jsavory from comment #54)
> Regarding the string, I think it would be best if the string suggested that
> the music is paused for the current activity. I would suggest something like
> "Music paused during call" for this example. 
> 
> However, this situation may pop up for other audio cases, in which case we
> could display "Music paused for current activity" 

I'm actually not sure this situation can happen for any case except a call; I'll let Dominic comment on that, though. We're only pausing music when an SCO link is established (i.e. you're using a hands-free device on a call). Maybe the first string would be better.

Comment 56

[Dominic Kuo [:dkuo]]

(In reply to Jim Porter (:squib) from comment #55)
> (In reply to jsavory from comment #54)
> > Regarding the string, I think it would be best if the string suggested that
> > the music is paused for the current activity. I would suggest something like
> > "Music paused during call" for this example. 
> > 
> > However, this situation may pop up for other audio cases, in which case we
> > could display "Music paused for current activity" 
> 
> I'm actually not sure this situation can happen for any case except a call;
> I'll let Dominic comment on that, though. We're only pausing music when an
> SCO link is established (i.e. you're using a hands-free device on a call).
> Maybe the first string would be better.

Since we only disable the music player when SCO link is connected, as Jim said we are using a hands-free device on a call, I also prefer something like Jacqueline said, "Music paused during the call", or probably "Music paused during the call via bluetooth headset". For the other situations we don't really block the ui, so let's have the first string for the notification.

Testing the modified patch takes time because I need to test on real devices, including the bluetooth headset and making a call from one to another, but I should have a modified patch today, and hopefully I can get a second r+ from Jim and land it tomorrow.

Comment 57

[Dominic Kuo [:dkuo]]

Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

Okay, I think I have addressed the issues Jim and Jacqueline mentioned above, one is to disable the playback controls on lockscreen/utility while SCO is connected, another is change the notification string to "Music paused during the call via bluetooth headset". After tried the patch myself, I really think this is a worthy modification! though it became a feature...

Jim, would you please review it again? thanks, I am attaching the screenshots later and you can see what it looks like.

Comment 58

[Dominic Kuo [:dkuo]]

Attachment: Screenshots for disabling music player, lockscreen and utility playback controls

Comment 59

[Jim Porter (:squib)]

Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

This looks good to me! I have a couple of minor comments on the pull request, though; mostly just capitalizing a few other instances of "Sco". You only really need to capitalize the public attributes of the MediaRemoteControls. The private variables/functions can use whatever capitalization you want.

Also, I think we should go with Jacqueline's suggestion for the string: "Music paused during call", since it's shorter. That will make it easier for localizers to fit their translation in the available space.

Comment 60

[Jacqueline Savory [:jsavory] UX]

Comment on attachment 8366826 [details]
Video - disable the music player when SCO is connected

This looks great! I agree that the string probably doesn't need to add the via bluetooth headset, as all they have to do is end the call to resume their music.

Comment 61

[Jim Porter (:squib)]

Dominic: If you happen to see this, I'll probably land your patch with some small changes later today. Hopefully this won't cause any problems for you. If you're actively working on stuff here, just let me know and I'll leave it to you. Otherwise, I'll handle it so you can go celebrate.

In either case, Happy New Year!

Comment 62

[Dominic Kuo [:dkuo]]

Hey Jim, I am about to land my patch and just checking again on my device, thanks for taking care of this bug, I will land it later!

Comment 63

[Dominic Kuo [:dkuo]]

Landed on master: 0646bf911164ca9e2f80a6b703558bed35b449b2

Thanks everyone and Happy New Year!

Comment 64

[Dominic Kuo [:dkuo]]

v1.3: f9a37c77efb4621a1f57e4695b497d18601fe134

Comment 65

[Dominic Kuo [:dkuo]]

The patch has broken the music open activity but fortunately it just needed an easy fix, I am reopening this bug and re-commit it again later. Sorry about the inconvenience.

Reverted on master: 9c0fb85e150a87fcf7fc1857f52f0168baf2ff72
v1.3: b8ce45734ccb1bcb7713c66f631bee5375fdf9e8

Comment 66

[Dominic Kuo [:dkuo]]

The patch is now re-landed again.

master: 0ed0da7ea8c31e6df902deadaef4012f12911c39
v1.3: ebc3b39b7e8360c7a2e7823ab3a1d9dad07e22cf

Comment 67

[Harly Hsu[:harly]]

Comment on attachment 8366826 [details]
Video - disable the music player when SCO is connected

I must have missed the ui-review, thanks for the great work, Dominic!

Comment 69

[Alissa（leave from Mozilla）]

This issue has been verified failed on Nexus5 2.2.
While there is no BT device or headset connected, we can play the music, but when our face close to screen,the music will paused, if we need new a issue to follow this case?

N5_2.2
Build ID               20150309162503
Gaia Revision          166491b92278dc9e648f8d49ab02d9ca00d74421
Gaia Date              2015-03-06 18:26:27
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/9ae4bcc9b5f2
Gecko Version          37.0
Device Name            hammerhead
Firmware(Release)      5.0
Firmware(Incremental)  eng.cltbld.20150309.195940
Firmware Date          Mon Mar  9 19:59:53 EDT 2015
Bootloader             HHZ12d

Comment 70

[KTucker [:KTucker][Inactive 3/4/2016]]

Alissa, I think that this should be written as a new bug.