Closed Bug 946556 Opened 6 years ago Closed 6 years ago

Progress bar moves when music is not playing

Categories

(Firefox OS Graveyard :: AudioChannel, defect, major)

ARM
Gonk (Firefox OS)
defect
Not set
major

Tracking

(blocking-b2g:1.3+, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 fixed)

RESOLVED FIXED
1.3 C3/1.4 S3(31jan)
blocking-b2g 1.3+
Tracking Status
b2g-v1.3 --- fixed
b2g-v1.3T --- fixed
b2g-v1.4 --- fixed

People

(Reporter: bhargavg1, Assigned: dkuo)

References

()

Details

(Keywords: late-l10n, Whiteboard: [caf priority: p2][cr 584883], [AUDIO_COMPETING][fxos:media][v2.2-nexus-5-l])

Attachments

(5 files)

Steps to reproduce:
1.Connect to BT headset.
2.Make MT call. Attend it & keep it in background.
3.Start playback from Music app.

Expected behavior:Voice call+Playback should be audible through BT headset.

Observed that,Voice call is audible but playback is not audible.

Progress bar moves ahead in the music app but Music is not heard.
Blocks: 930299
blocking-b2g: --- → 1.3?
Whiteboard: PRISM:584883
ni? UX for expected behavior and Star for audio mixer capability.

Neo, what should be audible in BT headset when user plays music during voice call?

Star, do we have voice call + music playback audio mixing capability?
Flags: needinfo?(scheng)
Flags: needinfo?(nhsieh)
I will investigate it and test.
Flags: needinfo?(scheng)
Hi Ben, Normally we should pause the music and let user only hear the call voice. I think if we want music play in background, that is a new feature.
Flags: needinfo?(nhsieh) → needinfo?(bhuang)
According to current audio policy, the music stream can not be routed to SCO path.
(In reply to Star Cheng [:scheng] from comment #4)
> According to current audio policy, the music stream can not be routed to SCO
> path.

Thanks for the info. Will update the component to Music, to read the status correctly and not not update the scroll bar
Component: Bluetooth → Gaia::Music
blocking-b2g: 1.3? → 1.3+
Summary: no music heard on bluetooth headset while in call → Progress bar moves when not music is not playing
Blocks: 942267
No longer blocks: 930299
Hi,

In the current audio competing policy, all foreground audio/video can be playable (Bug 828200) so it is under design of music app is on playing. Then the issue is that the audio HAL didn't route music stream type into SCO connection.
(In reply to bhargavg1 from comment #5)
> (In reply to Star Cheng [:scheng] from comment #4)
> > According to current audio policy, the music stream can not be routed to SCO
> > path.
> 
> Thanks for the info. Will update the component to Music, to read the status
> correctly and not not update the scroll bar

I wouldn't say this is an issue for music according to comment 6, the current audio competing policy was intended not to let the apps handle the extra logic for audio competing, so for music app, it just play the song once the user bring it to the foreground and tap play button. Music app does play the song but the sound will not route to SCO connection, that's why the progress bar is still going without sound.

We are still lacking a complete plan for the audio competing policy, I am planning to start a thread to discuss about this between UX and devs, let's have this issue in the component of Gaia::Music first then see if we should change it after the whole picture has come out.
Just going by the bug desc and the latest comments by Dominic, music continues to progress without audio (i.e does not route to SCO channel) when a call is in the background.  And it is how this was designed as per current audio policy 

Sri/Marvin, would be great if we can come with the audio routing policy since there seem to be multiple bugs around the same issue. It will be great if we can document the various use cases and define the audio competing policy requirements.
Flags: needinfo?(skasetti)
I don't own the audio policy. Adding Marvin to the bug
Flags: needinfo?(skasetti) → needinfo?(mkhoo)
Hi Michael,

May I know your suggestion from the view of HAL layers?

In case that our UX spec defined music app should be allowed to play in the foreground no matter any other conditions.
Could Audio (HW/Policy) HAL be modified to output audio via output devices (speaker/receiver/headset/sco)?

Thanks.
Flags: needinfo?(mvines)
(In reply to Marco Chen [:mchen] from comment #10) 
> In case that our UX spec defined music app should be allowed to play in the
> foreground no matter any other conditions.

Why would we want such a UX?
Flags: needinfo?(mvines)
(In reply to Michael Vines [:m1] [:evilmachines] from comment #11)
> Why would we want such a UX?

Actually the initial request of this is fired by you. :)
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c0

Solution and feedback from UX is on the same bug.
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c5
https://bugzilla.mozilla.org/show_bug.cgi?id=828200#c6
Does this issue also manifest on v1.2?
Flags: needinfo?(bhargavg1)
(In reply to Michael Vines [:m1] [:evilmachines] from comment #13)
> Does this issue also manifest on v1.2?

Yes its the same behavior as in, no Music is heard when in a call over BT
Flags: needinfo?(bhargavg1)
No longer blocks: 942267
blocking-b2g: 1.3+ → ---
Summary: Progress bar moves when not music is not playing → Progress bar moves when music is not playing
Clearing my ni as it seems unrelated to BT specifically
Flags: needinfo?(bhuang)
blocking for 1.3, as there needs to be a fix still in the UI
blocking-b2g: --- → 1.3?
Hi Macro, Michael,
I think this quite make sense to stop music playback once call is activated.
let's do the fix and Pause progress bar in Notification tray. When call is ended, resume the music playback.

doable?
Flags: needinfo?(mkhoo)
(In reply to Marvin Khoo [:Marvin_Khoo] from comment #17)
> Hi Macro, Michael,
> I think this quite make sense to stop music playback once call is activated.
> let's do the fix and Pause progress bar in Notification tray. When call is
> ended, resume the music playback.
> 
> doable?

I think it is doable but we need UX's input to release and confirm the spec.
By the way, Dominic fired a mail to dev-b2g/gaia and ux's mail loop for collecting and coming out the story for whole audio competing policy we knew it currently.
NI'ing jacqueline (assuming you are the right contact for this) based on comment 18. Please provide Ux input.

I agree that we need to pause the progress bar especially when there is *no* audio/music is being heard during voice call in the background. Dominic or Jim, can one of you take care of that part for 1.3 and look at the audio policy requirements (assuming that will fall under marco's team) for future release. Marvin/Sri, if there are any objections, please let us know. 

Thanks
Hema
Flags: needinfo?(squibblyflabbetydoo)
Flags: needinfo?(dkuo)
Flags: needinfo?(jsavory)
This sounds like an audio competing policy bug and a regression from bug 828200. We'd need input on what the proper UX is here, since I'm thinking some stuff *should* play muted while you're on a call (video), and some stuff should pause (audio).
Flags: needinfo?(squibblyflabbetydoo)
For 1.3 we can probably pause the progress bar while the music is playing but the sound is not routed to the bluetooth headset(SCO connection), I think there is one approach might be achievable:

1. Since music app already has the bluetooth permission for AVRCP, we should be able to know the states of SCO connection as well, so by checking the states, music app can pause/resume the progress bar.

I haven't try this in a real patch so don't know it work or not, but still we better have ux inputs that consider the audio competing policy for 1.3 and future release.
Flags: needinfo?(dkuo)
Blocks: 956409
No longer blocks: 956409
Jacqueline, 

Could you please provide your input. 

Thanks
Hema
Yes, I agree with comment 18, regarding music playback. Once a call is initiated, the music and progress bar should pause and automatically continue after the call has ended.
Flags: needinfo?(jsavory)
Jim, 

Please sync with dkuo and address comment 17 for the scope of this bug

Thanks
Hema
Assignee: nobody → squibblyflabbetydoo
Marking this a blocker since this seems to be a basic case and bad experience when music seems to progress but user cannot hear anything
blocking-b2g: 1.3? → 1.3+
Jim, since the patch needs bluetooth headset that supports hands-free profile, I remember you don't have one so I will take this first. I will first try the approach I mentioned in comment 21 to see if my assumption is workable, update later.
Assignee: squibblyflabbetydoo → dkuo
Target Milestone: --- → 1.3 C2/1.4 S2(17jan)
Whiteboard: PRISM:584883 → PRISM:584883, [AUDIO_COMPETING]
Whiteboard: PRISM:584883, [AUDIO_COMPETING] → [cr 584883] [AUDIO_COMPETING]
Whiteboard: [cr 584883] [AUDIO_COMPETING] → [cr 584883], [AUDIO_COMPETING]
Dominic - please provide an update on where we are on this fix? Can this be wrapped up end of the week 1/17?

Thanks
Hema
Blocks: 942267
Sorry about the late reply, after the offline meeting of audio competing, ux and devs decided we still let the music play while users are in a call and uses a bluetooth headset, only if they try to bring the music app to the foreground.

And once users bring the music app to the foreground, music should play and the sound should come out from the speaker, so the progress bar still goes on and users will not be confused.

We should be able to route the sound to the speaker, Star, since your are investigating this issue, would you please provide the information after you checked the code? thanks.
Flags: needinfo?(scheng)
I will try to set forceuse(media, speaker) function in the scenario of this case. The forceuse() function is an android AudioPolicy API, it can force to set the audio stream type (media) to the specific output device (speaker). After trying it, I will provide the result.
Flags: needinfo?(scheng)
PM retriaged this and it is definitely a blocker.
Star, have you been able to make progress on this issue?
Flags: needinfo?(scheng)
I set forceuse(media, speaker) in the scenario of this case, but the audio stream is routed to BT headset instead of speaker. I need chip vendor to help me to check the Audio Policy HAL.
Flags: needinfo?(scheng)
Hema

Adding you for music
Flags: needinfo?(hkoka)
Attached patch Bug946556.patchSplinter Review
This is WIP patch - forceuse (media, speaker) in this case scenario.
From the comments, Star Cheng has been working on this (changing owner to the right person)

Star,

Are you blocked on getting confirmation on Audio HAL? Or are you asking for feedback on the patch. Please clarify on the next step for this bug. This is a CS blocker and hoping to get it fixed as early as possible. 

The issue is surfacing on music but is a audio competing policy/routing bug. 

Thanks
Hema
Assignee: dkuo → scheng
Component: Gaia::Music → AudioChannel
Flags: needinfo?(hkoka)
Whiteboard: [cr 584883], [AUDIO_COMPETING] → [cr 584883], [AUDIO_COMPETING][fxos:media]
Target Milestone: 1.3 C2/1.4 S2(17jan) → 1.3 C3/1.4 S3(31jan)
Hi all,

Let me explain this again to clarify the question, there are three approaches that devs can provide to ux for this scenario, they are:

1. Route the background music to the phone speaker. (This is the latest decision from ux team in comment 28, but Star mentioned in comment 32 that he needs to check the audio HAL, so it takes time and won't be able to make it in 1.3.)
2. Route the background music to the bluetooth headset. (Star has the WIP in attachment 8363424 [details] [diff] [review])
3. Pause/Disable the background music while the user is in call via bluetooth headset. (Jacqueline mentioned in comment 23 and I have implemented it with the approach in comment 21, attaching WIP later)

So for 1.3 I think we probably only have two choices, they are 2 and 3, and 3 is more likely the one gives better ux.
I will bring these two patches to ux team and let them decide tomorrow, update later.
(In reply to Star Cheng [:scheng] from comment #34)
> Created attachment 8363424 [details] [diff] [review]
> Bug946556.patch
> 
> This is WIP patch - forceuse (media, speaker) in this case scenario.

Hi checked this, doesnt seem to help. Attached are the adb logs
Attached file log_bhargavg_bt2.log

(In reply to bhargavg1 from comment #40)
> Created attachment 8364611 [details]
> log_bhargavg_bt2.log


--
AudioPolicyManagerALSA: clearComboDevice - No Combo device use case with ULL for device with two differnet backends
--

After checking the log, it seems to clear ComboDevice (media to Speaker, SCO to BT device).
Hi, Michael

According to UX team design, UX team prefer to route music stream to speaker while SCO link is connected. I try to setforceuse(media, speaker) while SCO is connected (see Attachment 8363424 [details] [diff]), but the music stream is routed to BT SCO devices. Could you provide any information about AudioPolicy HAL and audio backend. 

device : Hamachi  / OS : 1.3 (ICS)
Flags: needinfo?(mvines)
(In reply to Dominic Kuo [:dkuo] from comment #38)
> I will bring these two patches to ux team and let them decide tomorrow,
> update later.

After discussed with Harly(who owns the hardware related ux), we decide to go for approach 3: Pause/Disable the background music and controls while the user is in a call via bluetooth headset, we all agreed it has better ux than approach 2 for 1.3, so I will finish the WIP but since it became like a feature, I need more time to complete it and hopefully the ETA will be tomorrow.
Thanks Dominic,

Appreciate it being on your radar.
Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

Okay, I think I almost finished the patch, I am still checking if there is any scenario the patch didn't handle well, but actually the patch is good to go. Jim, would you please review this? thanks. I guess you don't have a bluetooth headset so I am recording a demo video and attach it later, hope it helps on the review.
Attachment #8364473 - Attachment description: WIP - gaia patch → Disable the play controls when using a bluetooth headset in a call
Attachment #8364473 - Flags: review?(squibblyflabbetydoo)
Jacqueline and Harly,

The patch modified the music player and the hardware(bluetooth headset) is also involved, so I think it needs both of you to do the ui-review. Basically the idea is the music player will be disabled when the user is using the bluetooth headset in a call(SCO is connected), after the call is ended the player will be re-enabled and the music will resume automatically. I implemented this base on my understanding of comment 23 and feel free to give any feedback, thanks.

Jim,

I think this video can help you to review my patch, since you probably don't have a bluetooth headset that supports hands-free profile, let me know if you have any question, thanks.
Attachment #8366826 - Flags: ui-review?(jsavory)
Attachment #8366826 - Flags: ui-review?(hhsu)
I'm guessing we're going to need to do something with the lockscreen/utility tray controls, right? I'll take a look at that, but any UX input would be helpful.
(In reply to Jim Porter (:squib) from comment #47)
> I'm guessing we're going to need to do something with the lockscreen/utility
> tray controls, right? I'll take a look at that, but any UX input would be
> helpful.

Yes, you are correct, probably we should also disable the controls on lockscreen/utility to sync with music app, or just ignore all the remote controls when SCO is connected, I will see if we can just modify music instead of changing too many code on lockscreen/utility.
Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

I've commented over on GitHub about this. I *think* it's ok, but that's mostly because I assume it's temporary code. If this were going to stick around forever, I think I'd want to have something a little more general-purpose. That said, this looks like it should resolve the issue, and then we can figure out something better to do for future versions.

In the long term, we'll probably have to do a lot of thinking about how the audio competing policy works so that we have all of these cases addressed cleanly...

We might want a different string for the notification that we can't play music, but I'll leave that up to jsavory. :)
Attachment #8364473 - Flags: review?(squibblyflabbetydoo) → review+
(In reply to Dominic Kuo [:dkuo] from comment #48)
> Yes, you are correct, probably we should also disable the controls on
> lockscreen/utility to sync with music app, or just ignore all the remote
> controls when SCO is connected, I will see if we can just modify music
> instead of changing too many code on lockscreen/utility.

I think we'll need to modify apps/system/js/media_playback.js to know when playing music is disabled. Currently, if we're paused and the user taps the pause button, we immediately show the play icon so that things seem to respond faster. We'll need to disable that when you can't start playing music.

Here's the code I'm talking about: https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/media_playback.js#L121
Hi Jim and Dominic,

Thanks to follow up this issue.
And it seems that the WIP from Star for gecko patch is not the solution right now.

I unassigned Star and maybe one of you would take it.
Assignee: scheng → nobody
Thanks Jim, I agreed with this patch should only for 1.3, after the long-term solution for audio competing has come out, we should remove this patch. I will address the github issues first and take a look on apps/system/js/media_playback.js, that's the scenario I didn't consider of, so I need to add more code to handle it, after that I should ask review from you again, also let's wait Jacqueline and see if she had better string for the notification.

I am working on this so assigning to myself.
Assignee: nobody → dkuo
Dominic,

Please provide an ETA for the patch to land.
Flags: needinfo?(dkuo)
Keywords: late-l10n
Regarding the string, I think it would be best if the string suggested that the music is paused for the current activity. I would suggest something like "Music paused during call" for this example. 

However, this situation may pop up for other audio cases, in which case we could display "Music paused for current activity" 

Let me know if this makes sense to everyone :)
(In reply to jsavory from comment #54)
> Regarding the string, I think it would be best if the string suggested that
> the music is paused for the current activity. I would suggest something like
> "Music paused during call" for this example. 
> 
> However, this situation may pop up for other audio cases, in which case we
> could display "Music paused for current activity" 

I'm actually not sure this situation can happen for any case except a call; I'll let Dominic comment on that, though. We're only pausing music when an SCO link is established (i.e. you're using a hands-free device on a call). Maybe the first string would be better.
(In reply to Jim Porter (:squib) from comment #55)
> (In reply to jsavory from comment #54)
> > Regarding the string, I think it would be best if the string suggested that
> > the music is paused for the current activity. I would suggest something like
> > "Music paused during call" for this example. 
> > 
> > However, this situation may pop up for other audio cases, in which case we
> > could display "Music paused for current activity" 
> 
> I'm actually not sure this situation can happen for any case except a call;
> I'll let Dominic comment on that, though. We're only pausing music when an
> SCO link is established (i.e. you're using a hands-free device on a call).
> Maybe the first string would be better.

Since we only disable the music player when SCO link is connected, as Jim said we are using a hands-free device on a call, I also prefer something like Jacqueline said, "Music paused during the call", or probably "Music paused during the call via bluetooth headset". For the other situations we don't really block the ui, so let's have the first string for the notification.

Testing the modified patch takes time because I need to test on real devices, including the bluetooth headset and making a call from one to another, but I should have a modified patch today, and hopefully I can get a second r+ from Jim and land it tomorrow.
Flags: needinfo?(dkuo)
Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

Okay, I think I have addressed the issues Jim and Jacqueline mentioned above, one is to disable the playback controls on lockscreen/utility while SCO is connected, another is change the notification string to "Music paused during the call via bluetooth headset". After tried the patch myself, I really think this is a worthy modification! though it became a feature...

Jim, would you please review it again? thanks, I am attaching the screenshots later and you can see what it looks like.
Attachment #8364473 - Flags: review+ → review?(squibblyflabbetydoo)
Comment on attachment 8364473 [details] [review]
Disable the play controls when using a bluetooth headset in a call

This looks good to me! I have a couple of minor comments on the pull request, though; mostly just capitalizing a few other instances of "Sco". You only really need to capitalize the public attributes of the MediaRemoteControls. The private variables/functions can use whatever capitalization you want.

Also, I think we should go with Jacqueline's suggestion for the string: "Music paused during call", since it's shorter. That will make it easier for localizers to fit their translation in the available space.
Attachment #8364473 - Flags: review?(squibblyflabbetydoo) → review+
Comment on attachment 8366826 [details]
Video - disable the music player when SCO is connected

This looks great! I agree that the string probably doesn't need to add the via bluetooth headset, as all they have to do is end the call to resume their music.
Attachment #8366826 - Flags: ui-review?(jsavory) → ui-review+
Dominic: If you happen to see this, I'll probably land your patch with some small changes later today. Hopefully this won't cause any problems for you. If you're actively working on stuff here, just let me know and I'll leave it to you. Otherwise, I'll handle it so you can go celebrate.

In either case, Happy New Year!
Hey Jim, I am about to land my patch and just checking again on my device, thanks for taking care of this bug, I will land it later!
Landed on master: 0646bf911164ca9e2f80a6b703558bed35b449b2

Thanks everyone and Happy New Year!
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
v1.3: f9a37c77efb4621a1f57e4695b497d18601fe134
Flags: needinfo?(mvines)
The patch has broken the music open activity but fortunately it just needed an easy fix, I am reopening this bug and re-commit it again later. Sorry about the inconvenience.

Reverted on master: 9c0fb85e150a87fcf7fc1857f52f0168baf2ff72
v1.3: b8ce45734ccb1bcb7713c66f631bee5375fdf9e8
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
The patch is now re-landed again.

master: 0ed0da7ea8c31e6df902deadaef4012f12911c39
v1.3: ebc3b39b7e8360c7a2e7823ab3a1d9dad07e22cf
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
Comment on attachment 8366826 [details]
Video - disable the music player when SCO is connected

I must have missed the ui-review, thanks for the great work, Dominic!
Attachment #8366826 - Flags: ui-review?(hhsu) → ui-review+
Duplicate of this bug: 942710
Blocks: 973831
Flags: in-moztrap?
Flags: in-moztrap? → in-moztrap+
Whiteboard: [cr 584883], [AUDIO_COMPETING][fxos:media] → [caf priority: p2][cr 584883], [AUDIO_COMPETING][fxos:media]
See Also: → 1082503
This issue has been verified failed on Nexus5 2.2.
While there is no BT device or headset connected, we can play the music, but when our face close to screen,the music will paused, if we need new a issue to follow this case?

N5_2.2
Build ID               20150309162503
Gaia Revision          166491b92278dc9e648f8d49ab02d9ca00d74421
Gaia Date              2015-03-06 18:26:27
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/9ae4bcc9b5f2
Gecko Version          37.0
Device Name            hammerhead
Firmware(Release)      5.0
Firmware(Incremental)  eng.cltbld.20150309.195940
Firmware Date          Mon Mar  9 19:59:53 EDT 2015
Bootloader             HHZ12d
Flags: needinfo?(ktucker)
Whiteboard: [caf priority: p2][cr 584883], [AUDIO_COMPETING][fxos:media] → [caf priority: p2][cr 584883], [AUDIO_COMPETING][fxos:media][v2.2-nexus-5-l]
Alissa, I think that this should be written as a new bug.
Flags: needinfo?(ktucker)
You need to log in before you can comment on or make changes to this bug.