Closed Bug 1046578 Opened 11 years ago Closed 11 years ago

[Loop] In a call users cannot change volume

Categories

(Core :: Audio/Video, defect)

defect
Not set
normal

Tracking

()

VERIFIED FIXED
2.1 S4 (12sep)
blocking-b2g 2.0+
Tracking Status
firefox33 --- wontfix
firefox34 --- fixed
firefox35 --- fixed
b2g-v2.0 --- verified
b2g-v2.0M --- fixed
b2g-v2.1 --- fixed
b2g-v2.2 --- fixed

People

(Reporter: mbarone976, Assigned: baku, NeedInfo)

References

Details

(Whiteboard: ft:loop [blocking])

Attachments

(3 files)

Attached file log_receiver.txt
Device: Fire E / Flame Loop:1a7255d During a call between two Fire E or between Flame and Fire E, we have noticed that the audio volume is very low. Also there is no way to increase/decrease the volume. Attached the full log.
Retested with today Loop build: 609ec57 and Flame build: Gecko-7f737a2.Gaia-6155d46 and the audio quality and volume is fine. Still we have a delay of 2-3 seconds.
(In reply to mbarone from comment #2) > Retested with today Loop build: 609ec57 and Flame build: > Gecko-7f737a2.Gaia-6155d46 and the audio quality and volume is fine. Still > we have a delay of 2-3 seconds. Is this report from testing an audio only call or a audio+video call?
Only audio call
Where are the two phones that are calling each other? Are they on the same WiFi network? Or different WiFi networks? Or is this on 3G? Does this happen on every audio call or does it vary? Is this a recent regression (within the last 2 weeks or so)?
The test has been done with two devices in the same wifi and very close and also with two devices in different networks and different cities. And the audio is low in every calls.
How does this compare to yesterday (Comment 2: "audio quality and volume is fine. Still we have a delay of 2-3 seconds.")? What did you test today compared to yesterday? Thanks.
(In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #7) > How does this compare to yesterday (Comment 2: "audio quality and volume is > fine. Still we have a delay of 2-3 seconds.")? What did you test today > compared to yesterday? Thanks. Jayme, Peter, can you guys investigate this against the Flame? You'll need to grab the latest build and flash a KK build. Please report if you can reproduce this with the KK Flame image (v162-3) and run against 2.0.
Flags: needinfo?(pbylenga)
Flags: needinfo?(jmercado)
Keywords: qawanted
I was able to reproduce this issue on the latest 2.0 build running the v165 KK base. The audio was very low and could not be altered on either device. If the caller was in a video mode when the call started, the speaker phone option would work for that device and the sound output level was much lower than expected for a speaker-call; more at the volume level of a normal call. Environmental Variables: Device: Flame 2.0 BuildID: 20140815143240 Gaia: d889984833025f208cfd3f3c2c37c87940a529dc Gecko: 6329352ca531b977979451e77e5862af485388b2 Version: 32.0 (2.0) Firmware Version: v165 User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0 Loop version: 32219a2
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(pbylenga)
Flags: needinfo?(jmitchell)
Flags: needinfo?(jmercado)
Keywords: qawanted
Jayme, Can you see if this bug (low audio with an inability to adjust the volume) appears when using other audio apps or audio+video apps?
Flags: needinfo?(jmercado)
Marie, This issue did not occur when using other audio or audio/video apps, the sound was clear and could be turned up or down. Also the ringer for the Loop app could also be heard clearly and be turned in up and down.
Flags: needinfo?(jmercado)
Hi Jayme -- Were you using audio+video apps that were using both the microphone and camera at the same time? Also, could you try making a webrtc call in the browser? (You can use talky.io, for example.) Do webrtc calls made from inside the browser show this bug?
Flags: needinfo?(jmercado)
Marie, I had recorded a video with the camera app and then played it back as part of the tests above. The audio was at a level expected. This build does not seem to detect SIMs so I was not able to check a normal call done with the dialer. Do we have other apps that use both the camera, microphone, and speakers at once? Using talky.io the maximum audio was a little quiet but still audible, and the volume could be lowered or raised using the volume buttons.
Flags: needinfo?(jmercado)
Thanks, Jayme. I think we need to reach out to someone from the B2G audio channels team to figure out how to debug this. Whatever is going on, it's outside of WebRTC, and I think we should assume it's a platform issue until proven otherwise.
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(jmitchell)
Hi Randy, Steven, and Marco -- I'm not sure who the right person ia to look at this bug. I think we're looking for someone on the B2G audio channels team to debug why the audio volume is low and can not be changed during a Loop mobile call. Can you help me find the right person to triage this? Thanks!
Flags: needinfo?(slee)
Flags: needinfo?(rlin)
Flags: needinfo?(mchen)
Whiteboard: ft:loop
Josh - Can you make a blocking triage analysis here?
Flags: needinfo?(jmitchell)
Hi, Could someone help to get the log by calling dumpsys via adb? 1. There is a loop call between Device 1 & 2. 2. Get the logs from each device 1 & 2. 3. Adjust volume level from each device 1 & 2. 4. Get another logs from each device 1 & 2. By the way, could someone make sure that dialer have or not have this issue too? Thanks.
Flags: needinfo?(mchen) → needinfo?
Same as mchen, by run "adb shell dumpsys media.audio_policy" can check the volume setting on gonk side. It may adjust the wrong stream volume on this case.
Flags: needinfo?(rlin) → needinfo?
Hi Maire, I tried today's 2.0 and 2.1 with kk image. I'm using browser to connect to http://nightly-gupshup.herokuapp.com/login. Both devices work well. It seems only happens on loop?
Flags: needinfo?(slee)
Flags: needinfo?
(In reply to StevenLee[:slee] from comment #19) > I tried today's 2.0 and 2.1 with kk image. I'm using browser to connect to > http://nightly-gupshup.herokuapp.com/login. Both devices work well. It seems > only happens on loop? Yes, the reports are it only happens with the loop app
Do loop use the audioChannel API? For example, register the telephony channel?
(In reply to Randy Lin [:rlin] from comment #21) > Do loop use the audioChannel API? For example, register the telephony > channel? Hi rlin Does you mean this, https://github.com/mozilla-b2g/firefoxos-loop-client/blob/master/libs/tokbox/v2.2.6/js/TB.js#l4243 ?
Yes, fallback to normal channel I think can solve this problem. But why loop need to use this channel?
After discussing with Randy and Star, the cause should be resulted from the "wrong audiochannel type". loop registers the audiochannel as telephony, but gUM create the audio resource as normal. When user tries to change audio volume, audio channel manager only modifies "telephony" channel. So users cannot change the volume. So we could add one more channel type, ex voipchannel, and it has the same priority as telephony. Or when creating microphone resource in gUM, we should set the resource as telephony type. Then audio channel manager can change the volume of correct audio.
Flags: needinfo?(rjesup)
Flags: needinfo?(mreavy)
I know there was extensive discussion of the exact AudioChannel usage to get the desired ringing/override behaviors, but I wasn't involved in these. As for getUserMedia, it generally has no idea what the stream will be used for (and does nothing to set the type to anything). Options would involve setting a default audio channel type for the app, setting a type in a gUM constraint (webidl dictionary change), perhaps others - I don't know how AudioChannels are specified. Something that's purely app-only (especially) or is a simple, easy-to-test-and-quickly-uplift change to gUM is far preferred. I think moving the entire usage for getUserMedia to Telephony is likely to cause problems for other non-loop and especially non-VoIP-calling apps - I'm worried it may block calls, ringing, etc. But that depends a lot on what "Telephony" does. Needinfo'ing a few people I'm told were involved and weren't on the CC
Flags: needinfo?(rjesup)
Flags: needinfo?(josea.olivera)
Flags: needinfo?(21)
The Loop mobile app is scheduled to ship in v2.0. Do we need to make platform/FxOS changes in order to fix this, or is there something the app can do to work around this?
Flags: needinfo?(mreavy)
[Blocking Requested - why for this release]: (In reply to Jason Smith [:jsmith] from comment #16) > Josh - Can you make a blocking triage analysis here? nomming as a blocker - the low volume levels can be very bad UX as well as making features unusable for users with less than perfect hearing
blocking-b2g: --- → 2.0?
Flags: needinfo?(jmitchell)
QA Whiteboard: [QAnalyst-Triage+] → [QAnalyst-Triage+][lead-review+]
(In reply to Marco Chen [:mchen] from comment #17) > Hi, > > Could someone help to get the log by calling dumpsys via adb? > > 1. There is a loop call between Device 1 & 2. > 2. Get the logs from each device 1 & 2. > 3. Adjust volume level from each device 1 & 2. > 4. Get another logs from each device 1 & 2. > > By the way, could someone make sure that dialer have or not have this issue > too? > > Thanks. Can someone provide the information?
Flags: needinfo?(mreavy)
(In reply to StevenLee[:slee] from comment #28) > (In reply to Marco Chen [:mchen] from comment #17) > > Hi, > > > > Could someone help to get the log by calling dumpsys via adb? > > > > 1. There is a loop call between Device 1 & 2. > > 2. Get the logs from each device 1 & 2. > > 3. Adjust volume level from each device 1 & 2. > > 4. Get another logs from each device 1 & 2. > > > > By the way, could someone make sure that dialer have or not have this issue > > too? > > > > Thanks. > Can someone provide the information? Jayme -- Can you provide the info that Marco asks for? ^^ In particular, does the dialer have this same issue as the loop app?
Flags: needinfo?(mreavy) → needinfo?(jmercado)
(In reply to Randy Lin [:rlin] from comment #23) > Yes, fallback to normal channel I think can solve this problem. > But why loop need to use this channel? Because the webrtc calls needs to be treated as a GSM/CDMA phone call.
(In reply to StevenLee[:slee] from comment #24) > After discussing with Randy and Star, the cause should be resulted from the > "wrong audiochannel type". Why?, to use the telephony audio channel is a must. As I commented the webrtc calls needs to be treated as a GSM/CDMA phone call. Moreover webrtc calls can interrupt GSM/CDMA calls and the mechanism for notifying these interruption is based in the use of the telephony channel in both apps (loop and callscreen apps) > loop registers the audiochannel as telephony, but > gUM create the audio resource as normal. When user tries to change audio > volume, audio channel manager only modifies "telephony" channel. So users > cannot change the volume. Mmm, the audio/video stream we get through gUM is played on a audio/video element whose `mozAudioChannelType` is set to 'telephony'. User could change the volume. > So we could add one more channel type, ex voipchannel, and it has the same > priority as telephony. Or when creating microphone resource in gUM, we > should set the resource as telephony type. Then audio channel manager can > change the volume of correct audio. Too late and too complex IHMO.
(In reply to Randell Jesup [:jesup] from comment #25) > I know there was extensive discussion of the exact AudioChannel usage to get > the desired ringing/override behaviors, but I wasn't involved in these. Yes, actually It was quite a pain. There is a must which is to treat the webrtc call as a GSM/CDMA so that's the reason we are using the telephony audio channel. > As for getUserMedia, it generally has no idea what the stream will be used > for (and does nothing to set the type to anything). The app gets the stream from gUM and plays it by using an audio/video element whose `mozAudioChannelType` is set to 'telephony' and that's it. I'm a little lost of the root cause of the audio being very low.
Flags: needinfo?(josea.olivera)
(In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #10) > Jayme, Can you see if this bug (low audio with an inability to adjust the > volume) appears when using other audio apps or audio+video apps? The user can adjust the volume with the buttons as it does for GSM/CDMA calls. (In reply to mbarone (PTO 18/08 - 30/08) from comment #0) > During a call between two Fire E or between Flame and Fire E, we have > noticed that the audio volume is very low. Also there is no way to > increase/decrease the volume. I guess Massimo meant that even adjusting the volume through the buttons the audio was still low.
blocking-b2g: 2.0? → 2.0+
the Loop application consumes audio channels - but doesn't own them. We'd like to talk to someone with the knowledge on audio channel for advisement if it's possible to work around this in the app or if a FxOS code change is needed. Jose Antonio (comment 32) would be a likely good contact for the application side on the best approach, based on the FxOS audio channel recommendations. Need info to Kevin Hu: Can you help us find an engineer to talk about this?
Flags: needinfo?(khu)
This bug took much time back and forth. Steven, my understanding is that your team(maybe Randy) may be the best one to drive the discussion. Can you host a meeting with related engineers and QAs and come out a solution or work around soon?
Flags: needinfo?(khu) → needinfo?(slee)
In the latest 2.0 and loop builds on the v165 KK base, the audio is at a level that is reasonable. The audio level can be changed in between calls and uses the "Ringer and Notification" volume setting. During a call the volume controls change to modifying the telephone volume, preventing the call audio from changing. Will the requested logs still be useful knowing this? Environmental Variables: Device: Flame 2.0 Build ID: 20140821030000 Gaia: ed257456c512c3b345f5a89e9211581e7ae0f1c0 Gecko: 6329352ca531b977979451e77e5862af485388b2 Version: 32.0 (2.0) Firmware Version: 165 User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0 Loop version: 8c71431
Flags: needinfo?(jmercado)
Hi jaoo and JMercado, I want to clarify the problems first. As I know there are 2 problems, 1. audio level too low 2. volumes cannot be modified. From comment 36, #2 does not exist. But we still have #1 problem, is it right? I downloaded loop client yesterday, but I cannot run. It told me that "The verification server is unavailable at this time....". There is no way to reproduce the bug here. So we requested for the logs. If we got things clear and have enough information, it's easier for us to plan the next step. Thanks.
Flags: needinfo?(slee)
Flags: needinfo?(josea.olivera)
Flags: needinfo?(jmercado)
(In reply to StevenLee[:slee] from comment #37) > I want to clarify the problems first. As I know there are 2 problems, > 1. audio level too low IMHO the volume is acceptable but lets consider the volume is low as QA reported. > 2. volumes cannot be modified. That's wrong. I can handle the volume through the HW buttons. > I downloaded loop client yesterday, but I cannot run. It told me that "The > verification server is unavailable at this time....". There is no way to > reproduce the bug here. So we requested for the logs. You can log into the app with the phone number and with FxA. I just gave a try and was able to log into. The error you show might happen, but it is not permanent and I you fail logging into the app with the phone number try with FxA.
Flags: needinfo?(josea.olivera)
In this other bug https://bugzilla.mozilla.org/show_bug.cgi?id=1057263 we are forcing the library to use the 100% of the volume available (the one you can control with the HW buttons) due to sometimes it was 50% by default. I dont know if this could help here, but it would be great if somebody from QA could test it as it is merged now. Thanks!
(In reply to José Antonio Olivera Ortega [:jaoo] from comment #38) > (In reply to StevenLee[:slee] from comment #37) > > 2. volumes cannot be modified. > > That's wrong. I can handle the volume through the HW buttons. Let me rewrite this. I can handle the volume through the HW buttons but actually the volume doesn't change. I wondering if the volume of the remote party must be controlled through [1]. [1] https://tokbox.com/opentok/libraries/client/js/reference/Subscriber.html#setAudioVolume
Finally, I got loop client works by using FxA(phone number still not work). But it seems that the server is down. I cannot call another device now. The audio level is acceptable to me. So that what's the "acceptable" audio level? Who can decide it? Here is my environment. Firmware version: 165 gecko: 2.0 from hg, 203686:2092ac87eceb loop: d911c572086beac5bab7ee268a60b59e26249f61 (In reply to José Antonio Olivera Ortega [:jaoo] from comment #40) > Let me rewrite this. I can handle the volume through the HW buttons but > actually the volume doesn't change. I wondering if the volume of the remote > party must be controlled through [1]. > > [1] > https://tokbox.com/opentok/libraries/client/js/reference/Subscriber. > html#setAudioVolume On my device, we found that the audio volume changes but the difference is very little. Star is tracing the coding. His debug log shows that we did change the audio sound level. He will do some more tests when loop app can call other clients.
Flags: needinfo?(mreavy)
E/compress_voip( 202): [star] volume 20 V/compress_voip( 202): voip_set_volume: Setting voip volume index: 1 V/compress_voip( 202): voip_set_volume: exit E/compress_voip( 202): [star] volume 40 V/compress_voip( 202): voip_set_volume: Setting voip volume index: 2 V/compress_voip( 202): voip_set_volume: exit E/compress_voip( 202): [star] volume 60 V/compress_voip( 202): voip_set_volume: Setting voip volume index: 3 V/compress_voip( 202): voip_set_volume: exit E/compress_voip( 202): [star] volume 80 V/compress_voip( 202): voip_set_volume: Setting voip volume index: 4 V/compress_voip( 202): voip_set_volume: exit E/compress_voip( 202): [star] volume 100 V/compress_voip( 202): voip_set_volume: Setting voip volume index: 5 V/compress_voip( 202): voip_set_volume: exit I tried to add log in compress_voip.c and get the above information about setting volume. According to the log, the volume index is set to HAL but the audio volume is not changed. Hi, Michael Can you help to clarify why we can't set the volume to VOIP call? (The loop app sets the phone state as "AUDIO_MODE_IN_COMMUNICATION")
Flags: needinfo?(mvines)
Flags: needinfo?(mvines) → needinfo?(jaywang)
(In reply to StevenLee[:slee] from comment #37) > Hi jaoo and JMercado, > > I want to clarify the problems first. As I know there are 2 problems, > 1. audio level too low > 2. volumes cannot be modified. > > From comment 36, #2 does not exist. But we still have #1 problem, is it > right? > > I downloaded loop client yesterday, but I cannot run. It told me that "The > verification server is unavailable at this time....". There is no way to > reproduce the bug here. So we requested for the logs. If we got things clear > and have enough information, it's easier for us to plan the next step. > Thanks. In he latest build 1 no longer seems to be corect, and 2 is still correct during a call. When not in a call the volume that affects this can be modified.
Flags: needinfo?(jmercado)
Hi Steven -- Can you (or someone on this bug) confirm that I understand this correctly? If the call starts out with the ringer volume set too low, then the audio for call will be low for the entire call(probably too low for a reasonable call), but if the user sets the ringer volume to high/max, the audio is fine (sufficiently loud). If this is the case, then this bug is about being able to adjust the audio volume during a call. Related question: If the user adjusts the audio volume during a call, is he also changing the ringer volume on the phone? While I don't think that's a blocker issue for v2.0, it could be annoying to a user -- and certainly something we'd want to fix for v2.1 IMO.
Flags: needinfo?(mreavy)
(In reply to Star Cheng [:scheng] from comment #42) > > E/compress_voip( 202): [star] volume 20 > V/compress_voip( 202): voip_set_volume: Setting voip volume index: 1 > V/compress_voip( 202): voip_set_volume: exit > > E/compress_voip( 202): [star] volume 40 > V/compress_voip( 202): voip_set_volume: Setting voip volume index: 2 > V/compress_voip( 202): voip_set_volume: exit > > E/compress_voip( 202): [star] volume 60 > V/compress_voip( 202): voip_set_volume: Setting voip volume index: 3 > V/compress_voip( 202): voip_set_volume: exit > > E/compress_voip( 202): [star] volume 80 > V/compress_voip( 202): voip_set_volume: Setting voip volume index: 4 > V/compress_voip( 202): voip_set_volume: exit > > E/compress_voip( 202): [star] volume 100 > V/compress_voip( 202): voip_set_volume: Setting voip volume index: 5 > V/compress_voip( 202): voip_set_volume: exit > > > I tried to add log in compress_voip.c and get the above information about > setting volume. According to the log, the volume index is set to HAL but the > audio volume is not changed. > > Hi, Michael > > Can you help to clarify why we can't set the volume to VOIP call? (The loop > app sets the phone state as "AUDIO_MODE_IN_COMMUNICATION") VoIP call can be configured in two different way. One is going though the same path as Voice and the other one is going through the audio path. AudioTrack::set() makes the decision based on the stream parameters. Voice path is used typically for the compressed voice data such as AMR and etc. Also, it is limited to 8K and 16K sample rate. compress_voip is VOIP HAL layer. For webRTC use-case, it is using audio path so the volume should be applied through SW mixer not through the compress_voip. In other words, the stream volume should be used instead of voice volume. But I did notice that the AudioPolicyManager sets both stream volume and voice volume. To further debugging the issue, you should track if the stream volume is passed properly to the SW mixer or not. Please, let me know if more information is needed Jay
Flags: needinfo?(jaywang)
Star, please correct me if I am wrong. (In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #44) > Hi Steven -- Can you (or someone on this bug) confirm that I understand this > correctly? If the call starts out with the ringer volume set too low, then > the audio for call will be low for the entire call(probably too low for a > reasonable call), but if the user sets the ringer volume to high/max, the > audio is fine (sufficiently loud). ringer volume and voice volume use different path/channel. Adjusting a channel should not effect other channel's settings. > > If this is the case, then this bug is about being able to adjust the audio > volume during a call. Yes, that's the existing problem. > > Related question: If the user adjusts the audio volume during a call, is he > also changing the ringer volume on the phone? While I don't think that's a No, when you adjust volume during a call, you change voice volume only.
(In reply to Jay from comment #45) > (In reply to Star Cheng [:scheng] from comment #42) > > VoIP call can be configured in two different way. One is going though the > same path as Voice and the other one is going through the audio path. > AudioTrack::set() makes the decision based on the stream parameters. Voice > path is used typically for the compressed voice data such as AMR and etc. > Also, it is limited to 8K and 16K sample rate. compress_voip is VOIP HAL > layer. > For webRTC use-case, it is using audio path so the volume should be applied > through SW mixer not through the compress_voip. In other words, the stream > volume should be used instead of voice volume. > Hi, Jay I have checked AudioTrack::set(), and webRTC using voip audio patch indeed. And there are 2 questions: 1) How to apply the volume for voip audio & voip voice path? voip audio: SW mixer + compress_voip ? voip voice: only compress_voip ? Is it correct? If it is right, the volume of voip audio should be applied through compress_voip. 2) what is the setting of skype app? Does it use audio voip path? > But I did notice that the AudioPolicyManager sets both stream volume and > voice volume. To further debugging the issue, you should track if the stream > volume is passed properly to the SW mixer or not. > > Please, let me know if more information is needed > > Jay
Flags: needinfo?(jaywang)
(In reply to Star Cheng [:scheng] from comment #47) > (In reply to Jay from comment #45) > > (In reply to Star Cheng [:scheng] from comment #42) > > > > VoIP call can be configured in two different way. One is going though the > > same path as Voice and the other one is going through the audio path. > > AudioTrack::set() makes the decision based on the stream parameters. Voice > > path is used typically for the compressed voice data such as AMR and etc. > > Also, it is limited to 8K and 16K sample rate. compress_voip is VOIP HAL > > layer. > > For webRTC use-case, it is using audio path so the volume should be applied > > through SW mixer not through the compress_voip. In other words, the stream > > volume should be used instead of voice volume. > > > > Hi, Jay > > I have checked AudioTrack::set(), and webRTC using voip audio patch indeed. Hi Star, Did you confirm the following if statement is true for webRTC use-case? I checked earlier the webRTC sample rate is 48000 so it should fail here and go with audio stream. if ((streamType == AUDIO_STREAM_VOICE_CALL) && (channelCount == 1) && (sampleRate == 8000 || sampleRate == 16000)) { > > And there are 2 questions: > 1) How to apply the volume for voip audio & voip voice path? > > voip audio: SW mixer + compress_voip ? > voip voice: only compress_voip ? > > Is it correct? If it is right, the volume of voip audio should be applied > through compress_voip. > voip audio: should be only applied with SW mixer as it is opened as an audio stream. Currently, the audio policy manager generates sends the volume to both SW mixer and compress_voip, but compress_voip HAL should reject it. voip voice: only controls through compress_voip. > 2) what is the setting of skype app? Does it use audio voip path? > > > But I did notice that the AudioPolicyManager sets both stream volume and > > voice volume. To further debugging the issue, you should track if the stream > > volume is passed properly to the SW mixer or not. > > > > Please, let me know if more information is needed > > > > Jay The key here is if it uses higher sample rate or not. Any voIP app has the samplerate >16KHz will use audio path.
Flags: needinfo?(jaywang)
(In reply to Jay from comment #48) > (In reply to Star Cheng [:scheng] from comment #47) > > (In reply to Jay from comment #45) > > > (In reply to Star Cheng [:scheng] from comment #42) > > > > > > VoIP call can be configured in two different way. One is going though the > > > same path as Voice and the other one is going through the audio path. > > > AudioTrack::set() makes the decision based on the stream parameters. Voice > > > path is used typically for the compressed voice data such as AMR and etc. > > > Also, it is limited to 8K and 16K sample rate. compress_voip is VOIP HAL > > > layer. > > > For webRTC use-case, it is using audio path so the volume should be applied > > > through SW mixer not through the compress_voip. In other words, the stream > > > volume should be used instead of voice volume. > > > > > > > Hi, Jay > > > > I have checked AudioTrack::set(), and webRTC using voip audio patch indeed. > Hi Star, > Did you confirm the following if statement is true for webRTC use-case? I > checked earlier the webRTC sample rate is 48000 so it should fail here and > go with audio stream. > if ((streamType == AUDIO_STREAM_VOICE_CALL) && > (channelCount == 1) && > (sampleRate == 8000 || sampleRate == 16000)) { > Yes, you are right. WebRTC go with audio stream because the stream type is "AUDIO_STREAM_SYSTEM" and it's samplte rate is 48K. > > > > And there are 2 questions: > > 1) How to apply the volume for voip audio & voip voice path? > > > > voip audio: SW mixer + compress_voip ? > > voip voice: only compress_voip ? > > > > Is it correct? If it is right, the volume of voip audio should be applied > > through compress_voip. > > > voip audio: should be only applied with SW mixer as it is opened as an audio > stream. Currently, the audio policy manager generates sends the volume to > both SW mixer and compress_voip, but compress_voip HAL should reject it. > voip voice: only controls through compress_voip. > Jay, thanks for your response. And another thing I want to confirm with you if there are different gain value between voip voice and voip audio? Because the mixer_ctl_name in the voip_set_volume() function, there is only "Voip Rx Gain". I also saw the log in the Comment 42 and it set the correct volume for the mixer file description(FD) through ioctl() system call. AudioTrack::set() makes the decision the pcm data is from Mulit-Media path or "use.voice.path.for.pcm.voip" path, and the voip sound card mixer FD should be created whatever voip voice orvoip audio. If voip voice and voip audio both create the same mixer file description to control volume. The volume should be applied by through compress_voip to change the gain value, not reject by compress_voip HAL instead. > > 2) what is the setting of skype app? Does it use audio voip path? > > > > > But I did notice that the AudioPolicyManager sets both stream volume and > > > voice volume. To further debugging the issue, you should track if the stream > > > volume is passed properly to the SW mixer or not. > > > > > > Please, let me know if more information is needed > > > > > > Jay > The key here is if it uses higher sample rate or not. Any voIP app has the > samplerate >16KHz will use audio path.
Flags: needinfo?(jaywang)
(In reply to Star Cheng [:scheng] from comment #49) > > > Hi, Jay > > > I have checked AudioTrack::set(), and webRTC using voip audio patch indeed. > > Hi Star, > > Did you confirm the following if statement is true for webRTC use-case? I > > checked earlier the webRTC sample rate is 48000 so it should fail here and > > go with audio stream. > > if ((streamType == AUDIO_STREAM_VOICE_CALL) && > > (channelCount == 1) && > > (sampleRate == 8000 || sampleRate == 16000)) { > > > > Yes, you are right. WebRTC go with audio stream because the stream type is > "AUDIO_STREAM_SYSTEM" and it's samplte rate is 48K. > Sorry! The stream type is "AUDIO_STREAM_VOICE_CALL" instead of "AUDIO_STREAM_SYSTEM".
(In reply to Jay from comment #45) > But I did notice that the AudioPolicyManager sets both stream volume and > voice volume. To further debugging the issue, you should track if the stream > volume is passed properly to the SW mixer or not. > > Please, let me know if more information is needed > > Jay Hi, Jay Because AudioPolicyManager is set both stream volume and voice volume, the volume should be applied whatever SW mixer or compress voip HAL. Could you help to check this? Thanks
It looks like we need vendor's support. Assign the bug to Jay first.
Assignee: nobody → jaywang
Summary: [Loop] In a call the audio is very low → [Loop] In a call users cannot change volume
Whiteboard: ft:loop → ft:loop [blocking]
I can help to check if SW mixer received the stream volume setting or not. Also, can you point me to the code in loop app and webrtc stack that constructs the audio track and sets the volume.
Flags: needinfo?(jaywang) → needinfo?(scheng)
(In reply to Jay from comment #53) > I can help to check if SW mixer received the stream volume setting or not. > Also, can you point me to the code in loop app and webrtc stack that > constructs the audio track and sets the volume. Hi, Maire I am not familiar with loop app and webRTC. Can you help to answer Jay's questions? Thanks
Flags: needinfo?(scheng) → needinfo?(mreavy)
Input is handled via opensles (media/webrtc/trunk/webrtc/modules/audio_device/android/opensles_input.cc) (16KHz currently, though we may raise it at some point to 32 or 48, though perhaps not on mobile). Output is handled via cubeb (media/libcubeb/) and runs at the "ideal" rate of the output stream (usually 44.1 or 48K). Beyond that, it's a bit out of my area; I'm not exactly sure how the B2G audio channels are influenced by the app or how they're assigned.
Flags: needinfo?(mreavy) → needinfo?(rlin)
(In reply to Randell Jesup [:jesup] from comment #55) > Beyond that, it's a bit out of my area; I'm not exactly sure how the B2G > audio channels are influenced by the app or how they're assigned. Hi Jay, rlin is in PTO. For audio channel stuff, you can ask Star.
Flags: needinfo?(rlin)
(In reply to Star Cheng [:scheng] from comment #51) > (In reply to Jay from comment #45) > > But I did notice that the AudioPolicyManager sets both stream volume and > > voice volume. To further debugging the issue, you should track if the stream > > volume is passed properly to the SW mixer or not. > > > > Please, let me know if more information is needed > > > > Jay > > Hi, Jay > > > Because AudioPolicyManager is set both stream volume and voice volume, the > volume should be applied whatever SW mixer or compress voip HAL. Could you > help to check this? > > Thanks Here is what I found: #1 the voice volume is actually dropped by voice kernel driver not by compress_voip HAL. Earlier, I just looked the tinymixer output from adb shell and didn't see the voice gain mixer was set that's why I thought it is dropped by HAL. However, the voice gain mixer control doesn't provide the interface to retrieve the current setting that is why it didn't give the gain value. Anyway, in summary, voice volume is dropped in kernel driver when audio path is used for VoIP. #2 The Audio Mixer is getting stream volume from AudioPolicyManager so there is no issue in propagating the stream volume. #3 I see that multiple AudioTracks were created during webRTC session. 1 voice stream and later 1 system stream is created. Both streams are active during the call. #4 It looks like the audio data is going through the system stream track instead of voice stream track. When I forced mute the system stream, it actually mutes the voice call. You can do this in AudioFlinger::setStreamVolume() by adding if (stream == AUDIO_STREAM_SYSTEM) value = 0.0; So, someone needs to look at from loopmobile or webrtc stack see why the system stream is used to pass the voice data. Star, I am re-assigning this to you.
Assignee: jaywang → scheng
Hi, Maire According to #4 item in the comment 57 which Jay mentioned, I need help form loop app/webRTC expert to confirm which kind of stream type is used for? I wanna check whether voice stream type or system stream type. Thanks
Flags: needinfo?(mreavy)
> Here is what I found: > #1 the voice volume is actually dropped by voice kernel driver not by > compress_voip HAL. Earlier, I just looked the tinymixer output from adb > shell and didn't see the voice gain mixer was set that's why I thought it is > dropped by HAL. However, the voice gain mixer control doesn't provide the > interface to retrieve the current setting that is why it didn't give the > gain value. Anyway, in summary, voice volume is dropped in kernel driver > when audio path is used for VoIP. > Could you provide more detail explain to "dropped by voice kernel driver"? Do you mean: except "CS voice call" and "VOIP voice call", other cases (VOIP auido, music stream, system stream ...) we can't control voice gain mixer in driver layer? If it's yes, why there is *not* different setting for VOIP voice and VOIP audio in voice_set_volume() @qcom/audio/hal/Voice.c. There is only different setting base on the mode is AUDIO_MODE_IN_CALL or AUDIO_MODE_IN_COMMUNICATION? How to distinguish them(voip voice & voip audio)? > #4 It looks like the audio data is going through the system stream track > instead of voice stream track. When I forced mute the system stream, it > actually mutes the voice call. You can do this in > AudioFlinger::setStreamVolume() by adding > > if (stream == AUDIO_STREAM_SYSTEM) > value = 0.0; > And we test Skype app in Android phone (Nexus5) and get the dumpsys log. The voice stream and COMMUNICATION mode are used for Skype app. The sample rate is 48K, so AudioTrack::set() will determine the stream as VOIP audio. How does Skype app change audio volume? (ps. the volume level is 5) According to #1, the voice volume is drooped when audio path is used for VoIP. Even if we set correct stream type, it is determined to VOIP audio by AudioTrack::set(). How can we change the volume?
Flags: needinfo?(jaywang)
(In reply to Star Cheng [:scheng] from comment #59) > > Here is what I found: > > #1 the voice volume is actually dropped by voice kernel driver not by > > compress_voip HAL. Earlier, I just looked the tinymixer output from adb > > shell and didn't see the voice gain mixer was set that's why I thought it is > > dropped by HAL. However, the voice gain mixer control doesn't provide the > > interface to retrieve the current setting that is why it didn't give the > > gain value. Anyway, in summary, voice volume is dropped in kernel driver > > when audio path is used for VoIP. > > > > Could you provide more detail explain to "dropped by voice kernel driver"? Voice driver doesn't sent voice gain request to the DSP. As DSP doesn't have VoIP session opened on voice path. > Do you mean: except "CS voice call" and "VOIP voice call", other cases (VOIP > auido, music stream, system stream ...) we can't control voice gain mixer in > driver layer? If it's yes, why there is *not* different setting for VOIP > voice and VOIP audio in voice_set_volume() @qcom/audio/hal/Voice.c. There is > only different setting base on the mode is AUDIO_MODE_IN_CALL or > AUDIO_MODE_IN_COMMUNICATION? How to distinguish them(voip voice & voip > audio)? The volume control for other cases happens in the SW AudioMixer. AudioMixer mixes multiple stream together depending on the stream's volume preferences. It is different than the Voice or VoIP over voice path which doesn't get mixed in AudioMixer, but directly feed to DSP. That's why the volume is controlled through voice driver. > > > > > #4 It looks like the audio data is going through the system stream track > > instead of voice stream track. When I forced mute the system stream, it > > actually mutes the voice call. You can do this in > > AudioFlinger::setStreamVolume() by adding > > > > if (stream == AUDIO_STREAM_SYSTEM) > > value = 0.0; > > > > And we test Skype app in Android phone (Nexus5) and get the dumpsys log. The > voice stream and COMMUNICATION mode are used for Skype app. The sample rate > is 48K, so AudioTrack::set() will determine the stream as VOIP audio. How > does Skype app change audio volume? (ps. the volume level is 5) > > According to #1, the voice volume is drooped when audio path is used for > VoIP. Even if we set correct stream type, it is determined to VOIP audio by > AudioTrack::set(). How can we change the volume? As I mentioned earlier, the AudioPolicyManager generates the set volume request for both voice and audio stream. So, it uses AudioFlinger::setStreamVolume() to set the stream volume. You can add the print here to confirm.
Flags: needinfo?(jaywang)
(In reply to Star Cheng [:scheng] from comment #58) > Hi, Maire > > According to #4 item in the comment 57 which Jay mentioned, I need help form > loop app/webRTC expert to confirm which kind of stream type is used for? I > wanna check whether voice stream type or system stream type. > > Thanks Hi Star, I and my team are happy to help. Randell and I were just looking over the code and trying to figure out the best folks to get you the answers you need. Randell will respond very soon with what we found and what we know. This area of the code is actually outside of WebRTC and lower level than the code we typically work with. I believe that Jose Antonio worked with Andrea and Vivien when sorting through the audio channel requirements for the Loop App. Vivien is already needinfo'd. I'll add Andrea to see if he can help us get to the bottom of this. When Randell responds soon, he'll add Jose Antonio as well as a few other folks who we think can help. I think part of the issue is that this problem almost falls between groups -- between the Loop app group, WebRTC group and the audio channel group. Thanks for your help in tracking this down!
Flags: needinfo?(mreavy) → needinfo?(amarchesini)
(In reply to Star Cheng [:scheng] from comment #50) > (In reply to Star Cheng [:scheng] from comment #49) > > > > > Hi, Jay > > > > I have checked AudioTrack::set(), and webRTC using voip audio patch indeed. > > > Hi Star, > > > Did you confirm the following if statement is true for webRTC use-case? I > > > checked earlier the webRTC sample rate is 48000 so it should fail here and > > > go with audio stream. > > > if ((streamType == AUDIO_STREAM_VOICE_CALL) && > > > (channelCount == 1) && > > > (sampleRate == 8000 || sampleRate == 16000)) { > > > > > > > Yes, you are right. WebRTC go with audio stream because the stream type is > > "AUDIO_STREAM_SYSTEM" and it's samplte rate is 48K. > > > > Sorry! The stream type is "AUDIO_STREAM_VOICE_CALL" instead of > "AUDIO_STREAM_SYSTEM". The audio output from WebRTC goes via cubeb. I'm fairly certain B2G32 has the "run MediaStreamGraph at ideal output frequency" (44.1 or 48K), so I think that test (in AudioTrack.cpp) will fail. I note this in AudioManager.cpp: case AudioChannel::Telephony: status = SetStreamVolumeIndex(AUDIO_STREAM_VOICE_CALL, aIndex); in AudioManager::SetAudioChannelVolume() I also see in AudioFlinger::PlaybackThread::Track::Track(): ((audio_stream_type_t)streamType == AUDIO_STREAM_VOICE_CALL)? VOICE_COMMUNICATION_FLAG:0x0, The loop client sets the <video> element type to 'telephony' at libs/tokbox/v2.2.6/js/TB.js:4243. It also uses js/helpers/audio_competing_helper.js to monitor for other apps taking over the audio, I believe. Jose Antonio, am I correct? I'm just looking around; I'm not an expert on the Audio Stream controls/volume/etc for B2G; since rlin isn't around, perhaps jlin?
Flags: needinfo?(rlin)
Flags: needinfo?(josea.olivera)
Flags: needinfo?(jlin)
you're looking for jolin
Flags: needinfo?(jlin) → needinfo?(jolin)
(In reply to Jay from comment #60) > The volume control for other cases happens in the SW AudioMixer. AudioMixer > mixes multiple stream together depending on the stream's volume preferences. > It is different than the Voice or VoIP over voice path which doesn't get > mixed in AudioMixer, but directly feed to DSP. That's why the volume is > controlled through voice driver. Hi, Jay I try to summary this discussion. If there is any problem, please point it out! use case stream type setPhoneState mode set volume [1]CS Voice : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_CALL | (setStreamVolume) + setVoiceVolume [2]VOIP Voice : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | (setStreamVolume) + setVoiceVolume [3]VOIP Audio : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | setStreamVolume + (setVoiceVolume) [4]Audio : AUDIO_STREAM_MUSIC | AUDIO_MODE_NORMAL | setStreamVolume [5]Loop app : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | setStreamVolume + (setVoiceVolume) [6]Skype app : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | setStreamVolume + (setVoiceVolume) setStreamVolume -> out_set_volume() setVoiceVolume -> voice_set_volume -> (mode == AUDIO_MODE_IN_CALL) ? platform_set_voice_volume() : voice_extn_compress_voip_set_volume() Case[1]&[2], there is *not* real PCM data to AudioFlinger & AudioMixer, it don't apply the volume change through setStreamVolume() function. Case[3], [5], and[6], because of that's *not* real CS call or PS call, voice volume is dropped in kernel driver. Case[4] is not "AUDIO_STREAM_VOICE_CALL" or "AUDIO_STREAM_BLUETOOTH_SCO" stream type, so it don't invoke setVoiceVolume() function in AudioPolicyManager::checkAndSetVolume().
Flags: needinfo?(jaywang)
Luckily rlin is back from PTO already and is much more capable than me in those audio code. :-)
Flags: needinfo?(jolin)
(In reply to Randell Jesup [:jesup] from comment #62) > The loop client sets the <video> element type to 'telephony' at > libs/tokbox/v2.2.6/js/TB.js:4243. > It also uses js/helpers/audio_competing_helper.js to monitor for other apps > taking over the audio, I believe. Jose Antonio, am I correct? Yes, you are. Actually the audio is played through that video element. The AudioCompetingHelper object is a helper for implementing the audio competing policy we defined when using the telephony audio channel. FYI AudioCompetingHelper object plays silence through an AudioContext object.
Flags: needinfo?(josea.olivera)
Hi, To provide what I found and hope it is useful. 1. As I know that Loop app will have a MediaStream to represent local & remote video/audio. 2. In order to play remote video/audio, loop app puts one MediaStream into video tag. 3. That video tag is assigned to "telephony" channel type. 4. According to source from mediastream, HTMLMediaElement play it through MSG not MediaDecodeStateMachine. 5. The audio sink used by MediaDecoderStateMachine will set audio channel type into audiostream. ## But I didn't find who help to set audio channel type into mediastream. (In case of WebAudio, AudioDestinationNode will help to do this.) Maybe that is the root cause.
Flags: needinfo?(amarchesini)
OS: Mac OS X → All
Hardware: x86 → All
Attached patch audio.patchSplinter Review
Attachment #8482330 - Flags: review?(roc)
Ops, sorry scheng, I didn't see this bug was assigned to you. Marco, good catch! I think this patch fixes the issue assigned the AudioChannel to the MediaStream when this is initialized and when the user does mediaElement.mozaudiochanel = 'something'.
https://tbpl.mozilla.org/?tree=Try&rev=89fff1631067 So, this patch has to land because it fixes a bug. Would be nice if someone can test if it fixes this particular issue. I land the patch if green on try.
Assignee: scheng → nobody
Andrea -- Since this bug is 2.0+ and you created the patch up for review, can you take this bug?
Flags: needinfo?(amarchesini)
(In reply to Andrea Marchesini (:baku) from comment #69) > Ops, sorry scheng, I didn't see this bug was assigned to you. Marco, good > catch! > I think this patch fixes the issue assigned the AudioChannel to the > MediaStream when this is initialized and when the user does > mediaElement.mozaudiochanel = 'something'. I reset the owner as default. Thanks for your great help.
Hi José Antonio & Maria -- Can you or someone on your team try Andrea's patch and see if it solves the problem you're seeing? Thanks. Star, Marco, & Andrea -- Many thanks for your help in analyzing this problem and putting together a solution!
Flags: needinfo?(oteo)
Flags: needinfo?(josea.olivera)
Assignee: nobody → amarchesini
Flags: needinfo?(amarchesini)
(In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #73) > Hi José Antonio & Maria -- Can you or someone on your team try Andrea's > patch and see if it solves the problem you're seeing? Thanks. I told Andrea I'll give a try but I couldn't do it yesterday. I'm on it right now.
Flags: needinfo?(josea.olivera)
Flags: needinfo?(oteo)
(In reply to José Antonio Olivera Ortega [:jaoo] from comment #75) > (In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #73) > > Hi José Antonio & Maria -- Can you or someone on your team try Andrea's > > patch and see if it solves the problem you're seeing? Thanks. > > I told Andrea I'll give a try but I couldn't do it yesterday. I'm on it > right now. Sorry for the lag. It seems the volume increases/decreases correctly. This is my perception so I'd like other to confirm that. Massimo, Jay?
Flags: needinfo?(mbarone976)
(In reply to Star Cheng [:scheng] from comment #64) > (In reply to Jay from comment #60) > > The volume control for other cases happens in the SW AudioMixer. AudioMixer > > mixes multiple stream together depending on the stream's volume preferences. > > It is different than the Voice or VoIP over voice path which doesn't get > > mixed in AudioMixer, but directly feed to DSP. That's why the volume is > > controlled through voice driver. > > Hi, Jay > > I try to summary this discussion. If there is any problem, please point it > out! > > use case stream type setPhoneState mode > set volume > [1]CS Voice : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_CALL | > (setStreamVolume) + setVoiceVolume > [2]VOIP Voice : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | > (setStreamVolume) + setVoiceVolume > [3]VOIP Audio : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | > setStreamVolume + (setVoiceVolume) > [4]Audio : AUDIO_STREAM_MUSIC | AUDIO_MODE_NORMAL | > setStreamVolume > [5]Loop app : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | > setStreamVolume + (setVoiceVolume) > [6]Skype app : AUDIO_STREAM_VOICE_CALL | AUDIO_MODE_IN_COMMUNICATION | > setStreamVolume + (setVoiceVolume) > > setStreamVolume -> out_set_volume() > setVoiceVolume -> voice_set_volume -> (mode == AUDIO_MODE_IN_CALL) ? > platform_set_voice_volume() : voice_extn_compress_voip_set_volume() > > Case[1]&[2], there is *not* real PCM data to AudioFlinger & AudioMixer, it > don't apply the volume change through setStreamVolume() function. > Case[3], [5], and[6], because of that's *not* real CS call or PS call, voice > volume is dropped in kernel driver. Just add one thing here to clarify that the volume will be set by setStreamVolume in AudioMixer. The rest looks fine. > Case[4] is not "AUDIO_STREAM_VOICE_CALL" or "AUDIO_STREAM_BLUETOOTH_SCO" > stream type, so it don't invoke setVoiceVolume() function in > AudioPolicyManager::checkAndSetVolume().
Flags: needinfo?(jaywang)
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Note that due to recent policy changes, all B2G uplifts needs approval now regardless of blocking status. Please request b2g32 and aurora approval on this patch when you get a chance. Sorry for the inconvenience.
Flags: needinfo?(amarchesini)
Target Milestone: --- → 2.1 S4 (12sep)
I try to flash v166 flame image and replace gaia/gecko. Test this patch: https://bugzilla.mozilla.org/attachment.cgi?id=8482330 The volume can be adjusted, but reaction is slow.
Flags: needinfo?(rlin)
Comment on attachment 8482330 [details] [diff] [review] audio.patch Approval Request Comment [Feature/regressing bug #]: AudioChannel in HTMLMediaElements. Not a particular bug ID. [User impact if declined]: the volume cannot be changed in a loop call and the default value is too low. [Describe test coverage new/current, TBPL]: tested on device. [Risks and why]: none, as far I can see. [String/UUID change made/needed]: none NOTE: This flag is now for security issues only. Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings. [Approval Request Comment] Bug caused by (feature/regressing bug #): AudioChannel in HTMLMediaElements. Not a particular bug ID. User impact if declined: the volume cannot be changed in a loop call and the default value is too low. Testing completed: on device Risk to taking this patch (and alternatives if risky): none, as far I can see. String or UUID changes made by this patch: none
Attachment #8482330 - Flags: approval-mozilla-b2g32?
Attachment #8482330 - Flags: approval-mozilla-aurora?
Flags: needinfo?(amarchesini)
Component: Gaia::Loop → Video/Audio
Product: Firefox OS → Core
Attachment #8482330 - Flags: approval-mozilla-b2g32?
Attachment #8482330 - Flags: approval-mozilla-b2g32+
Attachment #8482330 - Flags: approval-mozilla-aurora?
Attachment #8482330 - Flags: approval-mozilla-aurora+
Tested with Flame v2.0 (kk v180) and Loop version eb3a712 and gecko-ef26d06.gaia-ddec117 and the end user can adjust the volume correctly
Status: RESOLVED → VERIFIED
Flags: needinfo?(mbarone976)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: