1200126 - (Use TTS) Can not use Earpiece to speak SpeechSynthesisUtterance.

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: AudioChannel, defect, P1)

Description

[xiupinglong]

When using Web Speech API SpeechSynthesis(TTS), we can not use Earpiece (AudioChannel::telephony) to speak SpeechSynthesisUtterance, it only can hear from speaker (AudioChannel::normal).

It should be set to hear from speaker or earpiece.

Comment 1

[Guoqiang.CHEN]

Dear Wesly,

We try to use voice command when user dialing a call, so we want tts can play from earpiece, please help to support it.
Thanks

Comment 2

[Alastor Wu [:alwu]]

Hi, 
Does your app have the telephony permission?
To get this permission, your app must be the certified app and declares the "audio-channel-telephony" in the manifest.

Comment 3

[Wesly Huang (EPM)]

Hi Guoqiang:

Would you help confirm the question in comment#2?

BTW I assume this question is also for goflip project? Thanks.

Comment 4

[Jack Liu]

Dear Alastor,

Because TTS implementation is at lower layer in Gecko, so we want to know how to adapt audio stream to telephony channel in TTS implementation. Are there some low level APIs to call?

Thanks.

Comment 5

[Guoqiang.CHEN]

Dear Alastor,

We play tts code like this, this functions seemed no way to set AudioChannel.
var u = new SpeechSynthesisUtterance();
     u.text = 'Hello World';
     u.lang = 'en-US';
     u.rate = 1.0;
     speechSynthesis.speak(u);

Comment 6

[Alastor Wu [:alwu]]

Thanks information!
Let me check the code of that, I'll reply later.
Set NI.

Comment 7

[Alastor Wu [:alwu]]

Hi, all,

The audio output decision is managed by Android HAL, instead of FxOS. The way to use receiver as output are "we have connected call" or "create a telephony stream".

For this case, the Web Speech API uses "normal" type to create audio stream [1]. If you change it to "AudioChannel::Telephony", then the audio can be heard from the receiver.

However, I don't prefer to do such modification. That would result all Web Speech API usages coming out from the receiver. We should implement a way to send the audio channel type or this stream. 

[1] https://dxr.mozilla.org/mozilla-central/source/dom/media/webspeech/synth/nsSynthVoiceRegistry.cpp?#783

Comment 8

[xiupinglong]

(In reply to Alastor Wu [:alwu] from comment #7)
> Hi, all,
> 
> The audio output decision is managed by Android HAL, instead of FxOS. The
> way to use receiver as output are "we have connected call" or "create a
> telephony stream".
> 
> For this case, the Web Speech API uses "normal" type to create audio stream
> [1]. If you change it to "AudioChannel::Telephony", then the audio can be
> heard from the receiver.
> 
> However, I don't prefer to do such modification. That would result all Web
> Speech API usages coming out from the receiver. We should implement a way to
> send the audio channel type or this stream. 
> 
> [1]
> https://dxr.mozilla.org/mozilla-central/source/dom/media/webspeech/synth/
> nsSynthVoiceRegistry.cpp?#783

Hi Alastor Wu,
But we are now using FFOS 2.2, there is no "MediaStreamGraph::AUDIO_THREAD_DRIVER". Could you please provide the relevant patch for FFOS 2.2? Thanks!

Comment 9

[Alastor Wu [:alwu]]

To adjust volume, you need to make sure that following points are corresponded correctly.
(1) The audio stream type [1] we used
(2) Volume category [2] we adjusted 

The volume changing notification is passed using MOZ_SETTING event. In v2.2, we would listen events in these places [3], and then set the volume index for the specific stream to the Android::AudioSystem.

---

[1] http://osxr.org/android/source/system/core/include/system/audio.h#0037
[2] Media/Notification/Alarm/Telephony/BT_SCO
[3] http://mxr.mozilla.org/mozilla-b2g37_v2_2/source/dom/audiochannel/AudioChannelService.cpp#830
    http://mxr.mozilla.org/mozilla-b2g37_v2_2/source/dom/system/gonk/AudioManager.cpp#383

Comment 10

[xiupinglong]

Attachment: add-audioChannelType-for-TTS.patch

Dear Alastor,

I have uploaded the patch in the attachment, please help us to check and give some suggestion. Thanks for your help!

Comment 11

[xiupinglong]

Dear Wesly,

Please help us to find the person who is expert in this topic to check the attachment patch. Thanks!

Comment 12

[Alastor Wu [:alwu]]

Hi,
Sorry for my late reply, because I have a audio workweek this week, I can't feedback it at timely.
But, I can provide some feedback later.

Comment 13

[Wesly Huang (EPM)]

Hi Xiuping:

Please kindly understand it's in fact not so proper that we review your code which is purposed for specific product, but as Alastor replied above he'll try to give some feedback/comment after finishing his workshop this week, for your reference. Thank you.

Comment 14

[Alastor Wu [:alwu]]

Comment on attachment 8659691 [details] [diff] [review]
add-audioChannelType-for-TTS.patch

Review of attachment 8659691 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, only one comment.

::: dom/media/MediaStreamGraph.cpp
@@ +950,5 @@
>            if (mLifecycleState == LIFECYCLE_RUNNING) {
> +            MYLOG("new AudioCallbackDriver mChannel = %d", mChannel);
> +
> +            // tcl_longxiuping add "mChannel" for TTS can hear from Earpiece
> +            AudioCallbackDriver* driver = new AudioCallbackDriver(this, mChannel);

You don't need to modify the AudioCallbackDriver.

What you need to do is to change the input channel parameter to the MediaStreamGraph, this code is in the MediaStreamGraph::GetInstance().

Comment 15

[Wesly Huang (EPM)]

Thanks Alaster's time these days and comment!

And ni XiuPing for checking this.


(In reply to Alastor Wu [:alwu] from comment #14)
> Comment on attachment 8659691 [details] [diff] [review]
> add-audioChannelType-for-TTS.patch
> 
> Review of attachment 8659691 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> LGTM, only one comment.
> 
> ::: dom/media/MediaStreamGraph.cpp
> @@ +950,5 @@
> >            if (mLifecycleState == LIFECYCLE_RUNNING) {
> > +            MYLOG("new AudioCallbackDriver mChannel = %d", mChannel);
> > +
> > +            // tcl_longxiuping add "mChannel" for TTS can hear from Earpiece
> > +            AudioCallbackDriver* driver = new AudioCallbackDriver(this, mChannel);
> 
> You don't need to modify the AudioCallbackDriver.
> 
> What you need to do is to change the input channel parameter to the
> MediaStreamGraph, this code is in the MediaStreamGraph::GetInstance().

Comment 16

[xiupinglong]

(In reply to Alastor Wu [:alwu] from comment #14)
> Comment on attachment 8659691 [details] [diff] [review]
> add-audioChannelType-for-TTS.patch
> 
> Review of attachment 8659691 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> LGTM, only one comment.
> 
> ::: dom/media/MediaStreamGraph.cpp
> @@ +950,5 @@
> >            if (mLifecycleState == LIFECYCLE_RUNNING) {
> > +            MYLOG("new AudioCallbackDriver mChannel = %d", mChannel);
> > +
> > +            // tcl_longxiuping add "mChannel" for TTS can hear from Earpiece
> > +            AudioCallbackDriver* driver = new AudioCallbackDriver(this, mChannel);
> 
> You don't need to modify the AudioCallbackDriver.
> 
> What you need to do is to change the input channel parameter to the
> MediaStreamGraph, this code is in the MediaStreamGraph::GetInstance().

Hi Alaster,

Because in FFOS 2.2, if I don't use the channel parameter, the default channel is normal, other than telephony for user can hear from earpiece.

See
explicit AudioCallbackDriver(MediaStreamGraphImpl* aGraphImpl,
                             dom::AudioChannel aChannel = dom::AudioChannel::Normal);

Comment 17

[BMO Automation]

Firefox OS is not being worked on