1051604 - Adapt VAD strategy on SpeechRecognition to be less strict on some devices with poor mics

Status: RESOLVED FIXED

(Core :: Web Speech, defect)

Description

[Andre Natal]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 (Beta/Release)
Build ID: 20140715215003

Comment 1

[Andre Natal]

We should benchmark and test  pocketsphinx vad instead the current one that is leading to a number of issues.

Comment 2

[André Natal]

Attachment: Relax current VAD algorithm increasing the amount of end silence required to be input to decree end of speech

Try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=97df26316f22

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

Just set "media.webspeech.silence_length" and "media.webspeech.long_silence_length" pref to some other
value than the defaults? I don't see need for the magical 3 in the code.

Comment 4

[André Natal]

(In reply to Olli Pettay [:smaug] from comment #3)
> Just set "media.webspeech.silence_length" and
> "media.webspeech.long_silence_length" pref to some other
> value than the defaults? I don't see need for the magical 3 in the code.

I know, but I tried modify these parameters but had no practical effect since requested_silence_length changes dynamically based on the amount of speech input. 

This Vad algorithm is not good enough and I don't understand why it was pick instead https://dxr.mozilla.org/mozilla-central/source/media/webrtc/trunk/webrtc/common_audio/vad/ that is much wider adopted by the industry and is already on gecko codebase.

Comment 5

[kdavis]

(In reply to Andre Natal from comment #4)
> (In reply to Olli Pettay [:smaug] from comment #3)
> > Just set "media.webspeech.silence_length" and
> > "media.webspeech.long_silence_length" pref to some other
> > value than the defaults? I don't see need for the magical 3 in the code.
> 
> I know, but I tried modify these parameters but had no practical effect
> since requested_silence_length changes dynamically based on the amount of
> speech input. 


What if you set both "media.webspeech.silence_length" and "media.webspeech.long_silence_length"
to 3 times their current values?

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

(In reply to Andre Natal from comment #4)
> (In reply to Olli Pettay [:smaug] from comment #3)
> > Just set "media.webspeech.silence_length" and
> > "media.webspeech.long_silence_length" pref to some other
> > value than the defaults? I don't see need for the magical 3 in the code.
> 
> I know, but I tried modify these parameters but had no practical effect
> since requested_silence_length changes dynamically based on the amount of
> speech input. 
I don't understand. requested_silence_length is set to be either 
long_speech_input_complete_silence_length_us_ or speech_input_complete_silence_length_us_
and both those variables are set based on the prefs.


 
> This Vad algorithm is not good enough and I don't understand why it was pick
> instead
> https://dxr.mozilla.org/mozilla-central/source/media/webrtc/trunk/webrtc/
> common_audio/vad/ that is much wider adopted by the industry and is already
> on gecko codebase.
IIRC at the time endpointer.cc was initial needed, we didn't have any webrtc code in tree. Though, webrtc and speech API code did land around the same time.
But feel free to make speech API to use the same code as what webrtc uses.

Comment 7

[André Natal]

Attachment: Relax current VAD algorithm increasing the amount of end silence required to be input to decree end of speech

Changed the patch to set default value of the preferences: PREFERENCE_ENDPOINTER_SILENCE_LENGTH and PREFERENCE_ENDPOINTER_LONG_SILENCE_LENGTH

Comment 8

[André Natal]

(In reply to Olli Pettay [:smaug] from comment #6)
> (In reply to Andre Natal from comment #4)
> > (In reply to Olli Pettay [:smaug] from comment #3)
> > > Just set "media.webspeech.silence_length" and
> > > "media.webspeech.long_silence_length" pref to some other
> > > value than the defaults? I don't see need for the magical 3 in the code.
> > 
> > I know, but I tried modify these parameters but had no practical effect
> > since requested_silence_length changes dynamically based on the amount of
> > speech input. 
> I don't understand. requested_silence_length is set to be either 
> long_speech_input_complete_silence_length_us_ or
> speech_input_complete_silence_length_us_
> and both those variables are set based on the prefs.
> 
> 

Ok, I set there.

>  
> > This Vad algorithm is not good enough and I don't understand why it was pick
> > instead
> > https://dxr.mozilla.org/mozilla-central/source/media/webrtc/trunk/webrtc/
> > common_audio/vad/ that is much wider adopted by the industry and is already
> > on gecko codebase.
> IIRC at the time endpointer.cc was initial needed, we didn't have any webrtc
> code in tree. Though, webrtc and speech API code did land around the same
> time.
> But feel free to make speech API to use the same code as what webrtc uses.

Unfortunately we don't enough have time to do so deep change, so better keep it and change the parameters.

Comment 9

[André Natal]

(In reply to kdavis from comment #5)
> (In reply to Andre Natal from comment #4)
> > (In reply to Olli Pettay [:smaug] from comment #3)
> > > Just set "media.webspeech.silence_length" and
> > > "media.webspeech.long_silence_length" pref to some other
> > > value than the defaults? I don't see need for the magical 3 in the code.
> > 
> > I know, but I tried modify these parameters but had no practical effect
> > since requested_silence_length changes dynamically based on the amount of
> > speech input. 
> 
> 
> What if you set both "media.webspeech.silence_length" and
> "media.webspeech.long_silence_length"
> to 3 times their current values?

Thank you Kelly. Yes, I did that. I changed to 2.5 times since 3 times was taking too long to decode.

Comment 10

[André Natal]

Try for this patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=82944aa09865

Comment 11

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8644114 [details] [diff] [review]
Relax current VAD algorithm increasing the amount of end silence required to be input to decree end of speech

This is fine too, though I wonder why you don't just set the prefs on b2g.js or somewhere.

Comment 12

[André Natal]

(In reply to Olli Pettay [:smaug] from comment #11)
> Comment on attachment 8644114 [details] [diff] [review]
> Relax current VAD algorithm increasing the amount of end silence required to
> be input to decree end of speech
> 
> This is fine too, though I wonder why you don't just set the prefs on b2g.js
> or somewhere.

Thank you Olli.

Well, I preferred to change the default because this level looks more accurate both on desktop and on the phone, mainly for sequence of digits that is the principal issue we're having with the VAD, so seems better to increase the defaults at all.

Comment 13

[Olli Pettay [:smaug][[e-mail]]]

Btw, for this kinds of changes, where the backend handling is changed, I'm totally fine if
just kdavis reviews your patches (that might be faster when my review load is high).

Comment 14

[André Natal]

(In reply to Olli Pettay [:smaug] from comment #13)
> Btw, for this kinds of changes, where the backend handling is changed, I'm
> totally fine if
> just kdavis reviews your patches (that might be faster when my review load
> is high).

thank you Olli, this gonna be really helpful.

Comment 15

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/79ecbf9133b1

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/79ecbf9133b1