1597760 - Mic button shows listening feedback animation until a string is returned from the API

Status: NEW

(Core :: Web Speech, defect, P3)

Description

[Daniel Bodea [:danibodea]]

Note

• When the user taps the mic button and speaks into the microphone if he pauses for about 1 second, the audio capturing will stop, but the mic button's "listening" animation will only stop after the API returns a string. This way, the user may be caught speaking for another 5 seconds before realizing that audio capture has stopped somewhere along the way.

Affected versions

• Nightly v72.0a1
• Other channels don't have the API yet.

Affected platforms

• All

Steps to reproduce

1. Open the browser and flip the following prefs to "true":

• media.webspeech.recognition.enable
• media.webspeech.recognition.force_enable

2. Go to https://translate.google.com/
3. Select English as the input language, click the mic button and speak into the mic.

Expected result

• The mic button's "listening" feedback animation should stop when sound capture actually stops.

Actual result

• The mic button's "listening" feedback animation only stops when the string is returned (a few seconds later).

Additional notes

• This issue cand also be observed when using Firefox Web Speech API Chatbot (https://andrenatal.github.io/webspeechapi/) or the Web Speech API Demonstration (https://www.google.com/intl/en/chrome/demos/speech.html).
• The behavior of the Chrome browser is different in this case because the actual sound capture does not stop at all. It needs to be stopped manually.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:anatal, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[Luka Zitnik]

Looking at the Web Speech API Demonstration (https://www.google.com/intl/en/chrome/demos/speech.html) only, how it stops the animation on end event, instead of on audioend event, this bug is most probably wrongly attributed to the web speech api.

Excerpt from https://www.google.com/intl/en/chrome/demos/speech.html:

  recognition.onstart = function() {
    recognizing = true;
    showInfo('info_speak_now');
    start_img.src = '/intl/en/chrome/assets/common/images/content/mic-animate.gif';
  };
...
  recognition.onend = function() {
    recognizing = false;
    if (ignore_onend) {
      return;
    }
    start_img.src = '/intl/en/chrome/assets/common/images/content/mic.gif';
...

Comment 3

[Daniel Bodea [:danibodea]]

Can you help with a proposition for a better component? Or maybe redirect this question to someone who knows? Thanks.

Comment 4

[Luka Zitnik]

Daniel, I'm anticipating two ways in which this bug can be resolved.

If this modified Web Speech API Demonstration matches your expected behavior, we can say that the bug is in the examples and the bug should be readdressed to their authors.

However, because the examples were written for continuous mode, that is dictation instead of voice commands, and continuous mode is not yet implemented in Firefox (see https://bugzilla.mozilla.org/show_bug.cgi?id=1185233), we can say that the bug is really a manifestation of the missing feature and that the bug depends on it.

What are your thoughts?

Comment 5

[Daniel Bodea [:danibodea]]

I understand the cause of the issue now. I do not think that a fix will be needed until the Web Speech API will be fully implemented and released.
This bug may not be necessary considering that the cause is an unimplemented part of the feature, but, in the long run, it's better to have these bugs logged and verify them when the parts are there.

I am not equipped to decide how this bug is going to be fixed, only the verification.