1456885 - Not able to use Voice for Google Docs Integration due to missing SpeechRecognition support

Status: NEW

(Web Compatibility :: Site Reports, defect, P2)

Description

[pgerman]

Attachment: Screen Shot 2018-04-25 at 11.29.58 AM.png

In Google Docs, there is an option for 'voice typing'.

This is currently unavailable in the build of firefox nightly that I'm on. It would be great for Firefox users to be able to make use of this feature.

Comment 1

[Marius Coman [:mcoman], Ecosystem QA]

It seems that the "Voice Typing" option is available only in "Chrome" browsers (https://support.google.com/docs/answer/4492226?hl=en). So I am not sure if this should be implemented on the Firefox side.

Mike could you please give us your opinion regarding this?

Comment 2

[Mike Taylor [:miketaylr]]

Interesting, I wasn't aware of this feature before.

I would guess this depends on Bug 1248897, can you try to verify Karl?

Comment 3

[Mike Taylor [:miketaylr]]

(proactively moving to TE::Desktop)

Comment 4

[Karl Dubost💡 :karlcow]

Steps to reproduce.

0. Go to docs.google.com with Firefox User agent
1. Login with your account
2. Create a new blank document
3. Head to Tools in the menu bar of Google Docs, there is a Voice Typing menu
4. (In chrome), choosing the menu item opens a pop up with an icon to click to start speaking.

Actual:
The menu is greyed out on Firefox (both with Firefox UA and faking Chrome UA). 
With Chrome, the menu is active.

So at least they seem to do the right thing here. They seem to detect if Voice recognition is possible.
There is no support for Speech Recognition yet in Firefox.
https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition


In /static/document/client/js/2112717014-kix_main_i18n_kix_tertiary__en_gb.js

```js
H(jxe, O);
var lxe = ["webkitSpeechRecognition"];
jxe.prototype.dispose = function() {
    jxe.aa.dispose.call(this);
    this.L && (this.H = "dispose",
    this.C.stop())
}
;
function kxe() {
    for (var a = 0; a < lxe.length; a++)
        if (Rl.hasOwnProperty(lxe[a]))
            return Rl[lxe[a]];
    return null
}
var mxe = ["webkitSpeechGrammarList"];
z = jxe.prototype;
z.rk = function(a, b, c) {
    this.C.lang = a;
    this.R = !c;
    b: {
        for (a = 0; a < mxe.length; a++)
            if (Rl.hasOwnProperty(mxe[a])) {
                a = Rl[mxe[a]];
                break b
            }
        a = null
    }
    a ? (a = new a,
    b && a.addFromUri(b, 1),
    c && a.addFromUri(c, 1),
    b = a) : b = null;
    b && (this.C.grammars = b)
}
;
```


This will become a webcompat issue if SpeechRecognition is activated and they do not modify the code.
So this is dependent on Bug 1248897 (SpeechRecognition)
and in the current state also on Bug 1409526 (webkitSpeechRecognition alias)

Comment 5

[Mike Taylor [:miketaylr]]

Thanks Karl!

Comment 6

[Firefox Bug Husbandry Bot]

See bug 1547409. Moving webcompat whiteboard tags to keywords.

Comment 7

[:gerard-majax]

The current WebSpeech API implem should make it exposed. Can we re-check that ? Last time I tested local DeepSpeech with WebSpeech API, that was (partly) on Google Docs. I also got some cases where you may need to fake Chrome user-agent.

Comment 8

[Karl Dubost💡 :karlcow]

Attachment: tools menu

Alexandre,

fwiw, Voice Typing menu has disappeared from chrome and is not available in Firefox after enabling the config.
https://support.google.com/docs/answer/4492226?hl=en
The voice typing menu is absent. The doc doesn't seem up to date.

OR I'm doing something wrong.

Comment 9

[:gerard-majax]

(In reply to Karl Dubost💡 :karlcow from comment #8)

Created attachment 9112186 [details]
tools menu

Alexandre,

fwiw, Voice Typing menu has disappeared from chrome and is not available in Firefox after enabling the config.
https://support.google.com/docs/answer/4492226?hl=en
The voice typing menu is absent. The doc doesn't seem up to date.

OR I'm doing something wrong.

Thanks Karl, was that when faking Chrome user-agent ? If not, can you try with UA spoofing?

Comment 10

[Karl Dubost💡 :karlcow]

I tried:

1. with Chrome
2. with Firefox
3. With spoofing ua on Firefox to be chrome.

Menu is not here.

Comment 11

[Eric Ziegenhorn :ericz]

I just tried Voice Typing in Chrome today and it works just fine. Confirmed there is no such menu item on Firefox (without spoofing UA). I'm not sure how niche this feature is but I've seen schools push it for kids to use, which is to say pushing them to use Chrome as it stands.

Comment 12

[Karl Dubost💡 :karlcow]

so impersonation Chrome UA on Firefox Nightly, I still don't get the menu.
They are detecting

var JHh = [
'webkitSpeechRecognition'
],
LHh = [
'webkitSpeechGrammarList'
];

in https://docs.google.com/static/document/client/js/777684991-client_js_prod_kix_tertiary__en_gb.js

And according to Bug 1409526 it has been implemented.
https://searchfox.org/mozilla-central/rev/e0eb861a187f0bb6d994228f2e0e49b2c9ee455e/dom/media/webspeech/recognition/SpeechRecognition.h#74-77
and
https://searchfox.org/mozilla-central/rev/e0eb861a187f0bb6d994228f2e0e49b2c9ee455e/dom/media/webspeech/recognition/SpeechGrammarList.h#38-41

André,
do you know what could be missing?

Or maybe I should directly ask Google what do they need to move forward with Voice Typing.
I'll do that.

Comment 13

[Karl Dubost💡 :karlcow]

Sent a message to Google gsuite discuss to understand what their blockers on enabling voice typing for Firefox.

Comment 14

[Dennis Schubert [:denschub]]

(In reply to Karl Dubost💡 :karlcow from comment #12)

do you know what could be missing?

Note that Speech Recognition is disabled by default everywhere, and is available Nightly-only behind the media.webspeech.recognition.enable preference. I believe the plan is not to ship the current implementation into release and enable it by default, but the plan is to build something that works offline on the user's device using DeepSpeech (see bug 1474084), but Adam or Nils might know more.

Comment 15

[André Natal]

There's no current plans to work on this feature anymore. @Dennis is right, the recognition part of the WSA is available only in Nightly behind that pref, but compatibility with Google properties is unknown given the way they implemented the API in Chrome.

Comment 16

[Karl Dubost💡 :karlcow]

Let's downgrade as P3. But we don't have any other websites having this issue.

Comment 17

[Raul Bucata]

The feature is still not available when using Firefox:

Chrome:
https://prnt.sc/LhyOeS7PehdN

Firefox:
https://prnt.sc/_9vdpHLhUn4v

Tested with:

Browser / Version: Firefox Nightly 101.0a1 (2022-04-10) (64-bit) / Chrome Version 100.0.4896.75 (Official Build) (64-bit)
Operating System: Windows 10 PRO x64

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last months and this bug has priority 'P1'.
:denschub, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 19

[Suhaib Mujahid [:suhaib]]

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

Comment 20

[Raul Bucata]

The feature is still N/A for Firefox: https://prnt.sc/ixX73PybxUqM

Tested with:

Browser / Version: Firefox Nightly 111.0a1 (2023-01-18) (64-bit)
Operating System: Windows 10 PRO x64

Comment 22

[J. Bumblebee Extraordinaire]

Comment on attachment 9382754 [details]
voice typing is now easily available on docs

Hi @divya

I see your screenshot. But I want to ask you: does voice typing actually work in Firefox? What did you do differently that the other posters didn't?

Comment 23

[:gerard-majax]

(In reply to J. Bumblebee Extraordinaire from comment #22)

Comment on attachment 9382754 [details]
voice typing is now easily available on docs

Hi @divya

I see your screenshot. But I want to ask you: does voice typing actually work in Firefox? What did you do differently that the other posters didn't?

This is spam. If you are not a spammer, please say so :)

Comment 24

[:gerard-majax]

(In reply to :gerard-majax from comment #23)

(In reply to J. Bumblebee Extraordinaire from comment #22)

Comment on attachment 9382754 [details]
voice typing is now easily available on docs

Hi @divya

I see your screenshot. But I want to ask you: does voice typing actually work in Firefox? What did you do differently that the other posters didn't?

This is spam. If you are not a spammer, please say so :)

Ok maybe I'm mistaken. The screenshot just shows that the menu entry is back, echo'ing to comments above where it was removed. But unfortunately, we still have no implementation for it (we dropped the work on deepspeech)