1432719 - webSpeech API calls silently fail if speech-dispatcher is not installed

Status: RESOLVED FIXED

(Core :: Web Speech, defect)

Description

[Gian-Carlo Pascutto [:gcp]]

On Linux our implementation of Web Speech synthesis relies on calls to speech-dispatcher, but this is an optional, not a required, dependency on many Linux distros (it for sure is that way on Ubuntu).

If speech-dispatcher is not installed, Firefox just silently fails to support webSpeech, leaving the user to guess as to what the problem is.

https://stackoverflow.com/questions/46617366/speechsynthesis-getvoices-not-listing-voices-in-firefox

It's also not trivial to find that speech-dispatcher is the thing required. I had to dig into the original implementation bug in Bugzilla to find out.

Comment 1

[:gerard-majax]

André, we should fix this right? What would be the correct way? Could we display some banner to the user informing of the situation?

Comment 2

[André Natal]

@eeejay is the right person here.

Comment 3

[Eitan Isaacson [:eeejay]]

I think this is more of a documentation/spec issue than an actual bug. If there is no speech synthesis support on the platform an empty list of voices is expected, and "voiceschanged" will not get fired because no voices were added. If speak() is called in the case of no underlying engine it won't throw an exception, but the given utterance will receive an error event because there were no available voices. This is due to the asynchronous nature of the underlying implementation, and when this API was designed promises were new and weren't considered.

I can see how this can be frustrating for Linux users who don't have speech-dispatcher installed or configured correctly, but that is more due to the fragmentation of Linux desktop.

I can see how this poses a webcompat issue because other Linux browsers bundle their own voices and don't rely on the platform, so they will always have a list of voices.

So there are 3 possible solutions/mitigations I can see, and none of them are an easy fix or something we can unilaterally do:

1. Package our own independent speech synthesis and don't rely on the platform's service.
2. Robustify Linux speech synthesis offerings (a note about speech-dispatcher, flatpak/snap Firefoxs can't access it).
3. Change the SpeechSynthesis spec to be promises based.

Comment 4

[:gerard-majax]

(In reply to Eitan Isaacson [:eeejay] from comment #3)

[...]

So there are 3 possible solutions/mitigations I can see, and none of them are an easy fix or something we can unilaterally do:

1. Package our own independent speech synthesis and don't rely on the platform's service.

DeepSpeech TTS is not a thing anymore, I dont know our official position toward Coqui, and that sounds like a heavy solution, I dont think it's really actionable

2. Robustify Linux speech synthesis offerings (a note about speech-dispatcher, flatpak/snap Firefoxs can't access it).

I really don't see how showing a banner (in a way we can ensure the instructions are security-OK, we dont want to teach user bad habits) when we detect we have no voice installed or we lack libspeechd2.so would be problematic. True it is a dependency-level issue, but what about users installing without packages? There is barely any documentation at all, one has to poke everywhere.

Snap will get fixed, I've got a PR that was merged to expose the socket, and someone else is contributing socket activation support to speechd, so it should be fixed in the near future.

3. Change the SpeechSynthesis spec to be promises based.

While it might be a good solution, it would ultimately be the website's job to notify the user to install stuff on his system ? I feel it already risky when done on our side, but giving the habit to follow that kind of instruction from random website would probably be a bad idea.

Comment 5

[Gian-Carlo Pascutto [:gcp]]

Hi Eitan,

as Alexandre explained, we discontinued our investment in this area so option (1) seems contrary to those decisions. As for (3), this would be a thing further away in the future if we can even get buy in from other browser vendors and get all websites to rewrite all their code. And (2) is the reason we got here in the first place; the major Linux distros do NOT mark speech-dispatcher as a dependency (this is explained in the original comment). And we can say that it's their problem, but the solution the users adopt will be our problem: trying and using another browser.

So yes, the net result is that speech synthesis APIs work in other browsers but "Firefox is broken" and again, as the first comments already describe, it's quite difficult to diagnose this unless you find this bug or find the problem on StackOverflow or something.

I believe for things like DRM support we already support notification-like things to warn the user an EME module is missing, so it does not seem out of line to have a similar notification for missing packages? What do you think?

Comment 6

[:gerard-majax]

Attachment: Bug 1432719 - Notify user on speechd errors r?eeejay!

Comment 7

[:gerard-majax]

Eitan, I believe there's a middle ground here actionable, see the attached WIP. While maybe it's too far reach to fix the spec and be promise-based to properly error the content, we can still catch the expected system-level errors (especially with speech-dispatcher, the fact we have been several loosing hours on that shows we can improve the situation) and expose a chrome-level notification that there's a misconfiguration and maybe direct the user to a support page. While those do not exists for the attached WIP, it's each to write.

Comment 8

[Eitan Isaacson [:eeejay]]

Thanks for this patch. While I am not in love with the idea of a user-visible banner I understand the need for some action on this. If browser peers/UX sign off on this I am OK with it.

This issue reminds me a bit of CORS errors where web pages just seem to be broken and unresponsive. At least personally, I have been accustomed to looking for "Cross-Origin Request Blocked" messages in the console. Maybe instead of a banner some output in the console will give web devs and frustrated users an idea of what's going on? This could be an alternative to the banner in your patch.

I'll write some comments in the patch.

Comment 9

[:gerard-majax]

Attachment: Capture vidéo du 2023-04-26 15-06-52.webm

Video showing the behavior.

Comment 10

[:gerard-majax]

I could verify that this properly detects the failure on Snap package, open-fail being triggered.

Comment 11

[:gerard-majax]

Attachment: image.png

As per some suggestion from #ux on slack, adding possibility for the user to just dont be bothered (saved via a pref)

Comment 12

[:gerard-majax]

Comment on attachment 9330377 [details]
Bug 1432719 - Notify user on speechd errors r?eeejay!

Asking for a UX review. There's one example of such notification, and the message will vary (as visible in the patch with the fluents strings) depending on the level of error. More details would be filled in some support pages.

Comment 13

[:Gijs (he/him)]

(I'm in a work week so I don't have a lot of time right now. I am definitely not the right person to give this UI/UX-review, but I can try to find that person for you.)

Comment 14

[:Gijs (he/him)]

Comment on attachment 9330377 [details]
Bug 1432719 - Notify user on speechd errors r?eeejay!

I passed this on on slack so going to clear this flag.

Comment 15

[:gerard-majax]

So I've got UX verified and SUMO content is getting ready. I'll merge this after the current soft-freeze.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4720a9935e8e
Notify user on speechd errors r=eeejay,fluent-reviewers,flod

Comment 17

[Iulian Moraru]

This is now backed out as per your request Alexandre.

• Push that got backed out

• Backout link

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9458874d5cf9
Notify user on speechd errors r=eeejay,fluent-reviewers,flod

Comment 19

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/9458874d5cf9