1907786 - piper module and voices not added for Web Speech API

Status: RESOLVED INVALID

(Core :: Widget: Gtk, defect, P3)

Description

[guest271314]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0

Steps to reproduce:

I have added the piper module to Speech Dispatcher following the instructions here module request: piper #866.

Tested on Chromium Version 128.0.6586.0 (Developer Build) (64-bit) and Firefox Nightly 130.0a1. Chromium works. Firefox does not load the piper voices.

In pertinent part.

Download the piper executable from releases, extract the contents and save to ~/.local/opt/piper.

Download a couple .onnx files and save to ~/.local/share/piper/voices.

Create a symbolic link to piper executable in ~/.local/bin, ln -s ~/.local/opt/piper/piper piper.

Install python3-speechd

sudo apt install python3-speechd
spd-conf -u

Modify ~/.config/speech-dispatcher/speechd.conf to add the piper module

 AddModule "piper"                     "sd_generic"   "piper.conf"

or set piper as the default module

 DefaultModule espeak-ng
 # piper

Create ~/.config/speech-dispatcher/modules/piper.conf

Debug 0

GenericExecuteSynth "printf %s \'$DATA\' | /home/xubuntu/.local/bin/piper --length_scale 1 --sentence_silence 0  --model ~/.local/share/piper/voices/$VOICE --output-raw | aplay -r 22050 -f S16_LE -t raw -"

# only use medium quality voices to respect the 22050 rate for aplay in the command above.

GenericCmdDependency "piper"
GenericCmdDependency "aplay"
GenericCmdDependency "printf"
GenericSoundIconFolder "/usr/share/sounds/sound-icons/"

GenericPunctNone ""
GenericPunctSome "--punct=\"()<>[]{}\""
GenericPunctMost "--punct=\"()[]{};:\""
GenericPunctAll "--punct"

#GenericStripPunctChars  ""

GenericLanguage  "en" "en_US" "utf-8"

AddVoice        "en"    "MALE1"         "en_US-hfc_male-medium.onnx"
AddVoice        "en"    "FEMALE1"       "en_US-hfc_female-medium.onnx"

DefaultVoice    "en_US-hfc_male-medium.onnx"

#GenericRateForceInteger 1
#GenericRateAdd 1
#GenericRateMultiply 100

Restart speech-dispatcher with speech-dispatcher restart.

Terminate and restart chrome, killall -9 chrome.

Open DevTools, test in console

var voices = speechSynthesis.getVoices().filter(({name}) => name.includes("piper"));
var u = new SpeechSynthesisUtterance();
u.voice = voices[0];
u.text = "Test, test, test. Test to the point it breaks.";
speechSynthesis.speak(u);

console.log(JSON.stringify(voices.map(({default:_default, lang, localService, name, voiceURI}) => ({_default, lang, localService, name, voiceURI})), null, 2));

VM924:6 [
  {
    "_default": false,
    "lang": "en",
    "localService": true,
    "name": "en_US-hfc_female-medium.onnx piper",
    "voiceURI": "en_US-hfc_female-medium.onnx piper"
  },
  {
    "_default": false,
    "lang": "en",
    "localService": true,
    "name": "en_US-hfc_male-medium.onnx piper",
    "voiceURI": "en_US-hfc_male-medium.onnx piper"
  }
]

Actual results:

Firefox does not load the piper voices.

Expected results:

Firefox loads the piper voices.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Gtk' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[guest271314]

The component is probably Web Speech or something like that, not "Widget".

Comment 3

[Martin Stránský [:stransky] (ni? me)]

You write at https://github.com/brailcom/speechd/issues/866 that Firefox ESR works but latest nightly is broken. Can you try nightly with MOZ_ENABLE_WAYLAND=0 env variable?
Thanks.

Comment 4

[guest271314]

I didn't write that. The developer who came up with the roadmap wrote that. I only run Nightly.

Are you talking about something like? That doesn't load piper voices.

MOZ_ENABLE_WAYLAND=0 ~/firefox/firefox-bin

Comment 5

[Thinker Li [:sinker]]

Please try MOZ_LOG=SpeechSynthesis:5 firefox and attach the log here.
Also try to run speech-dispatcher -D -l 5 in order to create a debug log from speech-dispatcher.

Comment 6

[guest271314]

(In reply to Thinker Li [:sinker] from comment #5)

Please try MOZ_LOG=SpeechSynthesis:5 firefox and attach the log here.
Also try to run speech-dispatcher -D -l 5 in order to create a debug log from speech-dispatcher.

This doesn't log anything to tts.log

~/firefox/firefox-bin -MOZ_LOG="SpeechSynthesis:5" -MOZ_LOG_FILE=tts.log https://gist.github.com/guest271314/9f09ab899df11e344c568a7b93f544c3

speechSynthesis.speak() actually does output sound. Looks like the name of the module is missing from Firefox implementation. Chromium 142 prints the "piper" name in the name of the module

Firefox 144

Array [ SpeechSynthesisVoice, SpeechSynthesisVoice ]
​
0: SpeechSynthesisVoice { voiceURI: "urn:moz-tts:speechd:en_US-hfc_male-medium.onnx?en", name: "en_US-hfc_male-medium.onnx", lang: "en", … }
​
1: SpeechSynthesisVoice { voiceURI: "urn:moz-tts:speechd:en_US-hfc_female-medium.onnx?en", name: "en_US-hfc_female-medium.onnx", lang: "en", … }
​
length: 2
​
<prototype>: Array []

Chromium

(2) [SpeechSynthesisVoice, SpeechSynthesisVoice]
0
: 
SpeechSynthesisVoice {voiceURI: 'en_US-hfc_female-medium.onnx piper', name: 'en_US-hfc_female-medium.onnx piper', lang: 'en', localService: true, default: true}
1
: 
SpeechSynthesisVoice {voiceURI: 'en_US-hfc_male-medium.onnx piper', name: 'en_US-hfc_male-medium.onnx piper', lang: 'en', localService: true, default: false}
length
: 
2
[[Prototype]]
: 
Array(0)

Comment 7

[Thinker Li [:sinker]]

(In reply to guest271314 from comment #6)

(In reply to Thinker Li [:sinker] from comment #5)

Please try MOZ_LOG=SpeechSynthesis:5 firefox and attach the log here.
Also try to run speech-dispatcher -D -l 5 in order to create a debug log from speech-dispatcher.

This doesn't log anything to tts.log

~/firefox/firefox-bin -MOZ_LOG="SpeechSynthesis:5" -MOZ_LOG_FILE=tts.log https://gist.github.com/guest271314/9f09ab899df11e344c568a7b93f544c3

It should be MOZ_LOG="..." MOZ_LOG_FILE="..." ~/firefox/firefox-bin https://.... Here is setting env variables before running the command firefox.

speechSynthesis.speak() actually does output sound. Looks like the name of the module is missing from Firefox implementation. Chromium 142 prints the "piper" name in the name of the module

So, it actually works. Your concern is the names of voices, right?

Comment 8

[guest271314]

When I originally filed this bug it didn't work at all.

Now I don't see a way to determine the engine and voices source in the name of the voice.

Comment 9

[Thinker Li [:sinker]]

Great! I will close this bug.
I don't know how the name of a voice is described in the Web Speech API document, not sure if the engine name should be part of a voice name.
If you think it is an issue, please file another bug for Web Speech component.

Comment 10

[guest271314]

If you think it is an issue, please file another bug for Web Speech component.

https://bugzilla.mozilla.org/show_bug.cgi?id=1988431