1346195 - speechSynthesis.getVoices() and/or querying voice information (for instance, by opening "Reader Mode" the first time) produces temporary audio distortion

Status: RESOLVED INVALID

(Core :: Web Speech, defect)

Description

[Tom Brossman]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20170303012758

Steps to reproduce:

Enable Reader View while audio is playing in an external application.


Actual results:

Reader View works, however the audio playback (using external applications) is badly distorted for 5-7 seconds and then resumes normal playback.


Expected results:

Reader View should work without distorting or affecting audio playback.

Comment 1

[:Gijs (he/him)]

Hard to know where to begin here - reader mode itself doesn't do anything with audio, except perhaps for the narrator part. So some questions:

- is this still happening on more recent versions of Firefox (54 being current release, 56 being current nightly)?
- what external application is playing audio?
- if you go to about:config, set "narrate.enabled" to false, and restart the browser, and then try again, does the same thing happen or not?

Comment 2

[Tom Brossman]

Admittedly it's a strange bug but it does happen. It started right away with a new computer build, and not with a new FF release, so the source of the problem is at least partly hardware-related. It also only ever happens the first time I toggle reader mode, and does not recur until the next day (assuming normal use with the computer off overnight).

- is this still happening on more recent versions of Firefox (54 being current release, 56 being current nightly)?

Yes, I'm on the default FF provided with Ubuntu 16.04 Desktop. Current version is 54

- what external application is playing audio?

Usually Radiotray, but I can reproduce the behaviour with other applications like Rhythmbox.

- if you go to about:config, set "narrate.enabled" to false, and restart the browser, and then try again, does the same thing happen or not?

Just tested this and rebooted, and did not get the distortion. I'll try again tomorrow morning (UK time) and update this report then. As this could have something to do with hardware, I want to check that leaving it plugged in and turned off overnight isn't part of the problem.

Comment 3

[Tom Brossman]

Just tested again and I confirm toggling narrate.enabled to 'False' in about:config has definitely stopped the audio distortion from happening again. It doesn't fix the underlying issue but I don't use the narrator feature and this is an acceptable workaround.

I'm happy to provide further info and troubleshoot some more if there are other questions.

Comment 4

[:Gijs (he/him)]

Thanks for checking!

If you load the following minimal testcase in a new tab, do you get the same audio distortion: http://output.jsbin.com/wivuzavame ?

If so, out of curiosity, how many voices are displayed on that page (if any) ? (if it's a lot, and you're comfortable with the developer tools, counting with the devtools might be easier - document.querySelectorAll('p').length would do it)


Eitan, any idea where to look for this issue specifically? If disabling narrate fixes it, I imagine it has to do with the speech synthesis / web speech implementation?

Comment 5

[Tom Brossman]

Visiting http://output.jsbin.com/wivuzavame does produce the same distortion, which stops after a few seconds just like the original issue.

Nice trick with the devtools, I get a value of 69 voices returned.

To try something different, I just tested streaming audio from the browser directly. With no third-party applications playing audio, enabling reader mode (with narrate.enabled = true) did not produce any distortion. The audio distortion is only heard when audio is coming from another application.

This only ever happens the first time reader mode is enabled. I have to reboot in between each test. Exiting and re-launching Firefox isn't enough to trigger the distortion.

I also attempted to record a sample of the distortion with Audacity, however Audacity was unable to record the default system audio at first. I used pavucontrol to switch Audacity to a 'monitor' of the system audio, which enabled capturing the system audio, but the distortion was not captured on the audio recording. That might not mean anything but I include it for completeness.

Comment 6

[:Gijs (he/him)]

Wild. Updating the summary now that we've narrowed it down. Hope the web speech folks can take it from here. :-)

Comment 7

[Eitan Isaacson [:eeejay]]

This sounds like an issue with speech-dispatcher. Not sure why its not reproduced when playing media from firefox itself, since speech-dispatcher is a separate app.

What external app are you testing with? Does this happen with more than one? For example, if you played a video with Totem, would you hear the distortion?

Comment 8

[Tom Brossman]

> What external app are you testing with? Does this happen with more than one? For example, if you played a video with Totem, would you hear the distortion?

I can reliably reproduce this with Rhythmbox, Radiotray, and gnome-sushi. I had not yet tested video, but just tried and I get the same distortion with Totem as well.

Looking at Ubuntu's Settings > Sound > Applications (tab) I see four instances of 'speech-dispatcher'. When playing audio from an audio or video source, only one instance of that application appears there.

Should I file this as a bug report against speech-dispatcher instead?

Comment 9

[Tom Brossman]

Apologies for the extra notification, but the point I made above about the distortion not happening with Firefox playing audio was incorrect.

I've just re-tested everything to double-check and I DO get temporary distortion with all applications playing audio, including Firefox.

Comment 10

[Eitan Isaacson [:eeejay]]

OK, that's consistent with my theory that this is a speechd thing. I don't know why they need to touch the audio device for simply querying voices and not actually playing anything. I would definitely file something upstream and point them to this bug.