Expose platform's text to speech functionality

RESOLVED DUPLICATE of bug 525444

Status

()

Core
DOM
RESOLVED DUPLICATE of bug 525444
6 years ago
5 years ago

People

(Reporter: eeejay, Assigned: eeejay)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

6 years ago
It would be nice if extension writers would be able to access the platform's TTS from within the browser. In Android it is almost impossible for extensions to use the TTS API without a reference to the application's instance object (the Context), so it really has to be done from the browser's native code.

This could all be exposed via XPConnect with a simple API for now (fancy stuff like SSML PLS, and Speech CSS could get their own methods down the road). An initial implementation would be for Android, and we could extend it to other platforms as we go.
Status: UNCONFIRMED → NEW
Ever confirmed: true
AFAIK, there are even plans to have TTS available for the web platform.
OS: Linux → All
Hardware: x86_64 → All
Version: unspecified → Trunk
(Assignee)

Comment 2

6 years ago
I have a WIP ctypes js module that currently supports picotts which is on Android/B2G.

https://github.com/eeejay/mozilla-central/tree/tts
Assignee: nobody → eitan
Summary: Expose platform's text to speech functionality as XPCOM component → Expose platform's text to speech functionality
Let's get WebAPI feedback on that. Mounir would that be you?

Comment 4

6 years ago
Something like http://www.w3.org/2005/Incubator/htmlspeech/XGR-htmlspeech-20111206/#tts-section
would be nice.
I don't know much about TTS. Could someone who knows about TTS draft an API and send it to dev-webapi and from there we could discuss it?
(Assignee)

Comment 6

5 years ago
(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #5)
> I don't know much about TTS. Could someone who knows about TTS draft an API
> and send it to dev-webapi and from there we could discuss it?

I'll draw something up and send it to the list. BTW, Chris Jones said that he doesn't see this as a webapi, just as a js module. I am not really sure what that means, or what the rationale is. Either way, it needs an API, and I'll send something out this week.
(In reply to Olli Pettay [:smaug] from comment #4)
> Something like
> http://www.w3.org/2005/Incubator/htmlspeech/XGR-htmlspeech-20111206/#tts-
> section
> would be nice.

Olli what is the status of that API? Has webkit or IE implemented it?  (Note it seems different from the chrome.tts work)

Comment 8

5 years ago
Chrome is implementing a subset of the recognition API, at least. Not sure about the TTS.
(In reply to Eitan Isaacson [:eeejay] from comment #6)
> (In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #5)
> > I don't know much about TTS. Could someone who knows about TTS draft an API
> > and send it to dev-webapi and from there we could discuss it?
> 
> I'll draw something up and send it to the list. BTW, Chris Jones said that
> he doesn't see this as a webapi, just as a js module. I am not really sure
> what that means, or what the rationale is. Either way, it needs an API, and
> I'll send something out this week.

The audio data API and JS itself are the only primitives we need to implement TTS.  That means web pages could include a tts.js or something that provides TTS.  But if it's horrendously complicated to implement well / efficiently at the moment, then it would be a candidate for a transitional API.  I'd like us to try the pure-content route first.
(Assignee)

Comment 10

5 years ago
Created attachment 659780 [details] [diff] [review]
Introduce tts.jsm
(Assignee)

Comment 11

5 years ago
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Here is my solution. It is expandable to other platform speech APIs and engines without depending on any build-time configuration.

The real question for me is where should something like this live? toolkit/content? don't know.
Attachment #659780 - Flags: feedback?(jones.chris.g)
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Sorry, this isn't my front yard, code-wise.  Fabrice and Vivien can provide better feedback.
Attachment #659780 - Flags: feedback?(jones.chris.g)
Attachment #659780 - Flags: feedback?(fabrice)
Attachment #659780 - Flags: feedback?(21)
(Assignee)

Updated

5 years ago
Blocks: 795984
OK let's target FF 19 for this. Feedback welcome in the meantime (as time permits).
Olli had a question about my comment 13. This is in the context of our TTS a11y solution on FF OS. I'm assuming the next FF OS version (we want to target for TTS) will be based off of FF 19+ (but I could be wrong).
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

As much as I would like to have time to look at this FFOS keeps pushing me back to reality. But hopefully I know someone that loves JS-ctypes and that could maybe also find a use for this in JetPack. So let's see if Alexandre want to help this things going along here.
Attachment #659780 - Flags: feedback?(21) → feedback?(poirot.alex)
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Vivien, The original question is: is this code and such usage of jsctypes acceptable for toolkit/? I'm not the right guy to confirm this, even if I would like to see it answered positively.
Attachment #659780 - Flags: feedback?(poirot.alex) → feedback?(21)
(Assignee)

Comment 17

5 years ago
Bug #525444 is progressing. Since it would provide the same features, but for content, this bug should probably be closed. I'll remove the flags on the patch so no one wastes more time on it.
(Assignee)

Updated

5 years ago
Attachment #659780 - Flags: feedback?(fabrice)
Attachment #659780 - Flags: feedback?(21)
(Assignee)

Comment 18

5 years ago
This is being implemented in content via Web Speech API
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 525444
You need to log in before you can comment on or make changes to this bug.