Last Comment Bug 687879 - Expose platform's text to speech functionality
: Expose platform's text to speech functionality
Status: RESOLVED DUPLICATE of bug 525444
:
Product: Core
Classification: Components
Component: DOM (show other bugs)
: Trunk
: All All
: -- normal with 1 vote (vote)
: ---
Assigned To: Eitan Isaacson [:eeejay]
:
Mentors:
Depends on:
Blocks: 795984
  Show dependency treegraph
 
Reported: 2011-09-20 09:15 PDT by Eitan Isaacson [:eeejay]
Modified: 2013-03-15 12:38 PDT (History)
15 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Introduce tts.jsm (25.58 KB, patch)
2012-09-10 10:49 PDT, Eitan Isaacson [:eeejay]
no flags Details | Diff | Review

Description Eitan Isaacson [:eeejay] 2011-09-20 09:15:04 PDT
It would be nice if extension writers would be able to access the platform's TTS from within the browser. In Android it is almost impossible for extensions to use the TTS API without a reference to the application's instance object (the Context), so it really has to be done from the browser's native code.

This could all be exposed via XPConnect with a simple API for now (fancy stuff like SSML PLS, and Speech CSS could get their own methods down the road). An initial implementation would be for Android, and we could extend it to other platforms as we go.
Comment 1 Mounir Lamouri (:mounir) 2011-09-20 09:40:07 PDT
AFAIK, there are even plans to have TTS available for the web platform.
Comment 2 Eitan Isaacson [:eeejay] 2012-03-29 01:08:23 PDT
I have a WIP ctypes js module that currently supports picotts which is on Android/B2G.

https://github.com/eeejay/mozilla-central/tree/tts
Comment 3 David Bolter [:davidb] 2012-03-30 09:04:32 PDT
Let's get WebAPI feedback on that. Mounir would that be you?
Comment 4 Olli Pettay [:smaug] 2012-03-30 09:09:58 PDT
Something like http://www.w3.org/2005/Incubator/htmlspeech/XGR-htmlspeech-20111206/#tts-section
would be nice.
Comment 5 Mounir Lamouri (:mounir) 2012-03-30 16:05:38 PDT
I don't know much about TTS. Could someone who knows about TTS draft an API and send it to dev-webapi and from there we could discuss it?
Comment 6 Eitan Isaacson [:eeejay] 2012-04-02 11:17:46 PDT
(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #5)
> I don't know much about TTS. Could someone who knows about TTS draft an API
> and send it to dev-webapi and from there we could discuss it?

I'll draw something up and send it to the list. BTW, Chris Jones said that he doesn't see this as a webapi, just as a js module. I am not really sure what that means, or what the rationale is. Either way, it needs an API, and I'll send something out this week.
Comment 7 David Bolter [:davidb] 2012-04-02 11:51:24 PDT
(In reply to Olli Pettay [:smaug] from comment #4)
> Something like
> http://www.w3.org/2005/Incubator/htmlspeech/XGR-htmlspeech-20111206/#tts-
> section
> would be nice.

Olli what is the status of that API? Has webkit or IE implemented it?  (Note it seems different from the chrome.tts work)
Comment 8 Olli Pettay [:smaug] 2012-04-02 11:53:08 PDT
Chrome is implementing a subset of the recognition API, at least. Not sure about the TTS.
Comment 9 Chris Jones [:cjones] inactive; ni?/f?/r? if you need me 2012-04-02 14:55:53 PDT
(In reply to Eitan Isaacson [:eeejay] from comment #6)
> (In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #5)
> > I don't know much about TTS. Could someone who knows about TTS draft an API
> > and send it to dev-webapi and from there we could discuss it?
> 
> I'll draw something up and send it to the list. BTW, Chris Jones said that
> he doesn't see this as a webapi, just as a js module. I am not really sure
> what that means, or what the rationale is. Either way, it needs an API, and
> I'll send something out this week.

The audio data API and JS itself are the only primitives we need to implement TTS.  That means web pages could include a tts.js or something that provides TTS.  But if it's horrendously complicated to implement well / efficiently at the moment, then it would be a candidate for a transitional API.  I'd like us to try the pure-content route first.
Comment 10 Eitan Isaacson [:eeejay] 2012-09-10 10:49:28 PDT
Created attachment 659780 [details] [diff] [review]
Introduce tts.jsm
Comment 11 Eitan Isaacson [:eeejay] 2012-09-10 10:53:02 PDT
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Here is my solution. It is expandable to other platform speech APIs and engines without depending on any build-time configuration.

The real question for me is where should something like this live? toolkit/content? don't know.
Comment 12 Chris Jones [:cjones] inactive; ni?/f?/r? if you need me 2012-09-11 17:03:27 PDT
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Sorry, this isn't my front yard, code-wise.  Fabrice and Vivien can provide better feedback.
Comment 13 David Bolter [:davidb] 2012-10-01 10:45:40 PDT
OK let's target FF 19 for this. Feedback welcome in the meantime (as time permits).
Comment 14 David Bolter [:davidb] 2012-10-03 11:02:12 PDT
Olli had a question about my comment 13. This is in the context of our TTS a11y solution on FF OS. I'm assuming the next FF OS version (we want to target for TTS) will be based off of FF 19+ (but I could be wrong).
Comment 15 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2013-02-20 03:18:08 PST
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

As much as I would like to have time to look at this FFOS keeps pushing me back to reality. But hopefully I know someone that loves JS-ctypes and that could maybe also find a use for this in JetPack. So let's see if Alexandre want to help this things going along here.
Comment 16 Alexandre Poirot [:ochameau] 2013-02-20 05:15:54 PST
Comment on attachment 659780 [details] [diff] [review]
Introduce tts.jsm

Vivien, The original question is: is this code and such usage of jsctypes acceptable for toolkit/? I'm not the right guy to confirm this, even if I would like to see it answered positively.
Comment 17 Eitan Isaacson [:eeejay] 2013-02-20 09:01:56 PST
Bug #525444 is progressing. Since it would provide the same features, but for content, this bug should probably be closed. I'll remove the flags on the patch so no one wastes more time on it.
Comment 18 Eitan Isaacson [:eeejay] 2013-03-15 12:38:23 PDT
This is being implemented in content via Web Speech API

*** This bug has been marked as a duplicate of bug 525444 ***

Note You need to log in before you can comment on or make changes to this bug.