863174 - Add speech synthesis libraries to Gonk

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Eitan Isaacson [:eeejay]]

There are two options. I would like to add both, if possible.

eSpeak (GPLv3 licensed)
Pros:
 1. Very responsive, makes it favorable for accessibility needs.
 2. Good language support.
Cons:
 1. Sounds very tinny and robotic.

picotts (Apache 2 licensed)
Pros:
 1. Sounds more natural than eSpeak
Cons:
 1. Not as responsive as eSpeak.
 2. Limited language support (no Portuguese).

Comment 1

[David Bolter [:davidb] (NeedInfo me for attention)]

David do you know who would make the call on whether we can add both eSpeak and picotts to gonk?

Comment 2

[Eitan Isaacson [:eeejay]]

I guess we should ping legal about a GPLv3 libary?

Comment 3

[Dave Hylands [:dhylands]]

I think we need to get legal's input on the licenses and then it's just a case of figuring out what's required to add it to the build/toolchain or whatever.

We also need to consider how big the libraries are (I know we're tight for space on some devices).

We need to set things up so that we can choose to include them or not (so we need to set them up as Android modules, and then each device can choose to include one or both or neither as part of the build.

Comment 4

[Michael Wu [:mwu]]

FWIW, library size isn't likely to be a big issue unless you're exceeding 10mb.

Comment 5

[Gervase Markham [:gerv]]

Is Festival also an option?
http://en.wikipedia.org/wiki/Festival_Speech_Synthesis_System

Gerv

Comment 6

[Eitan Isaacson [:eeejay]]

(In reply to Gervase Markham [:gerv] from comment #5)
> Is Festival also an option?
> http://en.wikipedia.org/wiki/Festival_Speech_Synthesis_System
> 
> Gerv

The biggest problem with festival is that there is almost no language support outside of English. espeak is attractive because it supports many languages.

At least in the past Festival was passed over in mobile/embedded because it is too large and slow. This might not be true anymore with current mobile systems. Anyway, flite was created (fesitval-lite), and it has a significantly smaller footprint according to a paper[1] from 2001.

http://www.cs.cmu.edu/~awb/papers/ISCA01/flite/node9.html

Comment 7

[Eitan Isaacson [:eeejay]]

OK. As per secret discussions in the dependent legal bug, we will stick to SVOX Pico and drop espeak for the time being.

Pico should be really easy to include it is part of the extended Android project as well:
https://android.googlesource.com/platform/external/svox

Comment 8

[Marco Zehe (:MarcoZ)]

So, what else does this need to land? Can we get moving on this so we can move forward with other dependent pieces for Firefox OS accessibility?

Comment 9

[Eitan Isaacson [:eeejay]]

https://github.com/mozilla-b2g/platform_build/pull/24
https://github.com/mozilla-b2g/b2g-manifest/pull/90

Comment 10

[Eitan Isaacson [:eeejay]]

Attachment: 0001-Added-svox-to-manifests.patch

This looks right-ish to me.

Comment 11

[Eitan Isaacson [:eeejay]]

Attachment: 0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch

This is a patch for the platform_build repo. I don't know if I did the right thing here, but I needed to somehow get libttspico.so into /system/lib, and I suspect that this helped.

Comment 12

[Peter Vágner]

From the users perspective having eSpeak available is verry important feature. Currently there is no other speech synthesiser capable of speaking in such impresive number of languages. I understand its output does not sound as natural as most of the sighted people can accept however visually disabled people are usually trained to take advantage of this and appreciate its responsiveness and speech rate targetting better efficiency.
For comparison and possible personal testing please note a free and open-source screen reader for Windows NVDA has eSpeak built-in and using the computer with this setup is really a breeze.
I have attempted to post about inability to include eSpeak into Firefox OS given the current status to the espeak-general email list. The message is not yet available in the online archive. I hope we might get some answers from eSpeak developer really soon. What I think is that while Mozilla is building on open standard making the software available for usage and modifications to anyone in the world with added benefit of being fully accessible eSpeak goal is verry similar so I am sure there must be a way to get this licencing stuff resolved.
Here is an up to date android port of eSpeak which might be possible to tweak for inclusion to Firefox OS once this is discussed and a working solution is found: https://github.com/rhdunn/espeak/tree/android

Greeting

Peter

Comment 13

[Michael Wu [:mwu]]

Comment on attachment 785309 [details] [diff] [review]
0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch

Please add this to gonk-misc/b2g.mk instead.

Comment 14

[Reece H. Dunn]

Is it possible to provide a plugin API that TTS providers can implement, then provide Firefox OS installers for them like Android does with its TTS framework? This should resolve the issue with eSpeak, allowing it to hook into the Firefox OS TTS framework like any other platform (e.g. Windows and Android).

This will also allow other TTS engines/voices like Acapella and IVONA to be ported to Firefox OS, giving users a greater choice of voices to use. And it opens up support for more languages over the ones that are supported out-of-the-box.

Comment 15

[Michael Wu [:mwu]]

Comment on attachment 785308 [details] [diff] [review]
0001-Added-svox-to-manifests.patch

Review of attachment 785308 [details] [diff] [review]:
-----------------------------------------------------------------

I recommend adding this to base-jb.xml only. A number of these device manifests are used by our infra to build and adding new repos on existing manifests usually requires some coordination with releng to do successfully. As 1.2 will be based on JB, this will cover us going forward.

However, if you open a bug to have releng add svox to their mirror list, we can do all manifests. Up to you.

Comment 16

[Eitan Isaacson [:eeejay]]

(In reply to Michael Wu [:mwu] from comment #15)
> Comment on attachment 785308 [details] [diff] [review]
> 0001-Added-svox-to-manifests.patch
> 
> Review of attachment 785308 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I recommend adding this to base-jb.xml only. A number of these device
> manifests are used by our infra to build and adding new repos on existing
> manifests usually requires some coordination with releng to do successfully.
> As 1.2 will be based on JB, this will cover us going forward.
> 

So far, it looks like only the emulators and nexus-4 use the base-jb.xml. Are you saying that all device manifests will use this soon?

> However, if you open a bug to have releng add svox to their mirror list, we
> can do all manifests. Up to you.

You mean add svox repo to git.mozilla.org?

Comment 17

[Eitan Isaacson [:eeejay]]

https://github.com/mozilla-b2g/gonk-misc/pull/105

Comment 18

[Eitan Isaacson [:eeejay]]

Attachment: Add-libttspico-to-core-PRODUCT_PACKAGES

This does not look like the right place for it, but you know better :)

Comment 19

[Eitan Isaacson [:eeejay]]

(In reply to Eitan Isaacson [:eeejay] from comment #16)
> (In reply to Michael Wu [:mwu] from comment #15)
> > However, if you open a bug to have releng add svox to their mirror list, we
> > can do all manifests. Up to you.
> 
> You mean add svox repo to git.mozilla.org?

OK, looks like releng has been great and responsive. Bug #901625 did just that. I guess this patch is good as-is?

Comment 20

[Michael Wu [:mwu]]

(In reply to Eitan Isaacson [:eeejay] from comment #19)
> (In reply to Eitan Isaacson [:eeejay] from comment #16)
> > (In reply to Michael Wu [:mwu] from comment #15)
> > > However, if you open a bug to have releng add svox to their mirror list, we
> > > can do all manifests. Up to you.
> > 
> > You mean add svox repo to git.mozilla.org?
> 
> OK, looks like releng has been great and responsive. Bug #901625 did just
> that. I guess this patch is good as-is?

Nope. The wrong one was mirrored. Our primary mirror is codeaurora (caf).

Comment 21

[Michael Wu [:mwu]]

(In reply to Eitan Isaacson [:eeejay] from comment #18)
> Created attachment 785888 [details] [diff] [review]
> Add-libttspico-to-core-PRODUCT_PACKAGES
> 
> This does not look like the right place for it, but you know better :)

Our partners do not always use our build repo, but they always use gonk-misc. This is the easiest way to give them the change.

The other issue is that there's two branches of the build repo that need to be modified if you want to go that route. gonk-misc is just much easier to deal with and concentrates all the gonk specific changes in one place.

Comment 22

[Michael Wu [:mwu]]

Comment on attachment 785888 [details] [diff] [review]
Add-libttspico-to-core-PRODUCT_PACKAGES

Review of attachment 785888 [details] [diff] [review]:
-----------------------------------------------------------------

::: b2g.mk
@@ +13,4 @@
>  	init.rc \
>  	init.b2g.rc \
>  	killer \
> +        libttspico \

Use a tab here - the rest of the list uses tabs.

Comment 23

[Eitan Isaacson [:eeejay]]

Attachment: 0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch

Comment 24

[James Teh [:Jamie]]

Any chance of declassifying some of the legal discussion so we actually have a chance of even starting discussions with the eSpeak developer regarding the licensing? As a screen reader power user (I use eSpeak at 852 wpm), Pico is pretty much unusable for me. It'd be a real shame if the great Firefox OS efforts were tarnished by an unresponsive, slow TTS. Poor TTS responsiveness makes the entire experience painful.

Btw, I'm not at all suggesting Pico should be abandoned altogether even if the legal issues *can* ever be resolved with eSpeak. Some users will absolutely prefer Pico.

Comment 25

[Eitan Isaacson [:eeejay]]

(In reply to Michael Wu [:mwu] from comment #20)
> Nope. The wrong one was mirrored. Our primary mirror is codeaurora (caf).

This is fixed in bug #901625.
Viewable at:
https://git.mozilla.org/?p=external/caf/platform/external/svox.git;a=summary

Comment 26

[Michael Wu [:mwu]]

Comment on attachment 785308 [details] [diff] [review]
0001-Added-svox-to-manifests.patch

Please notify the sheriffs and make sure someone can back out when this is landed just in case things break.

Comment 27

[Eitan Isaacson [:eeejay]]

https://github.com/mozilla-b2g/b2g-manifest/commit/32f2a53f87bfe82909d808f4e1c353769b028725

Comment 28

[Eitan Isaacson [:eeejay]]

(In reply to Eitan Isaacson [:eeejay] from comment #23)
> Created attachment 786019 [details] [diff] [review]
> 0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch

I'm assuming this is r+ as well?

Comment 29

[Michael Wu [:mwu]]

Comment on attachment 786019 [details] [diff] [review]
0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch

Looks like we've survived the manifest changes, so this should be fine.

As before, let the sheriffs know in case things break, though I expect this to be ok.

Comment 30

[Eitan Isaacson [:eeejay]]

(In reply to Michael Wu [:mwu] from comment #29)
> Comment on attachment 786019 [details] [diff] [review]
> 0001-Add-libttspico-to-core-PRODUCT_PACKAGES.patch
> 
> Looks like we've survived the manifest changes, so this should be fine.
> 
> As before, let the sheriffs know in case things break, though I expect this
> to be ok.

Since tara is not using base-jb.xml yet, this will break those device builds. Is that OK?

Comment 31

[Michael Wu [:mwu]]

AFAIK, things in PRODUCT_PACKAGE do not actually have to exist. It's more of a whitelist for what gets installed.

Are you planning to dlopen ttspico or directly link at compile time?

Comment 32

[Eitan Isaacson [:eeejay]]

https://github.com/mozilla-b2g/gonk-misc/commit/b86a19e397d0cfff1c240911a439488b48ee051c

Comment 33

[Eitan Isaacson [:eeejay]]

Peter, Jamie & Reece:

While I closed this bug, eSpeak is definitely still on the table. We needed someplace to start, and pico is an easier path right now. We could spin off another eSpeak specific bug.

Comment 34

[Gervase Markham [:gerv]]

eSpeak should be a separate bug.

Gerv