1211417 - Add Pocketsphinx speech model to gaia-l10n/en-US

Status: RESOLVED FIXED

(Mozilla Localizations :: Other, defect)

Description

[kdavis]

As part of Bug 1178288 the es language pack will include a speech
recognition model for en-US. Thus, this model should thus be included
in the gaia-l10n/en-US repository.

Comment 1

[Axel Hecht [:Pike]]

Sorry, but this is technically *really* hard, as the en-US repo is generated automatically from the gaia repo.

I also disagree that this would be a good thing to have in the en-US repo. For one, every localizer working on sources would need to check out that pretty big chunk of data.

Also, the dashboards would complain about that file missing in localized repos, and I don't expect to actually invest in that part.

I'd prefer if we could fix this on the gaia build side instead of exposing this to all localizers.

Comment 2

[kdavis]

(In reply to Axel Hecht [:Pike] from comment #1)
> Sorry, but this is technically *really* hard, as the en-US repo is generated
> automatically from the gaia repo.

Could you explain why simply committing the model by hand will not work?

> I also disagree that this would be a good thing to have in the en-US repo.
> For one, every localizer working on sources would need to check out that
> pretty big chunk of data.

This is already decided. The es and fr repos already have the corresponding
models. Similarly, the langpack builder has already been modified to handle
this case.

> I'd prefer if we could fix this on the gaia build side...

Currently the speech models and gaia are completely independent. There are no
speech models in gaia.

The en-US model is in gecko and we want the en-US model to behave like all other
speech models, be installed by langpack derived from gaia-l10n/en-US. Thus, we
want the speech model in gaia-l10n/en-US.

Comment 3

[Axel Hecht [:Pike]]

Just appeared to me that gaia-l10n/en-US isn't a repository that's used for building anything, and must not be. It's sole purpose is to be the repository of strings that are OK to expose to localizers at some point in time.
Using gaia-l10n/en-US for building stuff would break the way we use it today, and that's not something we can afford to loose. Notably, we use different branching times and mechanics, and a different cadence of publishing commits for gaia-l10n/en-US.

I don't think we'll ever have an en-US language pack, too. If there was, it'd be built from the files in gaia, not the files in gaia-l10n/en-US.

Sorry, this is just not going to work this way.

PS: technical background on how en-US is generated is on https://blog.mozilla.org/axel/2013/09/19/a-tale-of-convert-and-convert/, which populates https://gaia-l10n.allizom.org/ from which flod or I manually push to gaia-l10n/en-US and sibling. Manual commits break that automation badly, and aren't intended. Also, in this particular case, just not the right answer.

Comment 4

[kdavis]

We can easily build a en-US langpack containing just the speech model.

Comment 5

[Axel Hecht [:Pike]]

"Easily build" is partly the answer. This asset should really built somewhere that doesn't involve gaia-l10n/en-US.

We talked about language packs and stuff in one of the l10n calls today, and we realize that stas and I have different opinions on things here. Stas talks about getting as many locale-related things into language packs as possible, but I'm saying the opposite. I'm doing so based on the discussions that happen around the GoFaster project for desktop, where Tarek is developing hosting and update mechanisms for a variety of data assets. As much as I'm for a unified install UX, I'm optimistic that things like speech recognition and hyphenation dicts etc will want to be served out of band. Laura Thomson would be a good first contact here.

Based on this, I'm resolving this WONTFIX.

Comment 6

[kdavis]

Before resolving as WONTFIX I would suggest you involve all interested parties.

In particular, you should talk to Dylan first as this is not your call to make
and involves issues beyond gaia-l10n/en-US.

Comment 7

[Axel Hecht [:Pike]]

Quite frankly, I'm the module owner of the gaia-l10n repos, so this is my call to make.

I'm happy to leave this bug open while you're trying to find your way, but the outcome is going to be that this is not landing in gaia-l10n/en-US

Comment 8

[Gregor Wagner [:gwagner]]

Zibi, Stas, do you guys have any other ideas how to solve this issue?

Comment 9

[Staś Małolepszy :stas]

A (possibly temporary) solution which comes to my mind is to modify the langpack-builder and add a special speech-model-only mode which works only for en-US and pull data directly from a local clone of Gaia.  This way we can leave gaia-l10n/en-US untouched.

Re. comment 5, I think we should continue discussing this among our team, the marketplace team and webtools.   We don't have alternative vehicles for shipping things like speech models right now, and I'm happy we got them into langpacks.  Perhaps there's a potential for a better solution in the future, like general resource bundles of which langpacks can be only a part.

Comment 10

[kdavis]

(In reply to Staś Małolepszy :stas from comment #9)
> A (possibly temporary) solution which comes to my mind is to modify the
> langpack-builder and add a special speech-model-only mode which works only
> for en-US and pull data directly from a local clone of Gaia.  This way we
> can leave gaia-l10n/en-US untouched.

Unfortunately the en-US model is not in gaia, but in gecko. So, the mod of the
langpack builder will require, for en-US langpacks, one have gecko checked out
in addition to gaia.

Comment 11

[kdavis]

Created a new repository to hold a copy of gaia-l10n/en-US (http://bit.ly/1RDUk8h), added the en-US speech model, and removed all files not required to create a 'speech model only' langpack (http://bit.ly/1RDUfl6).

Comment 12

[Zibi Braniecki [:zbraniecki][:gandalf]]

Removing an old NI.