1182384 - Include digits in numerical form to standard english pocketsphinx dictionary

Status: RESOLVED FIXED

(Core :: Web Speech, defect)

Description

[André Natal]

Standard pocketsphinx dictionary does not support single digits in form of numbers.

Comment 1

[André Natal]

Attachment: This patch adds digits in numerical form to standard pocketsphinx english dictionary

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8631981 [details] [diff] [review]
This patch adds digits in numerical form to standard pocketsphinx english dictionary

Looks ok to me, though I wonder if there is some actual reason why numbers aren't included.


And kdavis could review this kind of patches.


Are we going to try to upload this kinds of patches to the original
repository of cmu07a.dic?

Comment 3

[Andre Natal]

(In reply to Olli Pettay [:smaug] from comment #2)
> Comment on attachment 8631981 [details] [diff] [review]
> This patch adds digits in numerical form to standard pocketsphinx english
> dictionary
> 
> Looks ok to me, though I wonder if there is some actual reason why numbers
> aren't included.
> 
> 
> And kdavis could review this kind of patches.
> 
> 
> Are we going to try to upload this kinds of patches to the original
> repository of cmu07a.dic?

The reasoj that digits aren't included because they aren't included. Actual situation of poceketsphinx is that developers are responsible for dictionaries by themselves, and they simple are not responsible to have phonemes for every word existent in the world.

Comment 4

[kdavis]

I took a quick look, and have a few comments, but I need to look in
a bit more detail into the pocketsphinx code, before I do an official
review. (I have to finish preparing a presentation tonight for the
talk at https://mozweekend.de so time is a bit tight.)

My few comments are:

1. It looks like the dictionary was modified by someone already, our
   "copy" has firefox as the first entry.
2. All entries, other than firefox and the new numerical values, are
   in alphabetic order.
3. The dict code in pocketsphinx may require the entries to be in al-
   phabetic order, in order to, for example, construct an internal
   data structure used to access the phonemes for a given word.(On
   this I am not sure and want to check before I give an official
   review.)

Comment 5

[André Natal]

> 
> 1. It looks like the dictionary was modified by someone already, our
>    "copy" has firefox as the first entry.


Yes, I added this word. And as you just copied the patches I wrote, you received this modification too.



> 2. All entries, other than firefox and the new numerical values, are
>    in alphabetic order.

Yes, they are.



> 3. The dict code in pocketsphinx may require the entries to be in al-
>    phabetic order, 

Clearly don't require, otherwise wouldn't be working at all, since "firefox" is already the first word since one year ago.

> in order to, for example, construct an internal data structure used to access the phonemes for a given word.(On this I am not sure and want to check before I give an official review.)

The internal structure to store the dictionary is a hashtable so clearly the words in the dict file does not need to be in alphabetical order

Comment 6

[kdavis]

Comment on attachment 8631981 [details] [diff] [review]
This patch adds digits in numerical form to standard pocketsphinx english dictionary

Review of attachment 8631981 [details] [diff] [review]:
-----------------------------------------------------------------

After looking at the Pocketsphinx code, it looks like adding
entries in non-alphabetical order is fine.

Also, it seems as if adding numerical entries is also, as far
as the dictionary is concerned, OK as it seems the keys are
uninterpreted. Thus is doesn't matter if they are numeric or
alphabetic.

A few other entries in the dictionary are not in alphabetic order
according to "sort u" in "vim" in my default locale. However, I
think we should alphabetize the entire file if for nor other reason
than helping us to not enter the same word twice.

Andre could you open another Bug and assign to to yourself,
whose sole purpose is to alphabetize the dictionary file by
the grapheme?

Comment 7

[kdavis]

FYI CMU already has a tool for, among other things, alphabetizing
such dictionaries https://github.com/cmusphinx/cmudict-tools

Comment 8

[André Natal]

> Andre could you open another Bug and assign to to yourself,
> whose sole purpose is to alphabetize the dictionary file by
> the grapheme?

Sure, I can do that.

Comment 9

[André Natal]

(In reply to kdavis from comment #7)
> FYI CMU already has a tool for, among other things, alphabetizing
> such dictionaries https://github.com/cmusphinx/cmudict-tools

Yes, I know that. Thank you.

Comment 10

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/26346ab6682c

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/26346ab6682c