940041 - Add Esperanto letters with diacritics to the keyboard layouts

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Keyboard, defect)

Description

[hugues de keyzer]

None of the currently provided keyboard layouts allow to type the Esperanto letters with diacritics. These are: ĉ (c circumflex), ĝ (g circumflex), ĥ (h circumflex), ĵ (j circumflex), ŝ (s circumflex) and ŭ (u breve) (and their upper-case equivalent).

As far as I know, there is no standard Esperanto keyboard layout. As Esperanto is normally used as a bridge language, it would probably make sense if people using it would be able to type it with the keyboard they are used to (the one of their first language).

I think that adding these characters to the alternate characters (the ones that appear with a long press) of (at least) the latin layouts would be a good solution.

Comment 1

[hugues de keyzer]

Attachment: GitHub Pull Request #13818

Comment 2

[hugues de keyzer]

I’m not sure about the place of these characters in the lists of alternate characters. I used the Unicode code point order, after the characters used by the language of the layout. For example, I put ĉ (c circumflex) after ć (c acute) and before č (c caron) in most layouts, following the Unicode order, but for the Czech layout for example, I put it after č (c caron) because the latter is used in the Czech language.

I had to reorder the characters in the tr-Q layout so that the Turkish letters appear first (which I think is desirable :)).

Moreover, the lines of the alternate characters seems to be strangely sorted. In some layouts, the vowels appear first, then the consonants (but not in alphabetic order). What is the logic behind this?

Comment 3

[hugues de keyzer]

Attachment: GitHub Pull Request #13818 (for v1.2)

Comment 4

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Comment on attachment 8335515 [details] [review]
GitHub Pull Request #13818 (for v1.2)

Hi Hugues,

First of all, thanks for your interest in Firefox OS.

I think it would be good to add support of Esperanto to the OS.
However, I don't think it is a good idea to make all these alternative character changes into each of the layouts of various languages, since most people using those languages may not know Esperanto.

Can you help us do some study on how other OS/system handle this specific case typing Esperanto, sine I am not sure we have l10n community for that?

Thanks.

Comment 5

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Comment on attachment 8334196 [details] [review]
GitHub Pull Request #13818

I'll clear the review flag first and please flag me or David again after we have a better idea to handle Esperanto.

Comment 6

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Arky,

I don't think we have a community of Esperanto, do we?

Comment 7

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

BTW, I found that there is Esperanto layout in Google stock keyboard.
I am testing Android 4.3.

Maybe we can create a specific keyboard layout for Esperanto?

Comment 8

[arky [:arky]]

(In reply to Rudy Lu [:rudyl] from comment #6)
> Arky,
> 
> I don't think we have a community of Esperanto, do we?

Yes, We do have a Mozilla Esperanto(eo) L10N team. 
https://wiki.mozilla.org/L10n:Teams:eo

Comment 9

[hugues de keyzer]

Hi Rudy,

First of all, thank you for your answer and your time reviewing my proposed changes. I think Firefox OS is a really great initiative.

I understand your concern about adding these characters into nearly every keyboard layout, as it indeed seems odd, especially since Esperanto users are not really the majority of the users :). The idea came when I noticed that most keyboard layouts include many characters that are not part of the corresponding language and that most users will most likely never use. For example, on the French keyboard: áãäåāćčēìíīńñňòóõöōøśšşßúū.

Even if useless for most users, these characters can be handy for people using mostly one language and occasionally some others that require some of these characters, as they will not have to switch to another keyboard, which could be very different from their usual keyboard.

This is exactly the use case of Esperanto, which is mostly used occasionally by most of its users.

Is there some logic behind the list of extra characters included in the layouts? Is it maybe based on some usage statistics?

This said, I think a better solution would probably be to let users add extra characters to their keyboard in the preferences, but that would probably be more challenging to implement while keeping it simple to use.

About the support of typing Esperanto on other systems, I think that the keyboards of iOS and Windows Phone don’t support it (though there are apps that allow to type and then copy the text). I just found a request for Windows Phone about this: [http://windowsphone.uservoice.com/forums/101801-feature-suggestions/suggestions/3844118-add-esperanto-accents-to-press-and-hold-letter-opt].

On Android, there is indeed an Esperanto keyboard since version 4.2. It is based on a modification of the US QWERTY layout, and is similar to the one found in X.org (symbols/epo). Here is a link to the change in the Android codebase (with discussion and patches): [https://android-review.googlesource.com/#/c/30970/]. This keyboard actually includes a lot of characters (MES-1, as explained in [http://www.evertype.com/standards/iso10646/pdf/cwa13873.pdf]), allowing to cover at least 44 languages. Adding the same keyboard to Firefox OS would likely be possible.

Another solution would be to allow “dead” accents. With a dead circumflex, it would be possible to type 5 out of the 6 characters missing for Esperanto.

What do you think would be the best solution? Nothing prevents even to have multiple solutions, like adding the keyboard as well as extra characters to the other layouts, to leave the choice to the user.

Anyway, I’m ready to help.

Comment 10

[hugues de keyzer]

Tim,

I saw that the lists of alternate characters with diacritics were added by you in 9c1e85a0d22df66739f891788c699a231c88e708. Could you please explain how you chose which characters to include?

Thanks.

Comment 11

[Tim Guan-tin Chien [:timdream] (please needinfo)]

You mean https://github.com/mozilla-b2g/gaia/commit/9c1e85a0d22df66739f891788c699a231c88e708 ?

That was almost 2 years ago and B2G was an experimental project. We do whatever we want to justify the project, and for this case I introduce a feature w/o UX design.

I simply came up the characters by looking at Unicode chart and mobile phones happens to be around.

In any case, if you are looking for an approval of symbols to include in the layout, you should ask the UX team. But given the international nature of this feature and our limited resource, I would say we would probably do whatever the localization team say so.

Comment 12

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Correcting flags.

Comment 13

[Eduardo Trápani]

The patches look fine to me (Esperanto localization team). Adding a handful of alternatives is the way to go right now. I asked Pike to review them.

Comment 14

[Axel Hecht [:Pike]]

Comment on attachment 8334196 [details] [review]
GitHub Pull Request #13818

Moving the review request over to UX, I'm not the right person to make a call here.

If it is OK to add these characters, I'd probably expect them to be appended to the lists, I've seen at least one being injected.

My personal opinion would be that it might be useful to enter names, but I can't evaluate how that affects the usability of the keyboards outside of that use case.

Comment 15

[Axel Hecht [:Pike]]

Comment on attachment 8335515 [details] [review]
GitHub Pull Request #13818 (for v1.2)

Not sure if we need a 1.2-specific PR at all.

Comment 16

[Stephany Wilkes]

Flagging Mike and Carrie, who own the keyboard.

Comment 17

[Carrie Wang [New account: [e-mail]]]

Hi everyone, 

We'll note down this issue, but from UX perspectives, I'd suggest that we shall focus on refining the current keyboard design first. Once we complete the basic user requirements, we can then take a step further to integrate this kind of "nice to have" feature. :)
Thanks!

Comment 18

[GoodMike]

Thanks for the suggestion and Carrie has given the comment.

Comment 19

[Aaron Irvine]

Hi. Some time ago I wrote the keyboard extension abcTajpu for Firefox, and have been involved in the Esperanto keyboards for Android and for Chromium, so I'm very interested in this issue.


Consider the current English keyboard layout, en.js, here for example there is a mapping c:'çćč', but there is no reason other than historical accident that ç and ć and č appear but ĉ does not.  How can we be objective as to which characters appear and which don't?  Maybe base the decision on the larger Wikipedia languages, or on the Google translator languages, or all Latin-script languages with more than say a million speakers?  Happily that difficult decision has been made for us in CEN WORKSHOP AGREEMENT CWA 13873:2000 (Multilingual European Subsets in ISO/IEC 10646-1), where we can choose level MES-1 namely àáâãäåæçèéêëìíîïðñòóôõöøßùúûüýþÿāăąćĉċčďđēėęěĝğġģĥħĩīįıijĵķĸĺļľŀłńņňʼnŋōőœŕŗřśŝşšșţťŧțũūŭůűųŵŷźżž to be the letters supported (level MES-1 rather than MES-2 or MES-3 so that there are no more than 10 characters on each popup).  I think you should revisit https://github.com/mozilla-b2g/gaia/pull/13818 and include all these characters from MES-1 and then please merge into mainline.

alt: {
    a: 'àáâãäåæāăą',
    b: '()[]{}<>',
    c: 'çćĉċč¢',
    d: 'ðďđ',
    e: 'èéêëēėęě€',
    f: '=+-*/\\^~|\'',
    g: 'ĝğġģ',
    h: 'ĥħ',
    i: 'ìíîïĩīįıij',
    j: 'ĵ',
    k: 'ķĸ',
    l: 'ĺļľŀł£',
    m: '+-*/^=%µ¤',
    n: 'ñńņňʼnŋ',
    o: 'òóôõöøōőœ',
    p: '&_#%₱',
    q: '@ªº«»ɛɵ',
    r: 'ŕŗř₹',
    s: 'śŝşšșߧ$',
    S: 'śŝşšș§$',
    t: 'þţťŧț',
    u: 'ùúûüũūŭůűų',
    v: 'δφγλωπψσθξ',
    w: 'ŵ',
    y: 'yýÿŷ¥',
    z: 'źżž',
    '.': ',¿?¡!;:"'
  },


While it is true that Esperanto is normally used as a bridge language, and it therefore makes a lot of sense if people using it would be able to type it with the keyboard they are used to (the one of their first language), and so adding ĉĝĥĵŝŭ (and indeed all of MES-1) to the alternate characters (the ones that appear with a long press) of (at least) the Latin layouts would be a good solution, I think that in addition, for power users of Esperanto, FirefoxOS should support the standard Esperanto keyboard layout (see for example Ubuntu or Android).

I have submited pull-request https://github.com/mozilla-b2g/gaia/pull/16921 to support this standard layout - it renders as screen shot http://lingvo.org/klavaro/firefoxos.png (later, I think keyboard_layouts.json and keyboard.js will also need amended so eo.js can be selected as a keyboard).

Thanks.

Comment 20

[Jan Jongboom [:janjongboom] (inactive)]

+9999 to add this stuff to all keyboards. Let's ask UX.

Comment 21

[Carrie Wang [New account: [e-mail]]]

ni? Omega since he's taking care of KB layout design. Thanks!

Comment 22

[Omega Feng [:Omega] [:馮於懋]]

Since Esperanto seems an important language, I suggest adding a new IME for Esperanto rather than adding Esperanto letters into all IMEs. However I don't read/speak Esperanto at all, so, Stephany, could you ni? to another UX designer who knows Esperanto to help review this bug? Thanks!

Comment 23

[Aaron Irvine]

Thanks.  The new IME for Esperanto is proposed here https://github.com/mozilla-b2g/gaia/pull/16921 and this implements the standard Esperanto layout.  It just needs reviewed and pulled.  (It creates a keyboard layout that looks like http://lingvo.org/klavaro/firefoxos.png)

Regarding the second suggestion, that the other IME's be enhanced by adding the characters from CEN WORKSHOP AGREEMENT CWA 13873:2000 (Multilingual European Subsets in ISO/IEC 10646-1) level MES-1 namely àáâãäåæçèéêëìíîïðñòóôõöøßùúûüýþÿāăąćĉċčďđēėęěĝğġģĥħĩīįıijĵķĸĺļľŀłńņňʼnŋōőœŕŗřśŝşšșţťŧțũūŭůűųŵŷźżž, well I still think this should be considered.  It solves problems not just for Esperanto, but for others eg Welsh.

Comment 24

[Stephany Wilkes]

Comment on attachment 8334196 [details] [review]
GitHub Pull Request #13818

I am changing the review? flag for the UX team (which we cannot take; only engineers can take review? flags) and flagging Carrie for ui-review? on the patch. No one on our team speaks the noted languages, but we'll do our best on ui-review? and hopefully make a recommendation on the IME.

Comment 25

[Aaron Irvine]

Not sure the procedure here.  I think
1) someone assigns this ticket to UX team (Stephany can you reassign it?)
2) UX team look at suggested patch for Esperanto keyboard layout (comparing aganst Ubuntu/Redhat/etc and against Android).  Don't see any problems here, the Esperanto layout is well enough established.
3) UX team consider the additional proposal to enhance the long press character lists on all existing Latin-based keyboards with the accented letters from CEN WORKSHOP AGREEMENT CWA 13873:2000 (Multilingual European Subsets in ISO/IEC 10646-1) level MES-1.  This MES-1 is not a mandatory agreement, it's only a recomendation, but it's the right and couragous way to go for a more open web.
4) UX team to reassign back to engineering? to test that the patch installs and functions correctly (although modify patch first if required based on UX team feedback).
5) finally integrate :-) and announce to Esperanto community.

Comment 26

[Omega Feng [:Omega] [:馮於懋]]

Comment on attachment 8334196 [details] [review]
GitHub Pull Request #13818

Works fine with the extra characters:
ĉĝĥĵŝŭ
ĈĜĤĴŜŬ

Comment 27

[Tony Mechelynck [:tonymec]]

(In reply to Omega Feng [:Omega] from comment #22)
> Since Esperanto seems an important language, I suggest adding a new IME for
> Esperanto rather than adding Esperanto letters into all IMEs. However I
> don't read/speak Esperanto at all, so, Stephany, could you ni? to another UX
> designer who knows Esperanto to help review this bug? Thanks!

The way I see it, since most Esperanto speakers use a different language for school, work, shopping, etc., they can be expected to be more familiar with the keyboard layout used in that language than with any specific "Esperanto" layout. OTOH, different "national" layouts can be more or less well suited to the Esperanto "ĉapelitaj literoj" (wfw. "hat-wearing letters"): at one end of the spectrum, keyboards for English-only (US, GB, etc.) probably have no diacritics at all, while at the other end, keyboards used for the French language (French AZERTY, Belgian AZERTY and Swiss QWERTZ) all include a "circumflex" dead key used for â ê î ô û (and which could be used for ĉ ĝ ĥ ĵ ŝ too) and a lowercase ù (u-grave) which was used as a fallback for the missing ŭ (u-breve) in the time of typewriters: so I expect that "power users of Esperanto" whose "city language" is French would use their French-language keyboard layout to type Esperanto. See for instance at http://users.skynet.be/antoine.mechelynck/other/keybbe.htm the Belgian keyboard layout on my Linux desktop. The circumflex-deadkey, which requires no modifier, is on the right of the P; and the breve-deadkey (harder to use, but of no use in French, Dutch or German, haha), is AltGr+Shift+£ using the key which, depending on the hardware, is either at top right next to the Bsp key, or two rows lower in the notch of the "small" variant of the Enter key.

Comment 28

[Benno Schulenberg]

Chiming in, for what it's worth: I think that comment #19 is the way to go: both add all the MES-1 characters to all the IMEs, and add the Esperanto "standard" keyboard.

What's the status on this?  Has this bug reached the right team yet?

Comment 29

[Rabbi Hossain]

(In reply to Omega Feng [:Omega] [:馮於懋] from comment #22)
> Since Esperanto seems an important language, I suggest adding a new IME for
> Esperanto rather than adding Esperanto letters into all IMEs.

As :Omega is talking about creating a new IME for Esperanto. I would like to help here in creating Esperanto IME for Firefox OS. Is anyone working on it? Or if the team has decided to Add an Esperanto IME for Firefox OS?

If everything goes positive, I would like to work on it. :Omega, Can you give me info regarding this? I am adding needinfo request to you.

Comment 30

[Omega Feng [:Omega] [:馮於懋]]

Hi Rabbi,
Rudy, the mentor of this bug, is the better one to guide you here. :)

Comment 31

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

In general, please refer to the following link about how to contribute to Gaia,
https://github.com/mozilla-b2g/gaia/blob/master/CONTRIBUTING.md

It seems we already have a patch here, is this ready for review?
https://github.com/mozilla-b2g/gaia/pull/16921

I would find some time to take a look.

Comment 32

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Attachment: Patch to add Esperanto layout

Comment 33

[Aaron Irvine]

Yes, https://github.com/mozilla-b2g/gaia/pull/16921 is ready for review please.

Comment 34

[Jan Jongboom [:janjongboom] (inactive)]

This patch contains a reference to the Esperanto dictionary as well, but as far as I know we don't have this one. Do we want to add that as well?

Comment 35

[Aaron Irvine]

You're referring to autoCorrectLanguage:'eo', well that's a good point (it's the one line I'd didn't test, hoping someone else knew how to).  There is a dictionary that works fine with the latest Firefox browser https://addons.mozilla.org/en/firefox/addon/esperanta-vortaro/ (you force install it) - can this be brought into Firefox OS, or is it more complicated than that to get it working with the keyboard?

Comment 36

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Comment on attachment 8487834 [details] [review]
Patch to add Esperanto layout

I've reviewed this newly added layout, and it looks good to me in general.
Also left some comments on the github pull request, please help take a look at them and address them in the next update if you agree.

For auto-correction, we would need a word list, see the .xml files in 
https://github.com/mozilla-b2g/gaia/tree/master/apps/keyboard/js/imes/latin/dictionaries

As commented on the github, we could focus on the layout file in this bug, and handle the word list/dictionary in a follow-up bug.

Thanks.

--
Please ask me for review when you update this patch or needinfo me if any questions.

Comment 37

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Comment on attachment 8487834 [details] [review]
Patch to add Esperanto layout

Aaron, I've seen you updated your patch, is this ready for 2nd review?
Thank you.

Comment 38

[Aaron Irvine]

Hi,

Resent the pull request for eo.js (it has all the review comments addressed).

regards,
Aaron

Comment 39

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Comment on attachment 8487834 [details] [review]
Patch to add Esperanto layout

Put this in my review queue.

Comment 40

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

Attachment: Patch squashed

Just created another pr to squash and amend some changes.

Comment 41

[Rudy Lu [:rudyl] (inactive after 2015/7/3)]

It's all good now, so land it,
https://github.com/mozilla-b2g/gaia/commit/2665e714beea5dc433862ca6bb8d2b47ffe2f2d1

--
Thanks for the contribution!