972884 - Investigate tools for localization

Status: VERIFIED FIXED

(Hello (Loop) :: General, defect)

Description

[Mark Banner (:standard8)]

We need to investigate tools for localization of web pages and consider what we need to make localization possible for the web pages. Suggested places to look are:

- Persona
- Marketplace
- Gaia

Comment 1

[Alexis Metaireau (:alexis)]

ccing Kazé as he worked on that for the Gaia team.

Comment 2

[Mark Banner (:standard8)]

Background from Persona:

Current: https://github.com/mozilla/persona/blob/dev/docs/I18N.md
Previous blog posts:
https://hacks.mozilla.org/2013/04/localize-your-node-js-service-part-1-of-3-a-node-js-holiday-season-part-9/
https://hacks.mozilla.org/2013/04/localization-community-tools-process-part-2-of-3-a-node-js-holiday-season-part-10/
https://hacks.mozilla.org/2013/04/localization-in-action-part-3-of-3-a-node-js-holiday-season-part-11/

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Just spent a bunch of time chatting with Kazé about this.  Here's my summary; (Kaze, feel free to correct any mistakes).

B2G uses <https://github.com/fabi1cazenave/webL10n>, which Kaze wrote, and is also in use by regular web apps and pdf.js.  pdf.js is a particularly interesting case, in the sense that it's built into both Firefox and B2G.  It uses .properties files, like Firefox Desktop, though extended a bit, in a backwards compatible way, to do better with plurals.

It can do simple variable substitution (and can be abused as a sort of HTML templating solution, but this isn't generally encouraged). After talking with him more, I'm wondering if the templating question belongs as much or more with our MVC solution than it does here.  For now, I'll find or file a bug about that.

Kaze had the sense that the localizers really disliked the .po format used by the node tools, which seems to contrast with at least the implications suggested in the persona posts.

One thing that's not obvious to me is whether we'll end up wanting to localize any strings at all on the server.  I.e. even for static content served up from the web like the link click page, and in some cases the conversation window, it looks like the stuff at <https://github.com/fabi1cazenave/webL10n/wiki/Language-Selection> ought to work.  Any thoughts here?

Next step is for me to talk to one of the pdf.js guys.

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

Talked to bdahl.  He said that Yury did most of the work, but bdahl's take was that webl10n.js had worked great for them, and he had no complaints.  I believe Kazé said that they had never needed to upgrade to the newest versions, as an older version was working just fine for them.

The way it works in the desktop client is that there is a small bit of chrome-privileged code that loads up the .properties file in <http://mxr.mozilla.org/mozilla-central/source/browser/extensions/pdfjs/components/PdfStreamConverter.js#314>.  Then, 
<http://mxr.mozilla.org/mozilla-central/source/browser/extensions/pdfjs/content/web/l10n.js> runs in unprivileged content space and uses the "FirefoxCom" object to to trigger special events that the chrome is looking for.

Comment 5

[Dan Mosedale (:dmosedale, :dmose)]

Also relevant is that there are two slightly different APIs, one of which is optimized for FirefoxOS by leaving out few polyfills and uses getters and setters.  Kazé suggested that since we're requiring browsers new enough to include WebRTC anyway, we'd likely be able to use that version if we want.

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

My gut so far, is that we should probably just start with this, and then investigate the other options if it turns out to be problematic.

Though I think we do want to have a bit more sense of how it'll interact with whatever MVC thingy we end up using before finalizing any choice.

Comment 7

[Dan Mosedale (:dmosedale, :dmose)]

Note that I also talked to another b2g engineer (mcav) who was doing gaia front end apps using this framework, and he seemed happy with it as well.

Comment 8

[Dan Mosedale (:dmosedale, :dmose)]

Since (presumably some of) the b2g apps are pure content and have no file or chrome privs, they get their locale files by XHRing them from the localhost (so they're available when offline as well).

Comment 9

[Dan Mosedale (:dmosedale, :dmose)]

Adam and I looked it at the code implements this, and it does the localization dynamically by doing a querySelectorAll to find all HTML elements with a specific attribute and then changes their text content by hand.  

Kinda gross, but in talking with jlal and tofumatt, neither of them could recommend anything better, other than doing l10n on the server side, which breaks one of the hypotheses that we want to test: that it makes sense to share the conversation code and markup.

jlal said that for sufficient performance (at least on some b2g phones), they created a tool run from xpcshell that inlines various things:
 
* https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-optimize.js
* https://github.com/mozilla-b2g/gaia/blob/master/Makefile#L451

So there's some risk here (in particular around the standalone version), but webl10n has the property that it appears to have been shown to be at least "good enough" in the environments that we care most about, so it seems like a reasonable start.

Comment 10

[u279076]

(In reply to Dan Mosedale (:dmose - needinfo? me for responses) from comment #9)
> So there's some risk here (in particular around the standalone version), but
> webl10n has the property that it appears to have been shown to be at least
> "good enough" in the environments that we care most about, so it seems like
> a reasonable start.

What's the risk here? Does this need QA attention before we release?

Comment 11

[Dan Mosedale (:dmosedale, :dmose)]

The risk was mostly that it wouldn't prove to be suitable for our needs.  I don't think this needs more QA attention.

Comment 12

[u279076]

Thanks Dan, assuming verified fixed. Please needinfo me if this changes and needs testing.