Closed Bug 706572 Opened 13 years ago Closed 12 years ago

Document best practices for localizing node.js services

Categories

(Mozilla Localizations Graveyard :: Documentation, defect)

Product:
Mozilla Localizations Graveyard
Component:
Documentation
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: Pike, Assigned: gueroJeff)

References

Details

Attachments

(2 files)

Screenshot of signin dialog localized to fake pl
132.14 KB, image/png
Details
BrowserID in db-LB
67.06 KB, image/png
Details 
browserid is our first site to be based on node.js.

Let's document what we do and why that works good or OK as we learn what we're doing.
Assignee: nobody → jbeatty
(In reply to Axel Hecht [:Pike] from comment #0)

Maybe you can re-work https://wiki.mozilla.org/L10n:Web_frameworks.

If the way you document this isn't that page, we can redirect it to your solution that closes this bug.
CCing Rik

Detailed work log is in https://etherpad.mozilla.org/localize-browserid

Update: I'm working with @Matjaž in two phases:
1) System perspective, making Node app localizable
2) Wrapping all strings in the app with gettext, extracting/merge po files

In first step we'll figure out which Node modules can be re-used and build missing glue based on sample set of template/JS files.

In second step we'll get the app codebase ready for the L10n community to contribute via PO files in SVN.
I briefly chatted with Austin, just to dump this somewhere in writing:

It might be easier to localize the templates server-side instead of doing both the templating and the localization in client js.
@mathjazz, llyod, stomlinson based on our work in the etherpad, I've got a proof of concept hacked together for the sign-in dialog localized with fake (sorry) Polish. See attachment.

Proposed changes to browserid to support localization:

EJS templates and node.js JavaScript code will remain as is, but any user facing strings will be wrapped in gettext or ngettext. English strings will be used for msgids.
Example: <p><%= gettext('Please, enter an Email address to continue.') %></p>

Two new bash scripts in the browserid repo will be used to manage creation of PO files.
scripts/extract_po.sh
scripts/merge_po.sh

These are run at string freeze or periodically to get new strings.

locale would be a SVN repo checked out from 
http://svn.mozilla.org/projects/l10n-misc/trunk/browserid/locale

The output of these scripts goes into the typical svn locale layout. There are also two scripts that live in the SVN repo:

locale/compile_json.sh – string freeze time (or whever po files change), this script is run and the output is committed to the git repo (details below).

locale/compile_mo.sh – deployment time “build” step, compiles PO into MO files. MO files are not checked into SVN.

New node.js dependencies:

node-gettext and iconv (a C node.js module) – Server side gettext will read strings from MO files. This will work for strings in JavaScript or EJS templates.

New client-side JavaScript dependencies:

Gettext.js and po2json.pl from the BerliOS project. (We'll probably fork and remove un-needed code from Gettext.js).

Machines which can run po2json.pl will need Perl and two CPAN libraries. This doesn't have to be done by browserid dev team and a post-commit hook could be setup to automate this. 

Usage:
compile_json.sh ./locale ./resources/static/i18n
For each locale the script creates a JSON formatted bag of strings that Gettext.js understands.

Example filename:
resources/static/i18n/pl/messages.json
These strings are used by client-side EJS template rendering as well as client-side JavaScript which contains strings.

EJS Templates:
EJS use gettext, ngettext, etc as well as strargs on the client side and util.format on the server side for string interpolation.

Server-side Examples:
<%= _('Hello') %>
<%- gettext(“I have <em>Embeded</em> HTML”) %>

Client-side Examples:
<%= strargs(gettext('Hello %1'), 'World') %>

I'd recommend using same methods on client and sever EJS templates to improve consistency.

Note: _ is already used in client side JavaScript and EJS templates, so gettext or another alias should be used. (Or we can noConflict underscore.js).

New Middleware:

Server-side we'll detect the user's preferred locale by looking at the URL, a path limited Cookie, and the request's accept-language header.

Current locale will be detected and used to setup node-gettext for server side gettext.

Client side gettext will be setup via:

In development mode, the dialog template will include a script file
/i18n/messages.json
which will redirect to
/i18n/<locale>/messages.json

In production mode, we'll include Gettext.js and messages.json in a performance sensitive way. Team is revisiting cache busting, which will inform this solution.

This middleware will be controlled by a new configuration parameter:
production.locales = ['en_US', 'pl', 'hu']
(or some such in lib/configuraiton.js)

Much of this proposal re-uses existing code, but none of the existing node.js modules for middleware are written with Mozilla l10n idioms in mind; Filling the gap won't be much code, so we'll either fork connect-i18n or write from scratch.

I think that covers most of what we'll need to localize the dialogs.

For more details on what was considered, but not chosen please see the updated etherpad.

How do we feel about this direction? Anything missing? Issues?
As I already said at the meeting today, I'm really impressed by the progress you made. Great job!

To test i18n support for the BrowsearID service, me and Axel agree we should localize a few dialogs with a fake locale. Perhaps Axel can say something more about translate toolkit, which creates similar translations.
I was thinking of podebug, http://translate.sourceforge.net/wiki/toolkit/podebug. I'll read and comment more.
I've found https://github.com/mozilla/browserid/blob/dev/lib/browserid/prove_template.txt, which seems to be unhandled at this point?

Anywho:

I'd love to join a call of you guys to get a verbal update. The etherpad is useful, but I'm better digesting information aurally.

I'm particularly wondering about the combination of l10n on both client and server, even more so if we're doing them slightly different on the js side, ie. local gettext library on the server vs json on the client.
(In reply to Axel Hecht [:Pike] from comment #8)

> I've found https://github.com/mozilla/browserid/blob/dev/lib/browserid/prove_template.txt, 
> which seems to be unhandled at this point?

Good call, I didn't spend much time on email. I'll look into how those templates work.

> I'd love to join a call of you guys to get a verbal update.

We're in IRC in #identity. I'll look for you, maybe we can Skype today.
Attached image BrowserID in db-LB — 
(In reply to Axel Hecht [:Pike] from comment #7)

podebug is very useful! See attachment.

Adding a new debug locale 'db-lb' "David Bowie - Labyrinth"

to trigger debug mode, Esperanto on development servers will switch to db-lb instead of 'eo'.

As noted in https://github.com/mozilla/browserid/issues/873 making db-lb a RTL locale, to add to disorientation for westerners.
Adding i18n to wsapi for email.

Organized strings into 
* messages.po - Node.js strings
* client.po - Client-side strings
                                                                                             
Switched email from mustache to EJS templates.                                                                             
                                                                                             
Created fake locale db-LB for development and debugging.       

Updated extract, merge, compile scripts

SVN updated
http://viewvc.svn.mozilla.org/vc?view=revision&revision=99911

Github updated:
https://github.com/ozten/browserid/commit/2c5e62c3bab7847b5d360f2f3593157e3baa7f5e
Here is a short no-audio screencast to show which screens we've wrapped with gettext while implementing the i18n portion of this bug.
http://ozten.com/random/browserid/browserid-i18n.ogv - Loads directly in Firefox.

Code is currently under review at
https://github.com/mozilla/browserid/pull/897
Update:

We're done with the i18n phase and things are looking good, so far.

Please try it out on dev
1) Set your preferred locale to Esperanto*
2) http://dev.myfavoritebeer.org/ and Sign in

If you see English strings in a couple places, we are still working out caching and deployment bugs, probably Monday.

This current release train will be 2 weeks, instead of 1 week as previously discussed.

* Until we have a language picker, we need a locale to trigger our easter egg db-LB locale.
In FireFox on Mac, selected Esperanto [eo] as my preferred (first pick) language.
Opened http://dev.myfavoritebeer.org/ and signed in with a known account.
I see nothing localized.

Same result for Chrome on Mac.

Restarted each browser just in case I needed that for setting locale.

PS: dev.myfavoritebeer.org + Safari is a sad face (I can not even Sign In)
@mathjazz has us in the homestretch here...

Temporary preview environment - http://temporary_preview.myfavoritebeer.org/

Localizers can start testing their locales on this temporary server.

Progress - https://localize.mozilla.org/projects/browserid/

A permanent l10n preview environment is being built, but due to lack of IP Addresses isn't ready. We're on schedule for a Feb 8th launch of signed off locales.
Depends on: 723775
Most locales are enabled on stage, we should test the deployment per Bug#723775

Live update is still only on temporary_preview.
We are scheduled to ship to produciton either next Wed 2/15 or Thursday.

All locales that are 100% translated will be enabled.
And where live in production with:
  "ca", "cs", "da", "de", "el", "en-US", "es", "et", "eu", "fi", "fr",
    "fy", "ga", "hr", "it", "lij", "nl", "pa", "pl", "ru", "sk", "sl",
    "sq", "sr", "sv", "tr", "zh-CN", "zh-TW"

Doing push verification, but marking bug fixed.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: Mozilla Localizations → Mozilla Localizations Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: