1385405 - Move locale files from kuma to a new repository

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Code Cleanup, enhancement)

Description

[John Whitlock [:jwhitlock]]

What feature should be changed? Please provide the URL of the feature if possible.
==================================================================================
gettext-style locale files are currently stored in the kuma project [1]. They should be moved to their own repository [2]


[1] https://github.com/mozilla/kuma/tree/master/locale
[2] https://github.com/mozilla-l10n/mdn-l10n



What problems would this solve?
===============================
Pontoon commits new translations to the repository. This has a few negative side effects:

* It can be challenging to see code changes, as the commit history is dominated by translations
* Builds are triggered in TravisCI and Jenkins, increasing load on those services, to test unrelated features.
* Some changes allowed by Pontoon break the build, and require manual intervention
* String updates create massive changesets in the kuma repo
* The Pontoon process requires push access to the kuma repo, without human code review

In addition, a separate repository would allow us to collect translatable strings from several MDN projects, and standardize on Pontoon for the one place for localizers (bug 1340342)

Who would use this?
===================
Localizers and developers

What would users see?
=====================
Localizers would continue to use Pontoon to localize MDN strings.

MDN users would continue to see the latest translations after a production push.

What would users do? What would happen as a result?
===================================================
Developers will need to update some processes and documentation

1) The locale submodule will need to be updated to latest and tested before a production push
2) The locale submodule will need to be updated to latest and then populated with UI string changes
3) Locale submodule updates will eventually need to be added to continuous deployment pipelines

Is there anything else we should know?
======================================
A locale submodule was originally used in the kitsune codebase (2010). The submodule was removed in 2011 (bug 625755) [3]. The locale files were stored in the Verbatim repo, and were added to development and deployment environments as a manual installation and upgrade step.

In 2015, the locale files were added to the repository as part of bug 1209185 [4]. Translations were switched to Pontoon.

Now we are returning to a submodule and process. This is known as the cycle of reincarnation or the Wheel of Samsara.

[3] https://github.com/mozilla/kuma/commit/ff0babf119e95a9f8b01a359842564a520bb568c
[4] https://github.com/mozilla/kuma/pull/3575
[5] http://www.hacker-dictionary.com/terms/cycle%20of%20reincarnation

Comment 1

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/bbede8a0298bcb45abcc92a5193d7d02e7fbc95b
bug 1385405: Move compile script to scripts folder

https://github.com/mozilla/kuma/commit/59ed7fefebf2a59942dcb7d22d77086df34b8c73
bug 1385405: Remove built l10n files from .*ignore

Continue excluding some in .dockerignore, to force rebuilding for
production images.

https://github.com/mozilla/kuma/commit/6513f6008736e225df3e06ea363fb38aeb9d2167
bug 1385405: Remove locale files

Locale files are moving to https://github.com/mozilla-l10n/mdn-l10n

https://github.com/mozilla/kuma/commit/9e8f8c9b83e1258f23f02b3350264d7711534aef
bug 1385405: Add locale as a submodule

https://github.com/mozilla/kuma/commit/e65e78cdee347d28f5d90a42a2f8ae730a14af57
bug 1385405: Mark TOXENV=locales job as required

Since Pontoon is no longer pushing directly to Kuma, the TOXENV=locales
should always pass, or be fixed before updating Kuma.

https://github.com/mozilla/kuma/commit/3b567e1a9bc96b3ff0146c74166721b5106dbe3a
bug 1385405: Update localization documentation

Update docs now that locale is a submodule loading
https://github.com/mozilla-l10n/mdn-l10n

https://github.com/mozilla/kuma/commit/6a1a6d19e3204d2b0ed12c8118f334d30e836173
bug 1385405: Update SCL3 deployment

Update the SCL3 deployment for the locale submodule.

https://github.com/mozilla/kuma/commit/cbf7f254247e9ac43adf8c1700e2d73a4a666f6c
Merge pull request #4338 from jwhitlock/locale-submodule-1385405

bug 1385405: Move locale files from kuma to mdn-l10n repo

Comment 2

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/86583c5c6b1dd02e49ccad171fb0d04b6d5c6c57
bug 1385405: Remove transitional deploy step

This worked locally for being able to checkout the repository after
switching to locales in a subfolder. It did not work on the SCL3
deployment server. Instead, manually replacing locales/ with an empty
folder, and then running deployment script, for the one-time transition.

https://github.com/mozilla/kuma/commit/a7eb274c51966d1bb6e17a65b78da64029c31856
Merge pull request #4344 from jwhitlock/locale-scl3-1385405

bug 1385405: Remove transitional deploy step

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/d99fc33f2a3b9f6d8ca7213e9cfd4368904ea6c6
bug 1385405: Add doc for deploying to SCL3

We're adding a pre-push tasks of updating the locales submodule, which
gives a good opportunity to document our push proceedures.

https://github.com/mozilla/kuma/commit/218f90155c0835f312d25e034680544146cb5588
bug 1385405: Simple submodule update with --remote

Recommended by @safwanrahman

https://github.com/mozilla/kuma/commit/f574742454dd9ceaee886d0b359771bac0c7ec4b
bug 1385405: Add ways for viewing submodule logs

Create long alias submodule-compare-urls for opening the logs in
GitHub's compare view.

Add a command for seeing just the merge commits, suggested by
@safwanrahman

https://github.com/mozilla/kuma/commit/9222ae5ec2b9a136af5421fc6a9c5a2da16c3b97
Merge pull request #4357 from jwhitlock/deploy-docs-1385405

bug 1385405: Add doc for deploying to SCL3

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/4a5a716a314b0d41f2d642c29126d52090ff78ea
bug 1385405: Update doc for deploying to SCL3

https://github.com/mozilla/kuma/commit/1042de2b845d142e7352bf445b4ca9d7ec18ac50
Merge pull request #4367 from mozilla/1385405-update-docs

bug 1385405: Update doc for deploying to SCL3

Comment 5

[[github robot]]

Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/c3f178e1f444bffb5a46e4d07aaa79c5564fe4de
bug 1385405: Correct URL for production

Comment 6

[John Whitlock [:jwhitlock]]

The locale files are now at: https://github.com/mozilla-l10n/mdn-l10n

I've manually updated them several times in the last few months, bringing new strings into Kuma and pushing changes to English strings to the l10n repo. We haven't automated this process, but that should be covered by a new bug.

Comment 7

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/ac6f48c88c2df2673214b4ed0ab4bdf3cd2df082
bug 1385405: Fix tox -e locales for submodule

The "git diff" command needs to run inside the locale submodule
directory, or it won't see any differences.

https://github.com/mozilla/kuma/commit/3fc72f9109b8251ffaebb370615e35f33fee5556
Merge pull request #4653 from jwhitlock/fix-tox-locales-1385405

bug 1385405: Fix tox -e locales for submodule