1445881 - Move fluent.migrate to its own repo and Bugzilla component

Status: RESOLVED FIXED

(Localization Infrastructure and Tools :: Fluent Migration, enhancement)

Description

[Staś Małolepszy :stas]

The migration code currently lives in projectfluent/python-fluent on GitHub as fluent.migrate. Individual migrations live in mozilla-central/python/l10n/fluent_migrations.

I propose that we move both the code and the migrations to a new repo: hg.mozilla.org/l10n/fluent-migrate

In the early stages of fluent.migrate's development it was useful to have it in the same repository as the rest of the python-fluent code. Both codebases changed hand in hand as the migration logic required some changes to the AST, the parser and to the serializer.

Today, changes to the AST are rare and python-fluent has been on a mature release cycle for a while: https://github.com/projectfluent/python-fluent/releases

By moving the code out of python-fluent, we'll fix the current semi-circular dependency: fluent.migrate depends on compare-locales which depend on fluent.

By moving it into an hg repository, we'll be able to review patches in Bugzilla and we'll make it clear that all migration bugs should be filed as bugs, not GitHub issues. (Currently, it's a mix...).

I also propose that we store migration files (currently stored in https://searchfox.org/mozilla-central/source/python/l10n/fluent_migrations) into the new fluent-migrate repo.

 - It will make it easier to test fluent-migrate on all previously written migrations (and to retroactively edit them if required by a change in fluent-migrate's code).

 - It will also make it easier to run migrations against gecko-strings-quarantine if that's what we'll end up doing.

 - Lastly, AFAICT, the reason to store the migrations in mozilla-central was threefold:

    1) They need to be archived somewhere (which a new repo also addresses).
    2) I thought they would be run via `mach python` (which is currently not the case).
    3) We want to be able to review and land migrations together with the corresponding patch to the Gecko codebase. With a new hg repo for migrations we can still attach migrations to bugs in which we work on migrating parts of Gecko's UI to Fluent.

The migration code has become an important part of the l10n infrastructure. As such, I also propose that we create a new component for it under the 
Localization Infrastructure and Tools product in Bugzilla and move all bugs blocking bug 1367803 to it.

Comment 1

[Francesco Lodolo [:flod]]

Right before Stas filed this bug, I started creating a repo to store my scripts. 
https://github.com/flodolo/fluent-migrations

I lost count of the times I've run the script, to realize that I commented out sections (push, pull, all locales or one), so I decided to make it a little smarter.

You will notice that it also includes the migration files, with paths adapted to gecko-strings.

The reason for this is bug 1417155. In bug 1445084 comment 23 we decided to keep around the removed string, while waiting for a decision. But we have other developers touching FTL files at this point, and I'm not sure the "let's keep strings around" process really scale. 

In case, it's also a quick thing to change.

Comment 2

[Axel Hecht [:Pike]]

I'm all for keeping the migrate code and the migrations in the same repo, but I think it should be mozilla-central instead of a new one.

For open migration, I think it's important that we make writing, reviewing, and testing migrations easy.

Note, that flod doesn't run `mach python` is just his ego about where to set up build environments ;-) .  I totally did use `mach python`.

Comment 3

[Francesco Lodolo [:flod]]

I don't think being able to run the migration independently from mach is too much of a ask. 

Having said that, I would like to understand what's the real benefit of having code and migrations in mozilla-central. The only pros I see are:
* Less confusion on which version of the migration code is where.
* If we start getting other developers contributing migrations, asking them to create a patch for mozilla-central, and another one for an external repository is confusing. But, to be honest, I'm not even sure if it will be on us or them to write such migrations.

On the other hand, having code and migrations in mozilla-central will add confusion:
* Dependencies (python-fluent, compare-locals) would still be developed outside of m-c.
* Given bug 1445001, I think that we need to move to run migrations against gecko-strings-* repository, so something that is not available in m-c anyway.

Comment 4

[Staś Małolepszy :stas]

Coming out of last week's meeting in Ljubljana, we decided to move the migration code out of python-fluent into a new repository at https://hg.mozilla.org/l10n/fluent-migration/. Bug 1449514 was about creating the new repo and bug bug 1450220 tracks bootstrapping its contents.

Migration code bugs and features will be tracked in Bugzilla in the Localization Infra and Tools > Fluent Migration component.

Migration recipes will continue to live in mozilla-central because they are about migrating Gecko's code. This approach scale well: in the future we expect more projects to opt in to use Fluent migrations and our recommendation for them will be to store the recipes in their own repos.

Comment 5

[Francesco Lodolo [:flod]]

I think all the work is done for this bug?

Comment 6

[Axel Hecht [:Pike]]

Yeah.