1569869 - Fluent migration confuses localization targets and sources

Status: VERIFIED FIXED

(Localization Infrastructure and Tools :: Fluent Migration, defect, P1)

Description

[Francesco Lodolo [:flod]]

I'm trying to test https://phabricator.services.mozilla.com/D34379

With master, I can't run the migration at all. The console is clogged with JSON content that looks like HG attribution data.

I've tried to reduce the migration to one string

def migrate(ctx):
    """Bug 1501886 - Migrate browser main menubar to Fluent, part {index}"""

    ctx.add_transforms(
        'browser/browser/menubar.ftl',
        'browser/browser/menubar.ftl',
        transforms_from(
"""
menu-file =
    .label = { COPY("browser/chrome/browser/browser.dtd", "fileMenu.label") }
    .accesskey = { COPY("browser/chrome/browser/browser.dtd", "fileMenu.accesskey") }
"""
        )
    )

And I get this weird error

INFO:migrate:Localization file browser/browser/menubar.ftl does not exist and it will be created
Traceback (most recent call last):
  File "/usr/local/bin/migrate-l10n", line 9, in <module>
    load_entry_point('fluent.migrate==0.6.4', 'console_scripts', 'migrate-l10n')()
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 179, in cli
    dry_run=args.dry_run
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 133, in main
    migrator.run(migration)
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 70, in run
    blame = Blame(self.client).attribution(files)
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/blame.py", line 26, in attribution
    blame_json = self.client.rawcommand(args)
  File "/Library/Python/2.7/site-packages/hglib/client.py", line 263, in rawcommand
    raise error.CommandError(args, ret, out, err)
hglib.error.CommandError: (255, '[', 'abort: browser/browser/menubar.ftl: no such file in rev cfc97e5040c6')

The file is obviously there.

Then I installed a virtualenv, cloned fluent-migration to f837137ba2d853a3337788a32ef2c2deb0d98b3e (before bug 1536507), and migration works.

Comment 1

[Francesco Lodolo [:flod]]

The second error can be replicated with

    ctx.add_transforms(
        'browser/browser/menubar.ftl',
        'browser/browser/menubar.ftl',
        [
            FTL.Message(
                id=FTL.Identifier('menu-sync-sign-in'),
                attributes={
                    "accesskey":COPY("browser/chrome/browser/browser.dtd", "syncSignIn.accesskey")
                }
            )
        ]
    )

Comment 2

[Francesco Lodolo [:flod]]

Ignore the second part, the migration is just wrong. Editing the first comment too for clarity.

Comment 3

[Francesco Lodolo [:flod]]

Adding one more detail: I thought it didn't work because I was migrating from an uncommitted file in the quarantine repository. I tried committing it locally, but it remains broken.

What's weird is that it complains about cfc97e5040c6, but that's behind the tip of the repository, and seems completely random.

Comment 4

[Axel Hecht [:Pike]]

This is the problem I vaguely remembered when landing bug 1536507.

I just didn't remember that I broke everything but fluent-fluent migrations.

Taking a look, if I don't come up with something, I'll back out bug 1536507.

Comment 5

[Axel Hecht [:Pike]]

Attachment: Bug 1569869, keep localization sources and targets apart, r=stas

It used to be OK to use one hash for both source and target
ASTs, but now that we migrate from ftl, too, we can't keep them
divided by file name. Let's create distinct dicts for both.

Comment 6

[Axel Hecht [:Pike]]

OK, this should be fixed now in https://hg.mozilla.org/l10n/fluent-migration/rev/acd02d83e0398dbd5d57925cd11b44f096323741.

I did do the test case first, which broke in the way this bug describes, and it doesn't break anymore ;-)

Comment 7

[Francesco Lodolo [:flod]]

Works locally.