1521402 - migration-l10n python tool uses methods deprecated in python3

Status: RESOLVED FIXED

(Localization Infrastructure and Tools :: Fluent Migration, defect)

Description

[Ian Kirkpatrick [:ink]]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:64.0) Gecko/20100101 Firefox/64.0

Steps to reproduce:

followed installation guide listed at https://firefox-source-docs.mozilla.org/intl/l10n/l10n/fluent_migrations.html#install-fluent-migration

ran migration recipe as described, worked as expected with only one transformation
ex.
running the command:

migrate-l10n
--lang test
--reference-dir PATH/TO/en-US
--localization-dir PATH/TO/test
--dry-run
name_of_the_recipe

with the migration script:
def migrate(ctx):
"""Bug 1518234 - Migrate the browser migration wizard, part {index}."""

ctx.add_transforms(
    "browser/browser/migration.ftl",
    "browser/browser/migration.ftl",
    transforms_from(

"""
migration-wizard =
.title = { COPY(from_path, "migrationWizard.title") }
""", from_path="browser/chrome/browser/migration/migration.dtd"
)
)

works as expected

running the same command with the migration script:
def migrate(ctx):
"""Bug 1518234 - Migrate the browser migration wizard, part {index}."""

ctx.add_transforms(
    "browser/browser/migration.ftl",
    "browser/browser/migration.ftl",
    transforms_from(

"""
migration-wizard =
.title = { COPY(from_path, "migrationWizard.title") }
import-from =
.label = { COPY(from_path, "importFrom.label") }
""", from_path="browser/chrome/browser/migration/migration.dtd"
)
)

fails with the output:
Running migration bug_1518234_migration for test
WARNING:migrate:Plural rule for "'test'" is not defined in compare-locales
Traceback (most recent call last):
File "/Users/iankirkpatrick/anaconda3/bin/migrate-l10n", line 11, in <module>
sys.exit(cli())
File "/Users/iankirkpatrick/anaconda3/lib/python3.6/site-packages/fluent/migrate/tool.py", line 143, in cli
dry_run=args.dry_run
File "/Users/iankirkpatrick/anaconda3/lib/python3.6/site-packages/fluent/migrate/tool.py", line 74, in main
for path, content in snapshot.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'

Actual results:

the migration tool fails with the output

Running migration bug_1518234_migration for test
WARNING:migrate:Plural rule for "'test'" is not defined in compare-locales
Traceback (most recent call last):
File "/Users/iankirkpatrick/anaconda3/bin/migrate-l10n", line 11, in <module>
sys.exit(cli())
File "/Users/iankirkpatrick/anaconda3/lib/python3.6/site-packages/fluent/migrate/tool.py", line 143, in cli
dry_run=args.dry_run
File "/Users/iankirkpatrick/anaconda3/lib/python3.6/site-packages/fluent/migrate/tool.py", line 74, in main
for path, content in snapshot.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'

Expected results:

the migration tool should generate an .ftl file from the migration script and compare to a handwritten one to verify the correctness of the migratino script

Comment 1

[Axel Hecht [:Pike]]

There's more problems than just the iterator, in particular, there's byte vs string and encoding foo.

tool.py needs some tests, aka, a refactor to have testable segments. :-(

Comment 2

[Axel Hecht [:Pike]]

Attachment: bug 1521402, refactor tool.py to testable class, part 1, r=stas

bug 1521402, refactor tool.py to testable class, part 2, r=stas

Comment 3

[Axel Hecht [:Pike]]

Attachment: Bug 1521402, make migrations run in python3, r=stas

The b() thing is a lie, let's just encode our world in
utf-8.
And a dash of iterators.

Depends on D17499

Comment 4

[Axel Hecht [:Pike]]

This landed on the upstream repository now, thanks.

It'll land in m-c on the next uplift.