1596703 - Can't write UTF-8 character in Variant

Status: RESOLVED DUPLICATE of bug 1596700

(Localization Infrastructure and Tools :: Fluent Migration, defect)

Description

[Francesco Lodolo [:flod]]

Consider the following transform

            FTL.Message(
                id=FTL.Identifier("screenshots-meta-key"),
                value=FTL.Pattern(
                    elements=[
                        FTL.Placeable(
                            expression=FTL.SelectExpression(
                                selector=FTL.FunctionReference(
                                    id=FTL.Identifier(name="PLATFORM"),
                                    arguments=FTL.CallArguments(positional=[], named=[])
                                ),
                                variants=[
                                    FTL.Variant(
                                        key=FTL.Identifier("macos"),
                                        default=False,
                                        value=FTL.Pattern(
                                            elements=[FTL.TextElement("⌘")])
                                    ),
                                    FTL.Variant(
                                        key=FTL.Identifier("other"),
                                        default=True,
                                        value=FTL.Pattern(
                                            elements=[FTL.TextElement("Ctrl")])
                                    ),
                                ]
                            )
                        )
                    ]
                )
            ),

This fails with

Traceback (most recent call last):
  File "/usr/local/bin/migrate-l10n", line 9, in <module>
    load_entry_point('fluent.migrate==0.6.4', 'console_scripts', 'migrate-l10n')()
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 179, in cli
    dry_run=args.dry_run
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 133, in main
    migrator.run(migration)
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 76, in run
    ctx, changeset['changes'], known_legacy_translations
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/tool.py", line 93, in snapshot
    known_legacy_translations
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/_context.py", line 314, in serialize_changeset
    changeset, known_translations
  File "/Users/flodolo/mozilla/mercurial/fluent-migration/fluent/migrate/_context.py", line 313, in <dictcomp>
    for path, snapshot in self.merge_changeset(
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 36, in serialize
    parts.append(self.serialize_entry(entry, state))
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 44, in serialize_entry
    return serialize_message(entry)
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 86, in serialize_message
    parts.append(serialize_pattern(message.value))
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 123, in serialize_pattern
    for elem in pattern.elements])
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 137, in serialize_element
    return serialize_placeable(element)
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 148, in serialize_placeable
    return "{{ {}}}".format(serialize_expression(expr))
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 179, in serialize_expression
    out += serialize_variant(variant)
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 190, in serialize_variant
    indent(serialize_pattern(variant.value))
  File "/Users/flodolo/mozilla/mercurial/python-fluent/fluent.syntax/fluent/syntax/serializer.py", line 123, in serialize_pattern
    for elem in pattern.elements])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Comment 1

[Francesco Lodolo [:flod]]

Duping, since the cause is the same.