1399547 - Parser: Allow Unicode characters variant keys

Status: NEW

(L20n :: Python Library, enhancement)

Description

[Staś Małolepszy :stas]

The following example doesn't currently work because of non-ASCII characters in the #zeński tag and in has-updated's variant keys.

brand-name = Aurora
    #żeński

has-updated = { brand-name ->
        [męski] { brand-name} został zaktualizowany.
        [żeński] { brand-name } została zaktualizowana.
       *[inny] Program { brand-name } został zaktualizowany.
    }

Comment 1

[Axel Hecht [:Pike]]

Tags and sections are gone, but variants are around still.

Variant key is a wordlist, (((char - line-break) - inline-space) - [#x5b#x5c#x5d#x7b#x7d])+ being a word.

There's a spec related question on whether term attributes should allow a word or an identifier. Stas, did we discuss that when we did Terms?

Also, flod, any indication of how we should prioritize this?

Comment 2

[Francesco Lodolo [:flod]]

(In reply to Axel Hecht [:Pike] from comment #1)
> Also, flod, any indication of how we should prioritize this?

Looks like a variant name with Unicode characters is currently raising an error (unparsed content) in compare-locales. 

As long as we don't fail silently, I think it's not high priority. Right now, we only have one locale (Italian) using custom variants, and one that might be planning to use them (cs). It's also not super-intuitive how to do that in Pontoon.

If we start expanding terms support, e.g. private terms files, that might change the scenario.

Comment 3

[Staś Małolepszy :stas]

There's a related spec issue about relaxing the grammar of VariantNames even further: https://github.com/projectfluent/fluent/issues/90

The question about what should be possible as term attributes is https://github.com/projectfluent/fluent/issues/82#issuecomment-359737284. I think it's a larger discussion related to https://github.com/projectfluent/fluent/issues/94 (Clarify Terms, Messages, and Variants). I'll comment there.