1778184 - Internationalise QuickActions

Status: RESOLVED FIXED

(Firefox :: Address Bar, enhancement, P3)

Description

[Dale Harvey (:daleharvey)]

Currently we hardcode one English command for each action.

We have had previously discussions about this with UrlbarInterventions and QuickSuggest, in those cases it looks agreed that we would not use .ftl files to translate the incoming "commands" due to the complexity of being able to handle spelling mistakes and such across different language. As a result these features are currently en-US only.

I think QuickActions are different from those example as the main thing they are doing is trying to capture users intentions and they need to be able to accept a broad range of input in order to be useful.

QuickActions are useful regardless of whether we can capture the users intentions, they are discoverable and learnable and a user in any language should be able to gain the benefits of the feature without a broad range of input being captured. We may and likely will want to be able to capture the users intentions in order to make the feature easier for users to discover, so I propose:

We set an initial limit on the number of keywords (phrases) than can trigger an action, we could say 3 initially, or even 1. Those keywords are maintained in .ftl file and translated for users across all languages. We do not handle spelling mistakes or large input lists. In this case entering "screenshot" (or any prefix thereof) in any language will present the screenshot option (but not "take a screenshot" or "svreenshot")

Seperately we look into having trained algorithms be able to match users input to actions, possibly via merino and combining with the work being done on QuickSuggest. We could have an algorithm recognise "how to photograph this website" as something that should present the screenshot action and the training done can span multiple locales.

Only presenting this in en-US is an option, but I think the above is a good chance to capture value for as many users as possible without incuring too much complexity. The situation I would like to avoid is trying to build out a half hearted algorithm on the client that can capture "some" of the users extra intentions but is technically complex and only really works in en-US.

Comment 1

[Dale Harvey (:daleharvey)]

I was waiting to ping about this until it was testable in nightly, but if you enable browser.urlbar.quickactions.enabled = true in the urlbar then you should be able to test them out now, I think that will help a bit understand the above. I believe Marco has been involved in a few of these discussions previously as well as Flod so if either of you have input (or a better idea about who would) it would be appreciated.

Comment 2

[Marco Bonardo [:mak]]

I don't have solutions, just a few thoughts.

We were in the past evaluating using a lib like CLD3 or whatlang-rs and compare the result to a list of locales including: Web Content locales list in about:preferences, build locale, english (always keep it as a fallback?). For just making straight string comparison, such complexity may not be necessary though.
You could just maintain english -> actions associations in Remote Settings or just some json. Those would always work.
You could then download a few localized files from the above list of locales (these could just be localized_word -> english_word). These lists would be Mozilla managed though, or at least verified before shipping. I don't think we want to fully push them into open localization.

Another alternative may be to cooperate with the Bergamot Project, have it translate words into English, and hopefully the result matches our command. Though that'd require Bergamot to be included in Firefox, rather than being an add-on. Not sure what's the status there.

For matching similarity there's a lot of algo we could use, we have an implementation of levenshtein in https://searchfox.org/mozilla-central/source/toolkit/modules/NLP.jsm or also see https://docs.rs/trigram/latest/trigram/ (could be reimplemented into nlp.jsm)... There's plenty of literature about these algos.

The server side of course could be a lot more powerful.

Comment 3

[Francesco Lodolo [:flod]]

To catch minor spelling errors, Levenshtein distance sounds like a reasonable approach (although with very short words, like update, you need a low threshold).

As for localizing the keywords, I agree that the best approach would be to use something stored outside of localizable files (remote-settings seems the natural choice). I wonder if we should maybe experiment with one or two locales at first, and see how the data compares to English? Or even how frequently users match these actions in English.

The final result might even be that we focus only on the largest locales, and accept that this is too complex to localize for the smaller ones.

P.S. Bergamot is unlikely to be a solution. It supports a very limited number of language pairs, and it's likely to fail on single words (at least in my experience).

Comment 4

[Dale Harvey (:daleharvey)]

Attachment: Bug 1778184 - Allow QuickAction commands to be translated. r=flod

Comment 5

[Dale Harvey (:daleharvey)]

Uploaded a WIP patch that uses l10n to define the commands.

Is there a downside to doing things this way? If we punt the localisation into some remoteSettings JSON file the reality is that this feature will be presented in English only for the foreseeable future. If that was an explicit choice, command lines are often english only, then thats fine. However if we wanted to present this feature in other locales then this way seems preferable than depending on inventing a new remote settings based internationalisation mechansm?

Comment 6

[Francesco Lodolo [:flod]]

I don't think it's very flexible, because:

• It's bound to trains, so updates would be slow and couldn't be uplifted.
• It forces locales to match what works for English (1:1 match).
• It could potentially "poison" translation memory, because the translation stored might not be an actual translation of the English text.

For example: you added two strings for "logins" and "password", what if I want to use three or four words in Italian instead? I guess technically one more flexible solution could be to have one string, maybe comma separated, so that each locale could have how many alternatives it needs.

quickactions-cmd-addons = addons

Did you spell it without hyphen on purpose ?

quickactions-cmd-viewsource = viewsource

Why the lack of space? We definitely need to support multiple words here. For example, add-on is 2 words in most languages, and you probably want to trigger this also for "extensions" and "themes" (BTW, I thought we were trying to avoid "add-ons" as a term, because users don't understand it?)
https://transvision.flod.org/string/?entity=browser/browser/preferences/preferences.ftl:sync-currently-syncing-addons&repo=gecko_strings

Comment 7

[Dale Harvey (:daleharvey)]

It's bound to trains, so updates would be slow and couldn't be uplifted.

I could see this being a problem for when we are looking to try to capture users intents and are supporting a broad range of inputs with a need to experiment, but for the use case where we have a ~1:1 mapping of translations I dont see a problem with it being bound to trains, as the feature itself is.

It forces locales to match what works for English (1:1 match).

To be clear this is only for a base set of translations, I think we would be able to layer a set of AI / Search related matches on top of this that would handle translation completely differently (I think this could potentially be wrapped up / served by merino)

more flexible solution could be to have one string, maybe comma separated, so that each locale could have how many alternatives it needs.

That sounds like a good idea.

Did you spell it without hyphen on purpose ?
Why the lack of space?

For both of these I was just considering them as "commands" which generally do not have spaces and often use hyphens instead of spaces, We dont actually need to do that at all though and I think itd be best to take that out.

"extensions" and "themes" (BTW, I thought we were trying to avoid "add-ons" as a term, because users don't understand it?)

Good to know, I will update the patch to include those and fix the spaces / hyphers and we can evaluate it

Comment 8

[Dale Harvey (:daleharvey)]

Francesco cheers for the feedback, I have updated the patch @ https://phabricator.services.mozilla.com/D152194 and would like your feedback.

To be clear, I think this method of translation would not go any further than it currently does, I would not be looking to turn those strings into long lists of everything that can possibly be captured (and would be fine with a hard limit, I think even 1 is reasonable although 3 looks like a nice number). This is solely to provide a base level of translation so the feature can be exposed to used in multiple locales.

I think the more complicated levels of capture and translation, AI trained matches and/or search history based stuff can be served by remote settings or potentially be done by merino, but either way I would expect them to be a seperate layer on top of these.

If we do not translate these strings, I think we have a potentially very confusing UI where we either have not translate any of the strings, even the UI, or we end up showing a button called "impresión" that the user cannot access by typing impresión but instead they have to type "print"

If we don't end up with something like this I expect much like UrlbarInterventions and Firefox Suggest that the feature will be exposed in en-US only for the foreseeable future.

Comment 9

[Francesco Lodolo [:flod]]

Left some comments, but I'm OK with the current approach.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d630d5cd4529
Allow QuickAction commands to be translated. r=flod,adw,fluent-reviewers

Comment 11

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/d630d5cd4529