1830381 - Implement a Language-Pair Dependency Solver

Status: NEW

(Firefox :: Translations, enhancement)

Description

[Erik Nordin [:nordzilla]]

Overview

In the future, it would beneficial to create an abstraction around having a language-pair solver, which would be a graph-like structure that is able to solve dependencies around which languages have the capability to translate to other languages based on available models, along with tracking the versions of the models between each pairing (particularly for labeling such pairings as being in beta or not in the UI).

The current implementation of Firefox Translations is simplified, but is forward-compatible with having a full-fledged solver in the future.

The two driving factors are to achieve more granularity for both the language-pair dependencies, and the version labeling.

Language-Pair Dependency Solving

An important invariant to maintain in Firefox Translations, perhaps obviously, is that we only list languages that have models available to perform the translation.

The current Firefox Translations code takes a simplified view of language-pair evaluation that ensures that each language pair has valid models available.

One key aspect of this is the idea of using a "pivot language" for translations:

https://firefox-source-docs.mozilla.org/toolkit/components/translations/resources/01_overview.html#pivot-translations

Currently, we only use English as a pivot language, and we ensure compatibility and interoperability with our initial set of languages by ensuring that each model has access to the en (English) pivot language:

https://searchfox.org/mozilla-central/rev/f32d5f3949a3f4f185122142b29f2e3ab776836e/toolkit/components/translations/actors/TranslationsParent.sys.mjs#662-733

Allowing for more granularity in solving these dependencies would provide the following improvements:

1. The ability to provide direct translations between languages that don't have access to the en (English) pivot language.

As stated above, we currently ensure that each language has access to the pivot, which ensures validity for our current language set. But it would be beneficial to relax this constraint, which would allow any direct pairings to be valid as long as the models are available.

For example, imagine a situation in which we add support to translate between ja (Japanese) and zh (Chinese), but neither of those languages has a translation to en (English), the pivot language.

Under our current implementation, these languages would be removed from display and we have tests to enforce such a constraint:

https://searchfox.org/mozilla-central/rev/566f69826b829767d0373169eef6c50cc7d2241b/toolkit/components/translations/tests/browser/browser_translations_actor.js#12-32

Having a full dependency solver would allow us to release direct translation pairings without requiring access to the pivot language.

2. The ability to solve transitive dependencies using languages other than en (English) as a pivot language.

As stated above, our simplified approach currently uses only en (English) as a pivot language, but it would be beneficial in the future to be able to use any language as a pivot as long as a dependency is satisfied.

For example, imagine a scenario in which models exist for the following language pairs:

zh => ja (Chinese to Japanese)
ja => zh (Japanese to Chinese)
ja => ko (Japanese to Korean)

Given that these models exist, it should be possible to solve a transitive dependency to translate zh => ko (Chinese to Korean) by using ja (Japanese) as a pivot language even though no direct translation models exist for zh => ko (Chinese to Korean).

zh => ja => ko

This is not possible in our current implementation because the only language we use as a pivot is en (English). Relaxing this constraint would allow for more robust ability to make pairings as we add more language models.

Language-Pair Versioning Granularity

Our current implementation takes a simplified approach that considers an entire language to either be in beta or not, based on the version of the language models, rather than individual language pairings.

This means, for example, that if fr (French) were a beta language, then each pairing from fr (French) to every other language would be labeled as being in beta.

This code is enforced here, where we store an isBeta value with the entire language itself, rather than to a specific pairing:

https://searchfox.org/mozilla-central/rev/566f69826b829767d0373169eef6c50cc7d2241b/toolkit/components/translations/actors/TranslationsParent.sys.mjs#489-509

It would be beneficial to have a full language-pair dependency solver that also keeps track of model versions at the granularity of the language pairing itself.

This will make it possible in the future, for example, if fr <=> en (French/English) is considered stable and nl <=> en (Dutch/English) is considered stable, but we add a new, direct language pairing for fr <=> dl (French/Dutch) that is considered to be in beta, then we would be able to display fr (French) or nl (Dutch) as being in beta respectively when translating to/from the other, but as being stable when translating to/from en (English).

Conclusion

Having such a solution to this dependency solving will likely be necessary in the future, but is not a blocker for an MVP release of Firefox Translations, because the simplified implementation that we are using is both sufficient for the initial set of language models that we intend to ship, and is also forward-compatible with the functionality that a more granular solver would provide as described above.