1842763 - Improve translation of sentences containing numbers

Status: RESOLVED MOVED

(Firefox :: Translations, enhancement)

Description

[Marco Castelluccio [:marco]]

This might be fixed by data cleaning and data augmentation.

E.g. in our datasets, there are "bad" sentence pairs where different units of measurement are used (e.g. pound in English, kg in Italian). It's even worse when there is no formula to translate between units (e.g. dollars in en-US, euros in Italian).

Comment 2

[MarjaE]

Aside from unit conversions, there are cases where it drops digits from numbers. For example, turning 244890 into 24,890. Since this is a silent error, and an order-of-magnitude error, it could be a problem...

See bug 1797667.

Comment 3

[Greg Tatum [:gregtatum]]

I wonder if we should also write some deterministic code to verify and audit numbers in translations, since the translation models are just statistical. This may be something to be done on the bergamot engine side of things.

An order-of-magnitude error is pretty concerning.

Comment 4

[Marco Castelluccio [:marco]]

Moved to https://github.com/mozilla/firefox-translations-training/issues/173.