1914159 - Translations to some languages destroy emoji (unicode glyphs)

Status: VERIFIED FIXED

(Firefox :: Translations, defect)

Description

[Jan Brasna [:janbrasna]]

Attachment: Screen Recording FXDE130 Unicode

Using on-device translations, mainly German and French (any direction) break many Unicode glyphs like Emoji and symbols as checkmarks etc. — see the attached video.

Originally reported: https://github.com/mozilla/bedrock/issues/13357

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:nordzilla, could you have a look please?

For more information, please visit BugBot documentation.

Comment 2

[Erik Nordin [:nordzilla]]

Hey janbrasna, thanks for filing this bug!

I think that this issue has actually been fixed in terms of our model training infrastructure.

However, the version-1.0 models for French and German that we currently have shipped do not include these robustness changes yet.

---

I chatted with Greg, and he's going to help compile a list of all the models that have been trained with these robustness fixes vs. models that still need them.

Comment 3

[Negritas Sergiu, Desktop QA]

Hello! I have managed to reproduce the issue with firefox 133.0a1(2024-10-08) and 132.0b4 on Ubuntu 22.04 and MacOS 15.

I will mark this issue as NEW and update the flags in order to get our developers involved.

Have a nice day!

Comment 4

[Greg Tatum [:gregtatum]]

This issue is tracked here as it involves re-training models: https://github.com/mozilla/firefox-translations-training/issues/757