1861698 - Add Telemetry for Language Identification in Translations

Status: RESOLVED FIXED

(Firefox :: Translations, enhancement)

Description

[Erik Nordin [:nordzilla]]

In Bug 1859081 we are going to change the way we do language detection in an effort to reduce false positives.

Following this change, we should instrument the Full-Page Translations functionality with telemetry to measure a few aspects about our translations offering process.

We want to measure the following qualities:

1. Whether visited pages have specified language tag vs. no language tag.
2. Among pages that have a specified language tag which is a supported language, how often did our language detection process agree with the language tag vs disagree?
3. How often is our language detection confident about is predictions vs. not confident?

We should ensure that we have this data instrumented for our current CLD2 implementation before we switch to fastText, CLD3, or potentially a different language identification mechanism, so that we can make informed inferences about any improvements or regressions.

Comment 1

[Marco Castelluccio [:marco]]

We might not even need Telemetry, we could use CommonCrawl. There could already some analysis of this done by somebody else.

Comment 2

[Erik Nordin [:nordzilla]]

Attachment: Bug 1861698 - Add translations langId telemetry r=#translations-reviewers!

This patch adds a new Translations telemetry event
for the language identification, such that we can
get a sense of the average performance of our language
detection code in aggregate.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/483b2f0f0788
https://hg.mozilla.org/integration/autoland/rev/82374a8c604b
Add translations langId telemetry r=translations-reviewers,gregtatum

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/36b0ce8f9d58
https://hg.mozilla.org/integration/autoland/rev/6993600fb880
Revert "Bug 1861698, Bug 1995634 - Add translations langId telemetry r=translations-reviewers,gregtatum" for causing tr*ns failures on browser_translations_full_page_telemetry_auto_translate.js

Comment 5

[Cristina Horotan [:chorotan]]

Backed out for causing tr8ns failures on browser_translations_full_page_telemetry_auto_translate.js

Backout link

Push with failures

Failure log

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/8cd5d52a46f8
https://hg.mozilla.org/integration/autoland/rev/c93ccc131fe1
Add translations langId telemetry r=translations-reviewers,gregtatum

Comment 7

[amarc]

https://hg.mozilla.org/mozilla-central/rev/c93ccc131fe1