1995634 - Utilize nsIDocumentEncoder for Translations Language Identification

Status: RESOLVED FIXED

(Firefox :: Translations, task)

Description

[Erik Nordin [:nordzilla]]

Description

In Bug 1967758 I transitioned translations language identification to use the DOMExtractor code.

Initial profiling is showing that the nsIDocumentEncoder is still faster at extracting text than the DOMExtractor code.

DOMExtractor (2048 code-unit limit)

• https://share.firefox.dev/4nhfAoC

nsIDocumentEncoder (2048 code-unit limit)

• https://share.firefox.dev/4njpMwV

In the short term, I would like to move Translations back to using nsIDocumentEncoder until such a time that we improve the performance and capabilities of DOMExtractor.

Comment 1

[Erik Nordin [:nordzilla]]

Attachment: Bug 1995634 - Use nsIDocumentEncoder for Translations langId r=#translations-reviewers!

This patch updates the Translations language-detection algorithm
to utilize nsIDocumentEncoder for the page-text extraction.

Comment 2

[Erik Nordin [:nordzilla]]

Attachment: Bug 1995634 - Add translations langId telemetry r=#translations-reviewers!

This patch adds a new Translations telemetry event
for the language identification, such that we can
get a sense of the average performance of our language
detection code in aggregate.

Comment 3

[Phabricator Automation]

Comment on attachment 9521484 [details]
Bug 1995634 - Add translations langId telemetry r=#translations-reviewers!

Revision D269461 was moved to bug 1861698. Setting attachment 9521484 [details] to obsolete.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a07fd9772bbb
https://hg.mozilla.org/integration/autoland/rev/b39c0de239fb
Use nsIDocumentEncoder for Translations langId r=translations-reviewers,gregtatum

Comment 5

[Cristina Horotan [:chorotan]]

Backed out for causing tr8ns failures on browser_translations_full_page_telemetry_auto_translate.js

Backout link

Push with failures

Failure log

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/5ef169b4891f
https://hg.mozilla.org/integration/autoland/rev/591439041acd
Use nsIDocumentEncoder for Translations langId r=translations-reviewers,gregtatum

Comment 7

[amarc]

https://hg.mozilla.org/mozilla-central/rev/591439041acd