1982594 - Update Firefox to utilize zstd for Translations models and WASM

Status: RESOLVED FIXED

(Firefox :: Translations, task)

Description

[Erik Nordin [:nordzilla]]

Description

In Bug 1947431 we implemented zstd for DecompressionStream and allowed it for internal use within privileged Firefox contexts.

We now have v2 Remote Settings collections that are designed to host and serve zstd compressed models and WASM.

We need to hook everything together to allow Firefox to pull from the compressed collections, and decompress the models as-needed.

Comment 1

[Isaac Briandt]

Attachment: Bug 1982594 - Update Translations RemoteSettings schemas r=#translations-reviewers!

Comment 2

[Isaac Briandt]

Attachment: Bug 1982594 - Rework TranslationsEnginePaylod IPC Transfer r=#translations-reviewers!

Comment 3

[Erik Nordin [:nordzilla]]

Attachment: Bug 1982594 - Fix Translations Tests r=#translations-reviewers!

Comment 4

[Erik Nordin [:nordzilla]]

Attachment: WIP: Bug 1982594 - Return blob directly form Remote Settings

This patch modifies the RemoteSettings client's download()
method to be able to return a Blob directly, in addition to
the ArrayBuffer that it already returns.

This prevents the Translations code from having to manually
construct a Blob from the buffer, prevening a jank in the parent
process.

Comment 5

[Erik Nordin [:nordzilla]]

Attachment: Bug 1982594 - Update Bergamot build hash r=#translations-reviewers!

The build should be exactly the same, though I technically
built it from a different hash on GitHub, so we should
update that.

Comment 6

[Erik Nordin [:nordzilla]]

Attachment: WIP: Bug 1982594 - (Temporary) Attempt to reproduce behavior

This patch is a condensed version of my initial WIP patch stack
for Bug 1982594 where I initially encountered this behavior.

The behavior reproduces by running the following command:

./mach test browser_translations_e2e

Rather than executing successfully, the WASM instantiation
will hang indefinitely until the test case times out.

This code is very likely to bit rot, though I hope it proves
to be useful in exploring this behavior. I haven't been able
to reproduce in a different way at the time of writing.

Comment 7

[Phabricator Automation]

Comment on attachment 9512975 [details]
WIP: Bug 1982594 - (Temporary) Attempt to reproduce behavior

Revision D264662 was moved to bug 1988289. Setting attachment 9512975 [details] to obsolete.

Comment 8

[Erik Nordin [:nordzilla]]

Attachment: Bug 1982594 - Patch Translations WASM Instantiation r=#translations-reviewers!

This patch modifies the WASM function for instantiating
to use synchronous functions instead of WebAssembly.instantiate().

See Bug 1988289 for more information.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a30981d711c7
https://hg.mozilla.org/integration/autoland/rev/eb6fd21e3556
Update Translations RemoteSettings schemas r=translations-reviewers,gregtatum
https://github.com/mozilla-firefox/firefox/commit/03f01898fa06
https://hg.mozilla.org/integration/autoland/rev/c7cbbf7cf116
Rework TranslationsEnginePaylod IPC Transfer r=translations-reviewers,gregtatum
https://github.com/mozilla-firefox/firefox/commit/c6b8c023e69c
https://hg.mozilla.org/integration/autoland/rev/915cfd067f0e
Fix Translations Tests r=translations-reviewers,gregtatum
https://github.com/mozilla-firefox/firefox/commit/02fefb1e1bae
https://hg.mozilla.org/integration/autoland/rev/8a41bb5bb9fc
Update Bergamot build hash r=translations-reviewers,gregtatum
https://github.com/mozilla-firefox/firefox/commit/9140dac1eab8
https://hg.mozilla.org/integration/autoland/rev/e0d24a0fe50f
Patch Translations WASM Instantiation r=translations-reviewers,gregtatum

Comment 10

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/eb6fd21e3556
https://hg.mozilla.org/mozilla-central/rev/c7cbbf7cf116
https://hg.mozilla.org/mozilla-central/rev/915cfd067f0e
https://hg.mozilla.org/mozilla-central/rev/8a41bb5bb9fc
https://hg.mozilla.org/mozilla-central/rev/e0d24a0fe50f