1905599 - Add a new remote type `inference`

Status: RESOLVED FIXED

(Core :: Machine Learning, enhancement)

Description

[Tarek Ziadé (:tarek)]

This type will be used for running the machine learning inference runtimes.

Comment 1

[Tarek Ziadé (:tarek)]

Attachment: Bug 1905599 - Add a new remote type `inference` r?smaug,gregtatum

Comment 2

[Tarek Ziadé (:tarek)]

We currently use the "web" remote type to create a browser environment and have a dedicated content process for both the Translation feature and the image-to-text PDF.js feature, that are running an inference runtime in WASM. Currently we need a full browser because we use the JS engine.

The browser is created here : https://searchfox.org/mozilla-central/source/toolkit/components/ml/content/EngineProcess.sys.mjs#282-288
and then we load a web poage containing the WASM runtime. example:

• https://searchfox.org/mozilla-central/source/toolkit/components/ml/content/EngineProcess.sys.mjs#231
• https://searchfox.org/mozilla-central/source/toolkit/components/translations/content/translations-engine.html#14

This isolation allows us to control the resources used by the inference, that content process gets killed after a timeout, and we can also avoid crashes if the process uses too much memory.

We've talked with :nika about creating a dedicated remote type to differentiate this process from others. This will come handy in about:processes, about:memory and telemetry, and also if we allow using inference in web extensions at some point - e.g. we will probably have to prioritize calls and have a separate process for web extensions so it does not interfere with what Firefox is doing.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d49b569b612b
Add a new remote type `inference` r=smaug,fluent-reviewers,flod

Comment 4

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/d49b569b612b