2012177 - Introduce & use a best-onnx backend

Status: RESOLVED FIXED

(Core :: Machine Learning: On Device, enhancement, P2)

Description

[Aristide Tossou]

best-onnx should mirror best-llama. Basically, if onnx-native cannot work on some user machine, it fallbacks to onnx wasm

Comment 1

[Greg Tatum [:gregtatum]]

Here's where the best-llama logic is being determined.

https://searchfox.org/firefox-main/rev/c90c4a59e34b5c6142691052bbb15b295ea0c37c/toolkit/components/ml/actors/MLEngineParent.sys.mjs#657-668

And a general search for best llama: https://searchfox.org/firefox-main/rev/c90c4a59e34b5c6142691052bbb15b295ea0c37c/toolkit/components/ml/actors/MLEngineParent.sys.mjs#657-668

Comment 2

[Greg Tatum [:gregtatum]]

There's also this fallbackBackend: "onnx", logic which should be migrated to this new best-onnx. I'm not sure if the fallbackBackend thing is actually wired up anywhere.

Comment 4

[Joe Bowser]

Attachment: Bug 2012177 - Add a "best-onnx" backend that chooses between onnx-native and wasm onnx r=gregtatum

This commit does the following:

• Creates the "best-onnx" backend, which works similarly to the "best-llama" backend
• Does a check to make sure that ONNX runtime can actually be loaded to detect issues in 1983444
• Provides a test to test those changes

Manual Tests that were done:

• Testing Smart Tab Grouping
• Console tests to make sure that the C++ method canUseOnnxNative() is exposed to JS

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/ff2569a4a447
https://hg.mozilla.org/integration/autoland/rev/ffc0c7fba82d
Add a "best-onnx" backend that chooses between onnx-native and wasm onnx r=tabbrowser-reviewers,ai-platform-reviewers,thasan,sthompson

Comment 6

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/ffc0c7fba82d