2074072 - MLEngine onnx backends leak memory

Status: RESOLVED FIXED

(Core :: Machine Learning: On Device, defect, P2)

Description

[Valentin Pollet]

Steps to reproduce

Just use the browser. My go-to way to trigger the leak is to alternate between typing in the url bar and sending chat messages. This usage pattern will cycle static embeddings and contextual embeddings on the same simple-text-embedder engine, effectively tearing down and respawning engines on the same process.

Expected behavior

A "floor" exists for memory usage.

Actual behavior

Memory grows seemingly without bounds when engines are cycled.

Potential cause

We seem not to release OrtValues in InferenceSession

Comment 1

[Valentin Pollet]

Attachment: Bug 2074072 - Test native ONNX session memory growth and engine cleanup after initialization failure. r?#ai-platform-reviewers!

Comment 2

[Valentin Pollet]

Attachment: Bug 2074072 - Release native ONNX inference outputs and metadata. Release engines and workers after initialization failure. r?#ai-platform-reviewers!

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/1d9bc9ed888d
https://hg.mozilla.org/integration/autoland/rev/385dfa9e0682
Test native ONNX session memory growth and engine cleanup after initialization failure. r=ai-platform-reviewers,nordzilla
https://github.com/mozilla-firefox/firefox/commit/ca6b1d03bd46
https://hg.mozilla.org/integration/autoland/rev/81ab3f523746
Release native ONNX inference outputs and metadata. Release engines and workers after initialization failure. r=ai-platform-reviewers,nordzilla

Comment 4

[amarc]

https://hg.mozilla.org/mozilla-central/rev/385dfa9e0682
https://hg.mozilla.org/mozilla-central/rev/81ab3f523746

Comment 5

[Performance Sheriff Bot]

Perfherder has detected a mozperftest performance change from push 81ab3f52374664c20560b90c018569c927ad87e2.

No action is required from the author; this comment is provided for informational purposes only.

Improvements	Test	Platform	Options	Absolute values [old vs new]
36%	ML Semantic History Search SHORT-SEMANTIC-total-memory-usage-NATIVE	windows11-64-24h2-hw-ref-shippable		367.71 MB -> 234.42 MB
36%	ML Semantic History Search SHORT-SEMANTIC-total-memory-usage-NATIVE	windows11-64-24h2-shippable		367.42 MB -> 234.71 MB
36%	ML Semantic History Search LONG-MULTIPLE-SEMANTIC-total-memory-usage-NATIVE	windows11-64-24h2-shippable		368.75 MB -> 235.75 MB
36%	ML Semantic History Search LONG-MULTIPLE-SEMANTIC-total-memory-usage-NATIVE	windows11-64-24h2-hw-ref-shippable		369.08 MB -> 236.00 MB
36%	ML Semantic History Search LONG-SEMANTIC-total-memory-usage-NATIVE	windows11-64-24h2-hw-ref-shippable		369.42 MB -> 236.33 MB
...	...	...	...	...
21%	ML Semantic History Search LONG-SEMANTIC-total-memory-usage-NATIVE	macosx1500-aarch64-shippable		297.08 MB -> 234.75 MB

Need Help or Information?

If you have any questions, please reach out to [e-mail]. Alternatively, you can find help on Slack by joining #perf-help, and on Matrix you can find help by joining #perftest.

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.