2005369 - Add inference performance metrics collection to StaticEmbeddingsPipeline

Status: RESOLVED FIXED

(Core :: Machine Learning: On Device, task, P3)

Description

[Taimur]

Summary Currently, the StaticEmbeddingsPipeline.mjs backend does not collect detailed performance metrics during inference. This task is to implement telemetry/tracking to measure embedding performance, bringing it to parity with other ML components.

Implementation Details We need to introduce a metrics object (or similar structure) to capture inference data within the StaticEmbeddingsPipeline class.

Since embeddings differ from generation tasks, the metrics should focus on:

Input Size: Number of characters and tokens (if tokenizer is accessible) processed.

Latency: Total wall-clock time for the run() execution.

Throughput: Calculated as Tokens/Second or Characters/Second.

Initialization: (Optional) Time taken for the pipeline to load and initialize.

Reference: toolkit/components/ml/content/backends/StaticEmbeddingsPipeline.mjs

Comment 1

[Joe Bowser]

Attachment: Bug 2005369 - Collect inference metrics in the static embeddings pipeline. r?thasan

StaticEmbeddingsPipeline now emits a metrics object on its run
result with a runTimestamps timeline (initializationStart/End,
runStart/End) plus scalar fields tailored to embeddings:
inputChars, inputTokens, inferenceTime, tokensPerSecond,
charsPerSecond.

The shape mirrors the LlamaCppPipeline change from bug 2005365,
adapted for embeddings: there is no decode stage or
time-to-first-token, but character throughput is added since
embeddings are commonly fed raw user text where character counts
matter alongside token counts.

The metrics field on EmbeddingResponse was previously typed as
an array and emitted as []; no downstream consumer read it. That
contract is replaced by the new EmbeddingMetrics interface in
StaticEmbeddingsPipeline.d.ts.

Adds a regression-guarding assertion in browser_ml_static_embeddings.js
covering the metrics shape, the four runTimestamps names, and
that inputChars/inputTokens/inferenceTime have sensible values.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/ebfa0da43e2f
https://hg.mozilla.org/integration/autoland/rev/c8cc5b716736
Collect inference metrics in the static embeddings pipeline. r=thasan,ai-platform-reviewers

Comment 3

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/c8cc5b716736