2005778 - [KR5] Phase 3 - MVP: Add Glean telemetry metrics for AI Window Security Layer

Status: RESOLVED WONTFIX

(Core :: Machine Learning: General, task)

Description

[Randy Concepcion]

Add Glean telemetry to track security layer performance in production. This data will help monitor performance impact and identify regressions as the security layer evolves.

Coordinate with Taimur to align with existing inference runtime telemetry patterns and naming conventions.

Metrics to Track

• Evaluation duration (time spent in SecurityOrchestrator.evaluate())
• Evaluation count (number of evaluations per session)

Notes

• Security layer is a bottleneck for LLM calls; production telemetry will help monitor real-world impact
• Start with basic metrics; expand as needed based on findings

Acceptance Criteria:

• Glean metrics implemented for security layer evaluation
• Metrics aligned with existing inference runtime telemetry naming
• Data visible in Glean dashboards

Comment 1

[Greg Tatum [:gregtatum]]

This is getting removed in Bug 2029729.