2003190 - Phase 2 POC - Part 1: Implement Policy Engine and Enforcement for Single Policy

Status: RESOLVED FIXED

(Core :: Machine Learning: General, enhancement)

Description

[Randy Concepcion]

• Please refer to Jira Ticket for more details: https://mozilla-hub.atlassian.net/browse/GENAI-2335
• Please refer to the security layer architecture slides for more information

Acceptance Criteria

• Policy engine evaluates the "block-unseen-links" rule and returns correct allow/deny decisions based on link ledger and context taint status
• Fail-safe behavior works correctly: tainted/unknown contexts fail-closed, clean contexts fail-open
• Preference to enable/disable enforcement and returns to pass-through behavior (for development/testing use)
• Unit and integration tests validate policy evaluation, taint detection, link ledger tracking, and fail-safe scenarios

---

NOTE:

• The original code exists in https://github.com/Firefox-AI/firefox-prototypes/pull/124 for testing purposes since it is integrated with the AI Window frontend.
• The only code that will be migrated to mozilla-central for now only includes the necessary files needed for the AI Window security layer component. Integration points with the frontend for AI Window are excluded since that is in the process of being migrated into mozilla-central. These security layer changes going into mozilla-central are isolated and should not affect other browser components.

Comment 1

[Randy Concepcion]

Attachment: browser console test for security layer + policy enforcement

Adding the Policy Engine to the ML Security Layer. These tests show that the orchestrator component is instantiated with policy enforcement tests to allow and deny requests.

Additionally, these are the logs in the terminal output to support the browser console output:

console.warn: "[Security] Loaded 1 policies from tool-execution-policies.json"
console.warn: "[Security] Policy loading complete: 1 loaded, 0 failed, 1 phases"
console.warn: "[Security] Orchestrator initialized for session test-session-123 with 1 policies"
console.log: "Orchestrator created:" [object Object]
console.log: "Seeded URLs:" ["https://example.com/page"]
console.warn: "[PolicyEvaluator] checkMatch criteria:" "{\"action.type\":\"tool.call\",\"action.tool\":\"get_page_content\"}" "action:" "{\"type\":\"tool.call\",\"tool\":\"get_page_content\",\"urls\":[\"https://example.com/page\"],\"tabId\":\"test-tab-1\"}"
console.warn: "[PolicyEvaluator] No policies applied to action:" "tool.call" "get_page_content"
console.log: "Allow result:" ({effect:"allow"})
console.warn: "[PolicyEvaluator] checkMatch criteria:" "{\"action.type\":\"tool.call\",\"action.tool\":\"get_page_content\"}" "action:" "{\"type\":\"tool.call\",\"tool\":\"get_page_content\",\"urls\":[\"https://evil.com/phishing\"],\"tabId\":\"test-tab-1\"}"
console.warn: "[ConditionEvaluator] URL not in ledger: https://evil.com/phishing" "All URLs must be present in the request-scoped ledger (merged from current tab + @mentioned tabs)"
console.warn: "[PolicyEvaluator] Policy block-unseen-links denied action:" "URL not in selected request context"
console.log: "Deny result:" ({effect:"deny", policyId:"block-unseen-links", code:"UNSEEN_LINK", reason:"URL not in selected request context", details:{policyId:"block-unseen-links", failedCondition:"allUrlsIn", conditionDescription:"All URLs must be present in the request-scoped ledger (merged from current tab + @mentioned tabs)"}})
console.log: "Normalized:" ({success:true, url:"https://example.com/Path?query=1"})

Comment 2

[Randy Concepcion]

Attachment: Bug 2003190 - Add policy engine to security layer r=tarek,#ai-ondevice-reviewers

Implements a policy-based security layer for the ML engine to protect against prompt injection attacks.
The security layer validates tool execution requests against a URL ledger that tracks user-authorized URLs.

Comment 3

[Randy Concepcion]

Attachment: AI Mode_ Client-Side LLM Security Layer POC.pdf

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/ecc2f60f296d
https://hg.mozilla.org/integration/autoland/rev/6fa729075adf
Add policy engine to security layer r=ai-ondevice-reviewers,gregtatum

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/6fa729075adf