Open Bug 2092649 Opened 5 days ago Updated 2 days ago

move the smart actions toolbar prompt into Remote Settings as its own Smart Window feature

Categories

(Core :: Machine Learning: Frontend, enhancement)

enhancement
Points:
5

Tracking

()

People

(Reporter: Mardak, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [aiasst])

Bug 2091509 adds the smart actions toolbar prompt as a long default in the browser.smartwindow.actionsToolbar.prompt pref in firefox.js. generateFromPrompt in SmartActionsToolbar.sys.mjs runs it with buildConversation(MODEL_FEATURES.CHAT). Raised in review of D342411.

Other Smart Window prompts live in the ai-window-prompts Remote Settings collection and are loaded with loadPrompt, and browser.smartwindow.customPrompts already allows local overrides. The toolbar prompt should work the same way:

  • add its own MODEL_FEATURES entry with a FEATURE_MAJOR_VERSIONS entry and a record in ai-window-prompts (and the local dump) holding the prompt, model and inference parameters
  • map it to its own purpose in FEATURE_PURPOSES, so these requests are not counted as user chats by MLPA or by the model_request and model_response telemetry, and do not run on whichever chat model the user picked
  • load it with loadPrompt, so browser.smartwindow.customPrompts overrides it, then remove the browser.smartwindow.actionsToolbar.prompt pref, keeping a way to switch back to the sidebar conversation starters if that is still wanted
  • decide where the placeholder limits such as {page_text|10000} and {open_tabs|5} belong: kept in the template, or moved to the record or to prefs
  • keep the JSON output schema and the allowed action types in code, since the toolbar actions depend on them

Which context the default prompt sends, page text in particular, depends on the security review requested in D342411.

conversation starters prompt has been pinned to qwen

we'll need to decide if we want to pin these if we keep the llm generated actions. this landed as reusing the assistant model primarily to make it easy to get a feel for how different models generate actions

You need to log in before you can comment on or make changes to this bug.