2018261 - [Models] [Convo Starters] History search starter is non-actionable

Status: NEW

(Core :: Machine Learning: General, defect)

Description

[Tim Burrell]

Convo starter about history search triggered a non-actionable result.

Relevant discussion: https://mozilla.slack.com/archives/C0ADUFX0QRW/p1771606495619399

Maybe we either try to remove these or give it another few-shot sample on how to make those work better?

Probably add a few of these to eval and see if it's actually an issue.

Comment 1

[Tim Burrell]

Attachment: Screenshot_20260302_161140.png

Another instance of non-actionable follow ups

Comment 2

[Tim Burrell]

Added screenshot with another instance of this: "Are there any beginner classes starting soon" <-- requires a search and results in the assistant saying it can't answer the question.

We can't solve 100% but probably some prompting around avoiding cases like these could help

Comment 3

[Tim Burrell]

Attachment: Screenshot_20260302_162052.png

Added another one: "Help me fill out this form" is probably going to make the user think we can do it for them when we can't

Comment 4

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:tburrell, could you have a look please?

For more information, please visit BugBot documentation.