1895813 - Add telemetry for chat interactions and state

Status: RESOLVED FIXED

(Core :: Machine Learning, enhancement, P1)

Description

[Ed Lee :Mardak]

We'll likely want interaction like showing the menu from bug 1894994 and probably which item index of how many was selected from bug 1894998.

Related to turning on/off the feature from bug 1894999 and selecting provider perhaps also reporting back index of how many from bug 1895433. Potentially also include link clicks, e.g., connect, terms, privacy.

In addition to those above events, we probably also want status (at startup?) of whether the feature is enabled.

Comment 1

[Ania]

Attachment: Chatbot Integration Experiment Data Collection Request.txt

Hi Chris,
We were hoping to ask for the Data Stewardship help with the review of the probes we are hoping to land on June 24th-25th with the Nightly Chatbot integration experiment launch (available for users to enroll via about preferences#experimental). Please let us know if this timing is doable.

Here are some materials for additional context:

• PRD
• Figma designs to give an overview of what we're launching

Comment 2

[Ed Lee :Mardak]

Attachment: Bug 1895813 - Add telemetry for chat interactions and state r=tarek!

Add new metrics.yaml for various metrics and events.

Depends on D214355

Comment 3

[Ed Lee :Mardak]

Comment on attachment 9408302 [details]
Chatbot Integration Experiment Data Collection Request.txt

Attachment 9408767 [details] patch has probes that has more specifics to get at the data request's general approach of adding interaction metrics and events of the chatbot feature where basically we report booleans and string ids, e.g., prompt "quiz" or provider "chatgpt," and if the user customizes to other values, we report back "custom" instead.

Comment 4

[Ed Lee :Mardak]

If someone can make this a plain moco non-security bug, we can then make it a regular public non-moco bug next week

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fb2ffd6d5fcf
Add telemetry for chat interactions and state r=tarek

Comment 6

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/mozilla-central/rev/fb2ffd6d5fcf

Comment 7

[Jan-Erik Rediger [:janerik]]

Hey!
This collection landed in Nightly, but this bug is still confidential (and so is the phabricator patch).
As per our Data Collection guidelines we "want to make our decisions about data collection public and accessible" (Transparency). That includes the data review, which these days is handled directly in Phabricator.

I know that this bug will be made public soon (comment #4), but regardless this data is collected today from a population which does not have access to the data review.
Is there anything else stopping this to become a public bug now?

Comment 8

[Ed Lee :Mardak]

Yes, the old data review has mentions of information related to bug 1895497 which is also moco until public targeting later today.

Comment 9

[Rares Doghi, Desktop QA]

Verified as fixed in our latest Nightly builds 129.0a1 (2024-07-08).