1879127 - Add telemetry for Keep Closed card buttons

Status: VERIFIED FIXED

(Firefox Graveyard :: Shopping, enhancement, P1)

Description

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Record an event when the keep closed buttons are clicked.

In related bug 1879119, we add telemetry to record changes to the autoOpen.userEnabled pref, so we don't need to record that here.

Comment 1

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

[Tracking Requested - why for this release]: We're missing a few telemetry probes for new UI that's already been uplifted into 123.

Comment 2

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

whoops, assigned the wrong bug

Comment 3

[kpatenio]

Attachment: Bug 1879127 - add telemetry for keep closed card buttons. r=#shopping-reviewers!

Comment 4

[kpatenio]

Comment on attachment 9378835 [details]
Bug 1879127 - add telemetry for keep closed card buttons. r=#shopping-reviewers!

DATA REVIEW REQUEST

1. What questions will you answer with this data?

Measure user interaction with experimental auto-open shopping changes shipping in 123, specifically, record when users click on either the "Yes, keep closed" button to disable auto-open behaviour pref'ed behind browser.shopping.experience2023.autoOpen.enabled, or "No thanks" to leave it running.

2. Why does Mozilla need to answer these questions? Are there benefits for users?
   Do we need this information to address product or business requirements?

Product wants to track if users will keep or disable auto-open behaviour after closing the Review Checker sidebar several times during a session.

3. What alternative methods did you consider to answer these questions?
   Why were they not sufficient?

We need telemetry to measure clicks.

4. Can current instrumentation answer these questions?

No.

5. List all proposed measurements and indicate the category of data collection for each
   measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Name	Measurement Description	Data Collection Category	Tracking Bug
shopping.surface_no_thanks_button_clicked	The user clicks the 'No thanks' button when asked if they want to disable auto-open behavior.	interaction	https://bugzilla.mozilla.org/show_bug.cgi?id=1879127
shopping.surface_yes_keep_closed_button_clicked	The user clicks the 'Yes, keep closed' button when asked if they want to disable auto-open behavior.	interaction	https://bugzilla.mozilla.org/show_bug.cgi?id=1879127

6. Please provide a link to the documentation for this data collection which
   describes the ultimate data set in a public, complete, and accurate way.

This collection is Glean so is documented in the Glean Dictionary.

7. How long will this data be collected?

This collection has expiry '134'.

8. What populations will you measure?

All channels, countries, and locales. No filters.

9. If this data collection is default on, what is the opt-out mechanism for users?

These collections are Glean. The opt-out can be found in the product's preferences.

10. Please provide a general description of how you will analyze this data.

Internal dashboards.

11. Where do you intend to share the results of your analysis?

Internally.

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you
    are proposing to use for this data collection?

No.

Comment 5

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Comment on attachment 9378835 [details]
Bug 1879127 - add telemetry for keep closed card buttons. r=#shopping-reviewers!

data-review+, thanks!

---

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the firefox data opt-out.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A. Temporary collection through 134.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data.

5. Is the data collection request for default-on or default-off?

Default-on for an experimental feature that is preffed off except for a small Nimbus experimental population.

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

7. Is the data collection covered by the existing Firefox privacy notice?

Yes.

8. Does the data collection use a third-party collection tool?

No.

Comment 6

[kpatenio]

Comment on attachment 9378835 [details]
Bug 1879127 - add telemetry for keep closed card buttons. r=#shopping-reviewers!

Beta/Release Uplift Approval Request

• User impact if declined: We need to track user interactions with the auto-open feature we've added in 123. If declined, we won't be able to measure how many users interact with these two specific buttons.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This change adds telemetry probes to existing features. Besides, the features impacted by this change are pref'ed behind browser.shopping.experience2023.autoOpen.enabled, which is false by default.
• String changes made/needed:
• Is Android affected?: No

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/195d7ec5584d
add telemetry for keep closed card buttons. r=shopping-reviewers,niklas

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/195d7ec5584d

Comment 9

[Pascal Chevrel:pascalc]

Comment on attachment 9378835 [details]
Bug 1879127 - add telemetry for keep closed card buttons. r=#shopping-reviewers!

Approved for 123 beta 9, thanks.

Comment 10

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/c0153173d3d8

Comment 11

[Pascal Chevrel:pascalc]

https://hg.mozilla.org/releases/mozilla-beta/rev/c0153173d3d8

Comment 12

[Rares Doghi, Desktop QA]

Verified as fixed in our latest Nightly build 20240208095423.

Comment 13

[Rares Doghi, Desktop QA]

Verfied as fixed in our latest Beta build 123.0b9 20240209102425.