Closed Bug 1501304 Opened Last year Closed Last year

Add telemetry for CFR opt-out controls

Categories

(Firefox :: Messaging System, enhancement, P1)

63 Branch
enhancement

Tracking

()

VERIFIED FIXED
Firefox 65
Iteration:
65.2 - Nov 16
Tracking Status
firefox64 + verified
firefox65 --- verified

People

(Reporter: tspurway, Assigned: rrosario)

References

(Blocks 1 open bug)

Details

Attachments

(3 files)

We need new pings to track the new 'Not Now', 'Don't Show Me This Recommendation' and 'Manage Recommendation Settings' controls on the CFR door-hanger.
Priority: P2 → --
Iteration: --- → 65.1 (Nov 2)
Priority: -- → P2
Assignee: nobody → rrosario
Hi  :francois, this is for two additional user interactions for the ping reviewed in Bug 1484035. Below is the documentation for the ping:
https://github.com/mozilla/activity-stream/blob/a0eb5ba7d8af35af662b1b528fa4d234af0e7a83/docs/v2-system-addon/data_events.md#cfr-interaction-pings-for-all-the-prerelease-channels-and-shield-experiment

Please let me know if you have any other questions.
Attachment #9022285 - Flags: review?(francois)
Iteration: 65.1 (Nov 2) → 65.2 (Nov 16)
Priority: P2 → P1
Comment on attachment 9022285 [details]
Data review request form

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in https://github.com/mozilla/activity-stream/blob/master/docs/v2-system-addon/data_events.md.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, either by disabling telemetry entirely or just browser.newtabpage.activity-stream.telemetry.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, Ricky and Nan.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 2.

5) Is the data collection request for default-on or default-off?

Default ON.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.
Attachment #9022285 - Flags: review?(francois) → review+
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Blocks: 1505929
MozReview-Commit-ID: 35f9hn3MZNJ
Steps for QA Verification:

1- 2- Enable "browser.newtabpage.activity-stream.asrouter.devtoolsEnabled", "browser.newtabpage.activity-stream.telemetry" and "browser.ping-centre.log" in about:config

2- Go to AS Router Admin: about:newtab#asrouter

3- Select "cfr" next to "Show messages from"

4- Click the "Show" button for any of the messages below (FACEBOOK_CONTAINER_3, for example)

5- Click on the "Recommendation" chiclet, then on the down arrow, then on "Don't Show Me This Recommendation"

6- Open the browser console and verify there is a BLOCK ping. For example:

TELEMETRY PING: {"locale":"en-US","topic":"activity-stream-router","client_id":"3e0133ba-f9e4-d645-b509-eb2a21d3e05f","version":"64.0","release_channel":"default","addon_version":"20181114213203","impression_id":"n/a","action":"cfr_user_event","source":"CFR","message_id":"FACEBOOK_CONTAINER_1","event":"BLOCK","profile_creation_date":17849,"region":"US"}

7- Now, in about:newtab#asrouter, click the "Show" button for any of the messages below again

8- Click on the "Recommendation" chiclet, then on the down arrow, then on "Manage Recommendation Settings"

9 - Open the browser console and verify there is a MANAGE ping. For example:

TELEMETRY PING: {"locale":"en-US","topic":"activity-stream-router","client_id":"3e0133ba-f9e4-d645-b509-eb2a21d3e05f","version":"64.0","release_channel":"default","addon_version":"20181114213203","impression_id":"n/a","action":"cfr_user_event","source":"CFR","message_id":"GOOGLE_TRANSLATE_1","event":"MANAGE","profile_creation_date":17849,"region":"US"}

10- DONE! Thank you!
Flags: qe-verify+
Verified - Fixed on the latest Nightly 65.0a1 (2018-11-19) (64-bit) on Windows 7/10 x64, Mac OS 10.13 and Ubuntu 16.04.

By following the Steps provided in Comment 9, the Block and the Manage ping will be displayed in the browser console as expected.
Thanks for the detailed steps, marking it as verified for fx 65.
Comment on attachment 9025240 [details]
Bug 1501304 - Add telemetry for CFR opt-out controls

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1501304

User impact if declined: No direct user impact but it affects our analysis of the engagement of the Contextual Feature Recommender (CFR).

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: Yes

If yes, steps to reproduce: https://bugzilla.mozilla.org/show_bug.cgi?id=1501304#c9

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): This adds telemetry pings for two user interactions that were previously not sending pings. The only difference between these pings and other existing pings is the event type string.

String changes made/needed: None
Attachment #9025240 - Flags: approval-mozilla-beta?
Comment on attachment 9025240 [details]
Bug 1501304 - Add telemetry for CFR opt-out controls

AS telemetry, approved for 64.0b12
Attachment #9025240 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Verified - Fixed on the latest Beta 64.0b12 (Build ID: 20181122182000)on Windows 7/10 x64, Mac OS 10.14 and Ubuntu 16.04.

By following the Steps provided in Comment 9, the Block and the Manage ping will be displayed in the browser console as expected.
Marking this issue as Verified - Fixed and removing qe+ flag.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
Component: Activity Streams: Newtab → Messaging System
You need to log in before you can comment on or make changes to this bug.