1681387 - Add Pocket and Spocs config telemetry to new config buttons

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement, P1)

Description

[Scott [:thecount] Downe]

Bug 1671943 is adding a new config page. We probably want telemetry that matches the telemetry we added to bug 1672518.

Should be mostly just a port of the dispatch inside the new buttons onchanges.

Comment 1

[Scott [:thecount] Downe]

[Tracking Requested - why for this release]:

Comment 2

[Julien Cristau [:jcristau]]

I don't think I need to track this.

Comment 3

[Scott [:thecount] Downe]

Attachment: Bug 1681387 - Add Pocket and Spocs config telemetry to new config buttons

Comment 4

[Scott [:thecount] Downe]

To test:

1. Load up a fresh profile with the multi process debugger open.
2. Set browser.newtabpage.activity-stream.newNewtabExperience.enabled to true
3. Set browser.ping-centre.log to true
4. Set browser.newtabpage.activity-stream.telemetry to true.
5. Open a new tab
6. Open the "personalize" menu.
7. Toggle the "sponsored stories", "Pocket", and "recent activity" sections on and off.

expected: should see some telemetry events being fired in the multi process debugger.

Comment 5

[Kirill Demtchouk]

Attachment: request.txt

Comment 6

[klong]

Comment on attachment 9193372 [details]
request.txt

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Scott Downe has added public documentation as part of this PR, but it is not yet merged:
https://phabricator.services.mozilla.com/D99831

It will eventually live here:
https://firefox-source-docs.mozilla.org/browser/components/newtab/docs/v2-system-addon/data_events.html#basic-shape

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. Users can opt-out the data collection by disabling the Firefox telemetry.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Kirill Demtchouk ([e-mail]) will be monitoring those metrics.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

This classified this as Category 2. It adds a new param to an existing event to indicate how the user disables/enables Pocket recommendations and SPOCs in the new tab.

Is the data collection request for default-on or default-off?

Default on for all channels and countries/locales.

Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

Does the data collection use a third-party collection tool?

No.

---

Result: r+

Comment 7

[Scott [:thecount] Downe]

[Tracking Requested - why for this release]: Going for uplift on this one. It's needed for an experiment.

Comment 8

[Scott [:thecount] Downe]

Comment on attachment 9193349 [details]
Bug 1681387 - Add Pocket and Spocs config telemetry to new config buttons

Beta/Release Uplift Approval Request

• User impact if declined: Impacts an experiment for a new about:home config page. We might not get all the metrics we want, delaying the experiment, thus delaying the improved config page, thus delaying the user from accessing it.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: 1. Load up a fresh profile with the multi process debugger open.

2. Set browser.newtabpage.activity-stream.newNewtabExperience.enabled to true
3. Set browser.ping-centre.log to true
4. Set browser.newtabpage.activity-stream.telemetry to true.
5. Open a new tab
6. Open the "personalize" menu.
7. Toggle the "sponsored stories", "Pocket", and "recent activity" sections on and off.

expected: should see some telemetry events being fired in the multi process debugger.

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's adding a function call to dispatch some telemetry to match similar telemetry that we're already using.
• String changes made/needed: none

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/60f2d34d1df1
Add Pocket and Spocs config telemetry to new config buttons r=nanj,ewright

Comment 10

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/60f2d34d1df1

Comment 11

[Robert Martin [:romartin], Ecosystem QA]

I have verified the following with the steps provided in comment 8 using Firefox Nightly 86.0a1 (Build ID: 20201222215026) on Windows 10 x64, macOS 10.15.6, and Ubuntu Linux 20.04 x64:

• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is toggled off using the "Customize menu".
  • The "menu_source" for this ping is "CUSTOMIZE_MENU".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is toggled on using the "Customize menu".
  • The "menu_source" for this ping is "CUSTOMIZE_MENU".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is unchecked in the "about:preferences#home" page.
  • The "menu_source" for this ping is "ABOUT_PREFERENCES".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is checked in the "about:preferences#home" page.
  • The "menu_source" for this ping is "ABOUT_PREFERENCES".

I have also verified that the pings are successfully sent and received using a local Telemetry Server.

Comment 12

[Julien Cristau [:jcristau]]

Comment on attachment 9193349 [details]
Bug 1681387 - Add Pocket and Spocs config telemetry to new config buttons

approved for 85.0b5

Comment 13

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/90ecbdf95cfa

Comment 14

[Robert Martin [:romartin], Ecosystem QA]

I have verified the following with the steps provided in comment 8 using Firefox Beta 85.0b5 (Build ID: 20210105185604) on Windows 10 x64, macOS 10.15.6, and Ubuntu Linux 20.04 x64:

• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is toggled off using the "Customize menu".
  • The "menu_source" for this ping is "CUSTOMIZE_MENU".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is toggled on using the "Customize menu".
  • The "menu_source" for this ping is "CUSTOMIZE_MENU".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is unchecked in the "about:preferences#home" page.
  • The "menu_source" for this ping is "ABOUT_PREFERENCES".
• A "PREF_CHANGE" telemetry ping is displayed in the "Browser Console" when the "Recommended by Pocket", "Sponsored Stories" section from "Recommended by Pocket", or "Recent activity" is checked in the "about:preferences#home" page.
  • The "menu_source" for this ping is "ABOUT_PREFERENCES".

I have also verified that the pings are successfully sent and received using a local Telemetry Server.