1645906 - Telemetry for navigation/topics component

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement, P1)

Description

[Scott [:thecount] Downe]

Right now we don't have much in the way of click and impression data on the topics component.

We wanted an experiment coming up soon so we should probably get this fixed so we can track it.

Comment 1

[Kirill Demtchouk]

We need to send an event ping for when users click on a popular topic. We do not currently do this.

An event ping with event = “CLICK” and source = “POPULAR_TOPICS” would work.

I don't think we need to send an impression. For now we can proxy with the existing tile impression events.

Comment 2

[Gavin Suntop [:gvn] (they/them)]

Attachment: Bug 1645906 - Adding telemetry to Popular Topics

Comment 3

[Gavin Suntop [:gvn] (they/them)]

Test steps:

• Click on a popular topic link on newtab
• Ensure ping matches the spec above (https://github.com/mozilla/activity-stream/blob/master/content-src/asrouter/docs/debugging-docs.md#how-to-test-telemetry-pings)

Comment 4

[Kirill Demtchouk]

Attachment: request.txt

Attaching Data Collection request for Kenny to review.

Comment 5

[Jessilyn Davis]

Will want to request this for uplift into Fx 79 Beta as soon as we can (but definitely by 14 July).

Comment 6

[klong]

Comment on attachment 9160148 [details]
request.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

The event is documented in: browser/components/newtab/docs/v2-system-addon/data_events.md

The specific change is in this PR: https://phabricator.services.mozilla.com/D81236

And will be available here when merged: https://firefox-source-docs.mozilla.org/browser/components/newtab/docs/v2-system-addon/data_events.html

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. Users can opt-out the data collection by either disabling the telemetry of Activity Stream or disabling the Firefox telemetry as a whole.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Kirill Demtchouk is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Kirill has classified this as Category 3 because it tracks clicks on "topic" names (currently: must-reads, productivity, health, finance, technology, more-recommendations). It is unclear to me whether this qualifies as Category 3 or Category 2, but to be safe I am escalating this for review/approval.

    Is the data collection request for default-on or default-off?

Default on for all channels and countries/locales that have Pocket new tab recs enabled.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

    Does the data collection use a third-party collection tool?

No.

---
Result: Pending

Comment 7

[Alicia Gray]

Trust review notes: This collection includes information about the user’s direct engagement with the service in which there is no or little risk of personal identification. My evaluation is that tracking clicks on "topic" names (currently: must-reads, productivity, health, finance, technology, more-recommendations) is information about the user’s in-product journeys and product choices helpful to understand engagement (attitudes) for example, selections of add-ons or tiles to determine potential interest categories etc.

Therefore, this is reasonably Cat 2 collection.

Comment 8

[klong]

Comment on attachment 9160148 [details]
request.txt

Updated to data-review+ based on agray's input in comment 7

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b4cd85ea898
Adding telemetry to Popular Topics r=nanj

Comment 10

[Gavin Suntop [:gvn] (they/them)]

[Tracking Requested - why for this release]:

Comment 11

[Gavin Suntop [:gvn] (they/them)]

Comment on attachment 9159525 [details]
Bug 1645906 - Adding telemetry to Popular Topics

Beta/Release Uplift Approval Request

• User impact if declined: None.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: https://bugzilla.mozilla.org/show_bug.cgi?id=1645906#c3
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is only an added telemetry event. Reporting mechanism is already in place and there is no user facing functionality added here.
• String changes made/needed:

Comment 12

[Razvan Maries]

Backed out for node perma failures.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=linux%2C18.04%2Cx64%2Copt%2Cnode%2Ctests%2Csource-test-node-newtab-unit-tests%2Cnode%28newtab%29&revision=3b4cd85ea898f1680991e95fc5a851dd5cca432d&selectedTaskRun=Z1CEbC9zRgKN2qr1BqjtWw.0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=308251340&repo=autoland&lineNumber=95

Backout: https://hg.mozilla.org/integration/autoland/rev/f043f319bb3ae66dc6985d4172f848df0a01bf6b

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3a2964b5bc99
Adding telemetry to Popular Topics r=thecount

Comment 14

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/3a2964b5bc99

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9159525 [details]
Bug 1645906 - Adding telemetry to Popular Topics

Approved for 79.0b5. We should try to verify the incoming Telemetry data as soon as we can, though.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/ed25a05854dc

Comment 17

[Marius Coman [:mcoman], Ecosystem QA]

This enhancement was part of the work done in PI-694 that was signed-off as GREEN. Considering this I am marking this as VERIFIED.