1597492 - Add Pocket tile type to assa_events_daily ping

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement, P1)

Description

[Kirill Demtchouk]

In order for Mozilla data scientists to look at Pocket New Tab metrics when running any experiment in Firefox (via Mozanalysis), we need a click event that has both 1) the main Firefox telemetry client_id and 2) differentiates between organic content and sponsored content.

We are currently sending two types of click events when a user clicks a Pocket tile in the New Tab page. One contains a content identifier (Level 3 data) and does not contain client_id. One does not contain a content identifier, but does have the client_id.

We propose adding a field to this latter event that indicates whether organic/sponsored content was clicked. This is the event that goes into the Redshift tiles db table called assa_events_daily.

To be clear, all that would be sufficient is something like:
sponsored
0
1
0
...

We do not seek to add a full content identifier to an event with the original client_id.

Comment 1

[Nan Jiang [:nanj]]

As assa_events_daily also records pings for other user events (e.g. search event, bookmark event), let's add this to the event context field value.

I'd recommend us to introduce a new key story_type to value, for example:

"value": {
    "card_type": "[ spoc | organic ]";
}

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5388

Comment 3

[Nan Jiang [:nanj]]

Attachment: data_review_request_card_type_spocs.txt

Hey Kenny,

Could you take a look at this telemetry addition, please?

You can find the document here.

Thanks

Comment 4

[Nan Jiang [:nanj]]

Hey chutten,

Would you be able to take a look at this request, please? This telemetry change is about Pocket, but Kenny is on PTO for this whole week.

Thank you!

Comment 5

[Chris H-C :chutten]

Comment on attachment 9111334 [details]
data_review_request_card_type_spocs.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is documented in [the Activity Stream repo](https://github.com/mozilla/activity-stream/blob/master/docs/v2-system-addon/data_events.md).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences or Activity Stream's preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Kirill Demtchouk and Nan Jiang are responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 6

[Nan Jiang [:nanj]]

Thanks for reviewing this, chutten!

It's now github-merged.

Comment 7

[Jessilyn Davis]

Nan, can you help get this exported and uplifted into 72? We'll need it for our streamlining our analytics for the beginning of the year.

Comment 8

[Nan Jiang [:nanj]]

Yep, sure thing. Will get it uplifted to 72.

Comment 9

[Jessilyn Davis]

Thanks, Nan!

Comment 10

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/ef380e3e542c

Comment 11

[Nan Jiang [:nanj]]

[Tracking Requested - why for this release]:

This telemetry addition for the newtab page is necessary for the upcoming experiments that will be conducted in Firefox 72.

Comment 12

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5393

Comment 13

[Nan Jiang [:nanj]]

Attachment: Bug 1597492 - Add Pocket tile type to Top Story click ping

Comment 14

[Nan Jiang [:nanj]]

Comment on attachment 9114174 [details]
Bug 1597492 - Add Pocket tile type to Top Story click ping

Beta/Release Uplift Approval Request

• User impact if declined: We cannot fully study the user engagement change for various newtab page experiments scheduled in Firefox 72.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): A minor addition to the existing telemetry collection, not a user-facing change.
• String changes made/needed: None

Comment 15

[Julien Cristau [:jcristau]]

Comment on attachment 9114174 [details]
Bug 1597492 - Add Pocket tile type to Top Story click ping

telemetry update for pocket new tab, approved for 72.0b5

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/810b2df0a3c4

Comment 17

[Valentina Virlics [:vvalentina], Ecosystem QA]

I have verified that the issue is no longer reproducible on Firefox Nightly 73.0a1 (Build ID 20191210212905) and Firefox Beta 72.0b5 (Build ID 20191210230245) using Win 10, Mac 10.14 and Linux Ubuntu 14.06.
Card type field with correct values is displayed for Pocket "CLICK" events in browser console. Other actions on Pocket cards (bookmark, Save to Pocket, dismiss, etc.) don't generate card type values.