1550119 - Send spoc shim to the SPOC telemetry calls

Status: VERIFIED FIXED

(Firefox :: New Tab Page, task, P1)

Description

[Scott [:thecount] Downe]

In addition to existing telemetry, we will also send SPOC-specific impression/viewable/click events to the Pocket Proxy. These events will also need to have the shim appended. For the shim work, see: https://bugzilla.mozilla.org/show_bug.cgi?id=1550118

Comment 1

[Matt Koidin]

A change to this ticket (which might suggest we file a new ticket -or- rename).

We are no longer going to fire SPOC-specific telemetry directly to the Pocket Proxy.

Instead, we will rely on the existing telemetry pipeline and rely on our telemetry infrastructure in Google Cloud to deliver the SPOC-specific events.

The only change needed in the Firefox client is to update the SPOC telemetry calls to include the "shim" value that will be included with each SPOC.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5064

Comment 3

[Scott [:thecount] Downe]

Attachment: Data review for spoc shim

Comment 5

[klong]

Comment on attachment 9068081 [details]
Data review for spoc shim

My review is below. This needs to be escalated to legal due to: (1) new id included, (2) potential category 3 data, and (3) 3rd-party collection tool.

1) Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

* PR: https://github.com/mozilla/activity-stream/pull/5064/files#diff-f92d07e23fd9de1eb2f4ad96bd041287
* Data Dictionary: https://github.com/mozilla/activity-stream/blob/258b3ad76d5284c87ef209d9b2c5aaddfffc8113/docs/v2-system-addon/data_dictionary.md


2) Is there a control mechanism that allows the user to turn the data collection on and off?

The user can opt-out the data collection by either disabling the telemetry of Activity Stream or disabling the Firefox telemetry as a whole. You could also opt out of sponsored content, and it would no longer send any pings related to sponsored content, including the shim. Also if you turned off Activity Stream or Discovery Stream, it would also opt out.


3) If the request is for permanent data collection, is there someone who will monitor the data over time?

The Pocket ad ops team (Lillian Bui) and analytics (Kirill Demtchouk) will monitor this data over time.


4) Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

This could be considered either Category 2 or Category 3 data. It is data about the loading of and interactions with the Firefox new tab page, but the impressions and clicks do imply web activity.


5) Is the data collection request for default-on or default-off?

Default on.


6) Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

Yes, click and impression events will now include a "shim", which is a base64 encoded id unique to the impression from the Ad server.


7) Is the data collection covered by the existing Firefox privacy notice?

Unsure.


8) Does there need to be a check-in in the future to determine whether to renew the data?

No, this is a technical requirement of the Ad server to work. It needs it to properly serve sponsored content and must be continued as long as the ad server is used.


9) Does the data collection use a third-party collection tool?

Yes, AdZerk will generate the shim, and receive it in anonymized click and impression events.

Comment 6

[Alicia Gray]

Hi Kenny,
Reviewing.....

Comment 7

[Alicia Gray]

Trust data review = r+.

The basic properties of the new tab collection are still being maintained even with the addition of the the shim. While this could hypothetically leak web activity we have a set of mitigations in place (only associated with the Pocket ID and not client_id and only the ad that was clicked with no information on the triggering parameters). Ok as part of the default on Cat 2 data.

Comment 8

[Scott [:thecount] Downe]

To test:

1. Set this pref browser.newtabpage.activity-stream.discoverystream.endpoints to "https,http"
2. Set this pref browser.newtabpage.activity-stream.discoverystream.config to {"api_key_pref":"extensions.pocket.oAuthConsumerKey","collapsible":true,"enabled":true,"show_spocs":true,"hardcoded_layout":false,"personalized":false,"layout_endpoint":"https://gist.githubusercontent.com/ScottDowne/b85c57830405db12a18222ca62b4a3f6/raw/8febea501f10ad43aa2ca9f90b531bdd40136127/adzerk-layout"}
3. Ensure you can see telemetry events
4. Open a new tab, you should see a layout of weird cards, heros, and lists.
5. You should see 6 spocs total.
6. Open your jsdebugger
7. Ensure click and view ping events have the shim prop being passes in for spocs, and not for regular cards.

Comment 9

[Ciprian Muresan [:cmuresan], Ecosystem QA]

It looks like the github counterpart was merged 4 days ago. @Scott, was this merged into Nightly as well? Or is it waiting for an export?

Comment 10

[Scott [:thecount] Downe]

Yup, just waiting on export :)

Comment 11

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/719da47cffec

Comment 12

[Scott [:thecount] Downe]

Updated steps to test because these old ones relied on a GET request which is now a POST request as of bug 1550115

New steps to test:

1. Clone this repo locally https://github.com/ScottDowne/test-data

2. Run npm install

3. Run node start.js

4. Set this pref browser.newtabpage.activity-stream.discoverystream.endpoints to "https,http"

5. Set this pref browser.newtabpage.activity-stream.discoverystream.config to {"api_key_pref":"extensions.pocket.oAuthConsumerKey","collapsible":true,"enabled":true,"show_spocs":true,"hardcoded_layout":false,"personalized":false,"layout_endpoint":"https://gist.githubusercontent.com/ScottDowne/1a08ca9f15ceb4ea0aafb9532e5d1381/raw/eef821bf4f40594d3c12ea684df3bdef8508a84e/adzerk-layout-2"}

6. Ensure you can see telemetry events

7. Open a new tab, you should see a layout of weird cards, heros, and lists.

8. You should see 6 spocs total.

9. Open your jsdebugger

10. Ensure click and view ping events have the shim prop being passes in for spocs, and not for regular cards. The shim prop might look close to this:

"shim":{"click":"spoc shim 6 click","impression":"spoc shim 6 impression","delete":"spoc shim 6 delete","save":"spoc shim 6 save"}

or

"shim": "spoc shim 6 click"

The numbers in there might change depending on which spoc you clicked on, and if it was a click or a view. You might also see all the data if 1558903 has not landed yet.

Comment 13

[Ciprian Muresan [:cmuresan], Ecosystem QA]

Marking this issue as verified as a part of bug 1558905 and bug 1558903.

Comment 14

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Comment on attachment 9068081 [details]
Data review for spoc shim

Clearing data-review? per Alicia's approval in comment 7: https://bugzilla.mozilla.org/show_bug.cgi?id=1550119#c7