1862670 - Add `tile_id` and `shim` to pocket impressions, clicks, saves [more gap bridging]

Status: RESOLVED FIXED

(Firefox :: New Tab Page, task, P1)

Description

[Chris H-C :chutten]

bug 1857324 bridged most of the gap needed to cover Activity Stream decisionmaking with New Tab Telemetry. Since its landing, clarification on what exact ids are and are not needed by pocket have come in, and wouldn't you know: they're gaps needing bridging.

This bug is about adding the tile_id to pocket.impression, pocket.click, and pocket.save.

What is the tile_id? Well, in existing instrumentation it is the id of the tile.

What does the id mean? I'm not sure. ni?kirill to please provide a description of the tile_id so I have something to put in the metrics.yaml.

Comment 1

[Kirill Demtchouk]

The tile_id is a content identifier. It is slightly different for organic Pocket recommendation and sponsored content. For sponsored content, it is the ad ID from Kevel. For organic recs, it is a content identifier produced by Pocket's recommendation systems.

Comment 2

[Chris H-C :chutten]

Attachment: Bug 1862670 - Add tile_id to pocket impressions|saves|clicks r?thecount!,mmiermans!

Comment 3

[Chris H-C :chutten]

Attachment: Bug 1862670 - Submit "spoc" ping with shim for pocket impressions|saves|clicks r?thecount!

Depends on D192606

Comment 4

[Chris H-C :chutten]

We also need shim for server-side processing.

Comment 5

[Chris H-C :chutten]

Attachment: data collection review request

For prior art on shim collections, consider Fenix's review

Comment 6

[klong]

Comment on attachment 9363345 [details]
data collection review request

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. These collections are Glean. The opt-out can be found in the product's preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, [e-mail], [e-mail], [e-mail], [e-mail], [e-mail] will be responsible for the permanent collections.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2 (Interaction) and Category 3 (Stored Content)

Is the data collection request for default-on or default-off?

Yes, default on for all visitors (All channels, countries, and locales. No filters.)

Does the instrumentation include the addition of any new identifiers?

The tile_id and shim are being added to these events, though they are arguably not new:

The tile_id is being re-implemented (with a new collection method), which may act as precedent.

The shim is being added, however it has been added on Fenix already.

Is the data collection covered by the existing Firefox privacy notice?

Unsure. Assume yes because similar events have already been implemented, but don't know if our Fenix privacy notice is the same as our Firefox privacy notice.

Because: (1) I am unsure if it is covered by the existing privacy policy, (2) it may include new identifiers, and (3) it relates to category 3 data, this needs to be escalated to legal.

Does the data collection use a third-party collection tool?

No.

Comment 7

[klong]

@chutten - Did I misinterpret any of the facts (or missed separate context) about this? If not, I believe this needs to follow the normal escalation process.

Comment 8

[Chris H-C :chutten]

Per https://bugzilla.mozilla.org/show_bug.cgi?id=1854245#c20, Kirill took care of requesting Sensitive review for these data, I believe.

Comment 9

[klong]

Ah, thank you. Awaiting some clarification from @nneka, will update as soon as I hear back.

Comment 10

[klong]

Added @kirill to thread with Nneka to facilitate bringing result back into this bug (since I'll likely be out on leave when she completes it early next week).

Comment 11

[Kirill Demtchouk]

The latest response from Nneka is:
"Hello! I’ve followed up with my colleagues who were diving into things in my absence. I think you all should be good to go soon, but awaiting technical confirmation from Mark Reid based on our privacy analysis."

Seems like we're still waiting on final, final approval for the shim change. tile_id is good to go, as per previous comments in this bug.

Comment 12

[Kirill Demtchouk]

We have the green light from Nneka!

"Happy to report y’all have the green light! Currently doing dinner time with my toddler, but can try to comment in the bug tonight. Otherwise ping me in the morning if it hasn’t been done"

Comment 13

[Nneka]

Confirming that what Kirill copied is indeed from me. To add some clarity, this shim is tied to the advertising ID which we do already provide notice for and honor opt-outs via the device. Thanks!

Comment 14

[Chris H-C :chutten]

Comment on attachment 9363345 [details]
data collection review request

Flipping to data-review+ on completion of Sensitive review.

This is ready to land (though I may need to rebase first)

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/96b375d4cc8e
Add tile_id to pocket impressions|saves|clicks r=thecount
https://hg.mozilla.org/integration/autoland/rev/23a76599562a
Submit "spoc" ping with shim for pocket impressions|saves|clicks r=thecount

Comment 16

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/96b375d4cc8e
https://hg.mozilla.org/mozilla-central/rev/23a76599562a

Comment 17

[Chris H-C :chutten]

Attachment: Bug 1862670 - Add tile_id to pocket impressions|saves|clicks

Original Revision: https://phabricator.services.mozilla.com/D192606

Comment 18

[Chris H-C :chutten]

Attachment: Bug 1862670 - Submit "spoc" ping with shim for pocket impressions|saves|clicks

Original Revision: https://phabricator.services.mozilla.com/D192702

Depends on D195450

Comment 19

[Phabricator Automation]

Uplift Approval Request

• Risk associated with taking this patch: low
• Is Android affected?: no
• Code covered by automated testing: yes
• Steps to reproduce for manual QE testing: n/a
• Fix verified in Nightly: yes
• Needs manual QE test: no
• String changes made/needed: none
• User impact if declined: none
• Explanation of risk level: Not user-visible, instrumentation-only, tested, verified in nightly.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9366890 [details]
Bug 1862670 - Add tile_id to pocket impressions|saves|clicks

Approved for 121.0b8.

Comment 21

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/432314763032
https://hg.mozilla.org/releases/mozilla-beta/rev/7db3ae9df06c