1857324 - Bridge Activity Stream's gap wrt New Tab Telemetry with new Glean instrumentation

Status: RESOLVED FIXED

(Firefox :: New Tab Page, task, P1)

Description

[Chris H-C :chutten]

We've got an idea of what the gap between Activity Stream's use cases and New Tab Telemetry's capabilities are. This bug is for bridging that gap with new instrumentation.

Specific tasks include and are limited to:

• Add sponsored tiles three-dot menu and show privacy info clicks to newtab ping
• Add sponsored tiles dismissals to newtab ping
• Add content identifiers to Pocket Glean pings (bug 1854245)
• Add PREF_CHANGED events to the newtab ping for sponsored topsites disabled and topsites disabled
• Creating a Glean "pocket-button" ping that closely reinstruments the activity_stream.pocket_button ping (no external bug)
• Add Advertiser ID for desktop SPOCs to newtab ping
• Add advertiser ID for tiles and SPOCs desktop dismissals to newtab for ads consolidated report
• Add flight_id and creative_id to newtab for Kevel SPOCs impressions/clicks

(( Since all (but one) of these already have bugs, why are you filing another? Because they're all in JIRA and that's not where Firefox Desktop engineering happens. To my knowledge, Phabricator would get very irate if I tried to push a patch that tried to use a JIRA id instead of a bug number. To say nothing of the fact that any new/expanded collection here must be reviewed in public, and JIRA's many things but not that. ))

Aside from the one bug above, I intend to file patches against this singular bug. I'll aim for one patch per task to make discussions a little easier to keep straight.

Comment 1

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument the 'about our sponsors' link r?nanj!

See also: DENG-1364

Comment 2

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument 'dismiss' actions for newtab topsites r?nanj!

See Also DENG-1363

Depends on D190855

Comment 3

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument topsites and sponsored topsites pref changes r?nanj!

See Also DO-1293

We instrument both the settings UI being interacted with and the underlying
preferences being changed since the former knows more about where and how these
were changed, and the latter will catch non-user-initiated (programmatic)
changes.

Depends on D190856

Comment 4

[Chris H-C :chutten]

Attachment: Bug 1857324 - Add new "pocket-button" ping r?thecount!,gvn!

Depends on D190857

Comment 5

[Chris H-C :chutten]

Some notes:

• For https://mozilla-hub.atlassian.net/browse/DENG-1364 (three-dot and show privacy info), the three-dot part would be a new piece of instrumentation requiring some plumbing and not something I can easily just drop in. Sergio will file a new task for that.
• For bug 1854245 (content identifiers) I'm going to give it another look, but it's likely this isn't a simple request, and so it may need to be popped into the new request pipeline.

Progress:
As you can see I have four of the tasks up for review. There's a bit of a feedback cycle here as I have some questions in the form of TODO notes in the patches, but I don't expect that to appreciably add to the code review timeline. I haven't looked at the "add advertiser ID|flight_id|creative_id" ones yet, but they smell an awful lot like bug 1854245 (content identifiers) which requires rather a bit more plumbing (from service call to internal model to interaction model to user event to TelemetryFeed) than just a reinstrumentation.

Comment 6

[Chris H-C :chutten]

bug 1854245 was a bit of a pain, but I figured it out without too much additional time. So it gets a pass, despite being net new.

Comment 7

[Chris H-C :chutten]

All three ads requests appear to be novel collections without sufficient information behind them to be tackled. When the information's provided, I'll evaluate whether they can ride along or whether they require more expertise and/or bandwidth than I can bring to bear in newtab. For now, I'll remove them from the list of things I intend to do on this particular bug and begin preparations for data reviews.

Comment 8

[Chris H-C :chutten]

Attachment: data collection review request

Comment 9

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Comment on attachment 9358404 [details]
data collection review request

sorry, just about to head out on PTO and don't have the bandwidth for this large data-review today

Comment 10

[Travis Long [:travis_]]

Comment on attachment 9358404 [details]
data collection review request

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the data preferences in the application settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Permanent collection to be monitored by [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail]

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1 (Technical data) & Category 2 (Interaction data)

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+

Comment 11

[Chris H-C :chutten]

Attachment: Bug 1857324 - Remove old pocket_button ping r?thecount!,gvn!

Depends on D190858

Comment 12

[Chris H-C :chutten]

Attachment: Bug 1857324 - tile_id is a number not a string r?nanj!

Depends on D190980

Comment 13

[Rob Miller [:rmiller]]

(In reply to Chris H-C :chutten from comment #7)

All three ads requests appear to be novel collections without sufficient information behind them to be tackled. When the information's provided, I'll evaluate whether they can ride along or whether they require more expertise and/or bandwidth than I can bring to bear in newtab. For now, I'll remove them from the list of things I intend to do on this particular bug and begin preparations for data reviews.

It's not clear to me from this comment what information you need(ed) to be able to work on this. Can you give any more context on that you're missing? Did you receive any of that info? Who would (theoretically) be providing this info?

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ceea7096f86c
Instrument the 'about our sponsors' link r=nanj
https://hg.mozilla.org/integration/autoland/rev/fcae1312285f
Instrument 'dismiss' actions for newtab topsites r=nanj
https://hg.mozilla.org/integration/autoland/rev/42dffec298e7
Instrument topsites and sponsored topsites pref changes r=nanj
https://hg.mozilla.org/integration/autoland/rev/d62bc5378d5d
Add new "pocket-button" ping r=thecount
https://hg.mozilla.org/integration/autoland/rev/c0e54c7cb896
tile_id is a number not a string r=nanj
https://hg.mozilla.org/integration/autoland/rev/0331678f7bf6
Remove old pocket_button ping r=thecount

Comment 15

[Chris H-C :chutten]

(In reply to Rob Miller [:rmiller] from comment #13)

(In reply to Chris H-C :chutten from comment #7)

All three ads requests appear to be novel collections without sufficient information behind them to be tackled. When the information's provided, I'll evaluate whether they can ride along or whether they require more expertise and/or bandwidth than I can bring to bear in newtab. For now, I'll remove them from the list of things I intend to do on this particular bug and begin preparations for data reviews.

It's not clear to me from this comment what information you need(ed) to be able to work on this. Can you give any more context on that you're missing? Did you receive any of that info? Who would (theoretically) be providing this info?

I asked some questions a couple weeks ago on the individual Jira items that Sergio partially answered and partially passed on to Curtis on Tuesday. The information from Sergio's answers confirm these are net-new, at least, meaning they aren't strictly gap-bridging metrics. Without the rest of the information about what these proposed new collections are (how they look, where they live in the code) they wouldn't be able to ride-along.

Comment 16

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ceea7096f86c
https://hg.mozilla.org/mozilla-central/rev/fcae1312285f
https://hg.mozilla.org/mozilla-central/rev/42dffec298e7
https://hg.mozilla.org/mozilla-central/rev/d62bc5378d5d
https://hg.mozilla.org/mozilla-central/rev/c0e54c7cb896
https://hg.mozilla.org/mozilla-central/rev/0331678f7bf6

Comment 17

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument the 'about our sponsors' link

See also: DENG-1364

Original Revision: https://phabricator.services.mozilla.com/D190855

Comment 18

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument 'dismiss' actions for newtab topsites

See Also DENG-1363

Depends on D190855

Original Revision: https://phabricator.services.mozilla.com/D190856

Depends on D192235

Comment 19

[Chris H-C :chutten]

Attachment: Bug 1857324 - Instrument topsites and sponsored topsites pref changes

See Also DO-1293

We instrument solely the underlying preferences being changed.

Depends on D190856

Original Revision: https://phabricator.services.mozilla.com/D190857

Depends on D192236

Comment 20

[Chris H-C :chutten]

Attachment: Bug 1857324 - Add new "pocket-button" ping

Depends on D190857

Original Revision: https://phabricator.services.mozilla.com/D190858

Depends on D192237

Comment 21

[Chris H-C :chutten]

Attachment: Bug 1857324 - tile_id is a number not a string

Depends on D190980

Original Revision: https://phabricator.services.mozilla.com/D191341

Depends on D192239

Comment 22

[Chris H-C :chutten]

Attachment: Bug 1857324 - Remove old pocket_button ping

Depends on D191341

Original Revision: https://phabricator.services.mozilla.com/D191238

Depends on D192241

Comment 23

[Phabricator Automation]

Uplift Approval Request

• Fix verified in Nightly: yes
• Needs manual QE test: no
• Explanation of risk level: changed instrumentation only
• Risk associated with taking this patch: small
• String changes made/needed: n/a
• Is Android affected?: no
• Code covered by automated testing: yes
• User impact if declined: none (delayed instrumentation)
• Steps to reproduce for manual QE testing: N/A

Comment 24

[Dianna Smith [:diannaS]]

Comment on attachment 9361012 [details]
Bug 1857324 - Instrument the 'about our sponsors' link

Approved for 120.0b5

Comment 25

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/9c37e5f38cb0
https://hg.mozilla.org/releases/mozilla-beta/rev/c0e5091d5acf
https://hg.mozilla.org/releases/mozilla-beta/rev/6d1b4b7bd752
https://hg.mozilla.org/releases/mozilla-beta/rev/bbc3d20239fd
https://hg.mozilla.org/releases/mozilla-beta/rev/bfa41d1b6b41
https://hg.mozilla.org/releases/mozilla-beta/rev/d0dabe2c8165