Closed Bug 1514006 Opened 10 months ago Closed 9 months ago

Implement telemetry for pocket + newtab

Categories

(Firefox :: New Tab Page, enhancement, P1)

enhancement

Tracking

()

VERIFIED FIXED
Firefox 66
Iteration:
66.3 - Jan 7 - 20
Tracking Status
firefox66 --- fixed

People

(Reporter: nanj, Assigned: nanj)

References

Details

(Keywords: github-merged)

Attachments

(4 files)

No description provided.
We want to capture all the new metrics for the new Pocket + Newtab experience.

The proposed metrics are mostly based on the current telemetry for Top Stories, with an augment of following attributes:

* module ID (section ID)
* experiment information
  - experiment ID
  _ cohort ID
* user preferences
Blocks: 1512725
Blocks: 1514008
Blocks: 1513319
Assignee: nobody → najiang
Severity: normal → enhancement
Iteration: --- → 66.1 - Dec 10-23
Priority: -- → P1
Iteration: 66.1 - Dec 10-23 → 66.2 - Dec 24 - Jan 6
Iteration: 66.2 - Dec 24 - Jan 6 → 66.3 - Jan 7 - 20
Blocks: 1520691
Keywords: github-merged
Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/62c87514b36d5fb46913224ebc64f7257a2a28cb
Bug 1514006 - Part 3, add impression and click pings for CardGrid and List (#4671)
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 66
Blocks: 1521093

Hi,

Can you please provide QA steps.

Thanks.

Flags: needinfo?(najiang)
Flags: needinfo?(najiang)

QA Results :

Tested on :

FF Nightly version : 66.0a1 (2019-01-22)
OS : Mac and Windows 10 Pro

Note :
I did few rounds of testing with the default new tab version for the pings which has 1 row for Recommended by Pocket.

So, as part of testing I switched Recommended by Pocket rows from 1 to 2 and choose 5th card (on a 0-based index).

Below are the observations from Mac OS:

Impression ping :

Open New Tab, this has 6 cards in Recommended by Pocket sectio

TELEMETRY PING: {"locale":"en-US","topic":"activity-stream","client_id":"n/a","version":"66.0a1","release_channel":"nightly","addon_version":"20190122215349","user_prefs":31,"session_id":"n/a","page":"about:newtab","source":"TOP_STORIES","tiles":[{"id":32480},{"id":32414},{"id":32504},{"id":32469},{"id":32494},{"id":32516}],"action":"activity_stream_impression_stats","impression_id":"{0ac09703-cf38-be49-a470-c8c72565c022}","profile_creation_date":17833,"region":"US"}

JSON response :

{
"locale":"en-US",
"topic":"activity-stream",
"client_id":"n/a",
"version":"66.0a1",
"release_channel":"nightly",
"addon_version":"20190122215349",
"user_prefs":31,
"session_id":"n/a",
"page":"about:newtab",
"source":"TOP_STORIES",
"tiles":[
{
"id":32480
},
{
"id":32414
},
{
"id":32504
},
{
"id":32469
},
{
"id":32494
},
{
"id":32516
}
],
"action":"activity_stream_impression_stats",
"impression_id":"{0ac09703-cf38-be49-a470-c8c72565c022}",
"profile_creation_date":17833,
"region":"US"
}

Click ping :

Clicked on the 5th card (0-based index).

TELEMETRY PING: {"locale":"en-US","topic":"activity-stream","client_id":"n/a","version":"66.0a1","release_channel":"nightly","addon_version":"20190122215349","user_prefs":31,"session_id":"n/a","page":"about:newtab","source":"TOP_STORIES","click":0,"tiles":[{"id":32494,"pos":4}],"action":"activity_stream_impression_stats","impression_id":"{0ac09703-cf38-be49-a470-c8c72565c022}","profile_creation_date":17833,"region":"US"}

JSON Response :

{
"locale":"en-US",
"topic":"activity-stream",
"client_id":"n/a",
"version":"66.0a1",
"release_channel":"nightly",
"addon_version":"20190122215349",
"user_prefs":31,
"session_id":"n/a",
"page":"about:newtab",
"source":"TOP_STORIES",
"click":0,
"tiles":[
{
"id":32494,
"pos":4
}
],
"action":"activity_stream_impression_stats",
"impression_id":"{0ac09703-cf38-be49-a470-c8c72565c022}",
"profile_creation_date":17833,
"region":"US"
}

Quick question :

I do see "source":"TOP_STORIES" when I click from Recommended by Pocket - is this fine?
or do we expect "HERO", "CARDGRID", "LISTS" ?

Other than this everything is fine.

Flags: needinfo?(najiang)

Synced with Najiang offline.

Per Najiang : All the existing Pocket telemetry metrics in release have source: TOP_STORIES, i believe this also applies to the control group of the MVP
for other MVP experiment variants, they should report source as one of HERO, CARDGRID, or LIST.

QA Note :
I also have the pref setting for browser.newtabpage.activity-stream.discoverystream.config = {"enabled":false,"layout_endpoint":"https://getpocket.com/v3/newtab/layout?version=1&consumer_key=40249-e88c401e1b1f2242d9e441c4&layout_variant=control"}

So this is working as expected.

Closing the bug as verified.

Status: RESOLVED → VERIFIED
Flags: needinfo?(najiang)
Component: Activity Streams: Newtab → New Tab Page
You need to log in before you can comment on or make changes to this bug.