972936 - Overall impressions metrics for New Tab

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect)

Description

[Marco Mucci [:MarcoM]]

1.  Goal
* As a product owner I need to know the overall impressions for the directory tiles so that we can compare this with unique impressions to help accurately price tiles.

2.  Acceptance Criteria (AC)
* General impressions per region when directory tiles are visible is sent back to Mozilla through a channel such as FHR

3.  Notes/Supporting Documentation
* Impressions are purely the number of page loads, not uniquely tied to a user but globally how many times was this page seen.

Comment 1

[Bryan Clark (DevTools PM) [:clarkbw]]

Note that bug 971171 is implementing a view count for about:newtab that will be pushed through telemetry.  Not sure if it's possible to reuse any of that.

Comment 2

[John Jensen]

So this is total New Tab page views, correct? Not total views of individual tiles?

Comment 3

[Bryan Clark (DevTools PM) [:clarkbw]]

(In reply to John Jensen from comment #2)
> So this is total New Tab page views, correct? Not total views of individual
> tiles?

Indeed this is how many times the new tab page is seen at all; ignoring whether Directory Tiles are present or not.  Metrics about Directory Tiles impressions (per tile) should be coming from bug 972933 instead.  (updating title to more accurately reflect this)

Comment 4

[Marco Mucci [:MarcoM]]

Converting to a work item following discussion with Bryan.

Comment 5

[Bryan Clark (DevTools PM) [:clarkbw]]

bug 971171 gets us an an equivalent to an overall impression via telemetry

Comment 6

[Ed Lee :Mardak]

What should this bug do differently that isn't covered by bug 971171?

Comment 7

[Bryan Clark (DevTools PM) [:clarkbw]]

(In reply to Ed Lee :Mardak from comment #6)
> What should this bug do differently that isn't covered by bug 971171?

I believe that bug covers what we need for this metric for now.

Comment 8

[Ed Lee :Mardak]

It seems that we should be able to add telemetry probes similar to both bug 971171 and bug 975570. Where bug 971171 shows us when we should be measuring (new tab shown) and bug 975570 shows us how to collect a count of how many directory tiles were shown.

clarkbw, I see your comment 7, but do we want to get a histogram of how many directory tiles were shown on the newtab page? (As opposed to relying on just 971171 that counts page views?)

Comment 9

[Bryan Clark (DevTools PM) [:clarkbw]]

Yes, I think bug 972933 was going to handle that but either that or this bug.  We need how many directory tiles are shown (this?) and how many times the page was shown (bug 97117).

Comment 10

[Ed Lee :Mardak]

Attachment: v1

Comment 11

[Drew Willcoxon :adw]

Comment on attachment 8393311 [details] [diff] [review]
v1

Review of attachment 8393311 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with the change below.

::: browser/base/content/newtab/page.js
@@ +93,4 @@
>          for (let site of gGrid.sites) {
>            if (site) {
>              site.captureIfMissing();
> +            if (site.link.telemetryId != null) {

if ("telemetryID" in site.link) {

Note the capitalization change that I requested in bug 972930.  But actually, I think the intent would be made clearer by this, so please do this instead:

if (site.link.type == "sponsored") {

... unless there are multiple links types, all of which are directory tiles that this bug wants telemetry on, defined in the JSON that I'm not seeing?  IOW, if this bug is tracking sponsored tiles in specific and not directory tiles in general (assuming sponsored tiles are but one type of directory tile), then please use the type check.  Otherwise the telemetryID check is fine.

Comment 12

[Ed Lee :Mardak]

(In reply to Drew Willcoxon :adw from comment #11)
> ... unless there are multiple links types, all of which are directory tiles
Directory tiles include sponsored (paid), organic (popular), affiliate (unpaid) that we want to get telemetry on. We do want the different types as the sponsored icon should only show for type=sponsored and not for organic.

Comment 13

[Ed Lee :Mardak]

Attachment: for checkin

Comment 14

[Ed Lee :Mardak]

Attachment: for checkin

Comment 15

[Ed Lee :Mardak]

Attachment: v2

r? for changes to the type of data we want to collect. Instead of total directory count, it's supposed to be by the 3 types.

Comment 16

[Ed Lee :Mardak]

Attachment: for checkin

Comment 17

[Ed Lee :Mardak]

https://hg.mozilla.org/integration/fx-team/rev/a695139e96ee

Comment 18

[Phil Ringnalda (:philor)]

Backed out in https://hg.mozilla.org/integration/fx-team/rev/4caa44af6d1b, see bug 975228 comment 30.

Comment 19

[Ed Lee :Mardak]

https://hg.mozilla.org/integration/fx-team/rev/d1e211921983

For testing, each new tab page will trigger 3 telemetry probes:
"NEWTAB_PAGE_DIRECTORY_AFFILIATE_SHOWN"
"NEWTAB_PAGE_DIRECTORY_ORGANIC_SHOWN"
"NEWTAB_PAGE_DIRECTORY_SPONSORED_SHOWN"

with a record of how many of those types were shown:
2 affiliate (wikipedia, mozilla foundation)
6 organic (facebook, youtube, twitter, yahoo, amazon)
2 sponsored (bbc, wired)

Comment 20

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/d1e211921983

Comment 21

[Paul Silaghi, QA [:pauly]]

(In reply to Bryan Clark (Firefox Search PM) [:clarkbw] from comment #3)
> Indeed this is how many times the new tab page is seen at all; ignoring
> whether Directory Tiles are present or not.  Metrics about Directory Tiles
> impressions (per tile) should be coming from bug 972933 instead.  (updating
> title to more accurately reflect this)
So bug 972933 is WONTFIX.
What exactly should happen if I open a new tab page that has both tiles and thumbnails from other sites?
I see 2 bars in the NEWTAB_PAGE_DIRECTORY_ORGANIC_SHOWN graph, and also can't find a logic for the counted data.

Comment 22

[Georg Fritzsche [:gfritzsche]]

(In reply to Paul Silaghi, QA [:pauly] from comment #21)
> What exactly should happen if I open a new tab page that has both tiles and
> thumbnails from other sites?
> I see 2 bars in the NEWTAB_PAGE_DIRECTORY_ORGANIC_SHOWN graph, and also
> can't find a logic for the counted data.

The description is:
"Number of organic directory links shown on about:newtab. For non-default row or column configurations, extra links fall into the '10' bucket."

So, every bar stands for a count of "organic directory links" shown on that page. The count for the bar is how often it encountered that number of organic links.
If it shows/counts a 10 bar/bucket, then you have a non-default configuration for the newtab page.

Comment 23

[Paul Silaghi, QA [:pauly]]

Thanks Georg.
Verified fixed 31.0a1 (2014-04-03), Win 7, Ubuntu 12.10 and Mac OS X 10.9.
Filed bug 991729 for a related issue.