1483297 - Add search shortcut count to session stats

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement, P1)

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4340

Comment 2

[Nan Jiang [:nanj]]

1) What questions will you answer with this data?
How many search shortcuts in the Top Sites for each newtab session.

2) Why does Mozilla need to answer these questions?  Are there benefits for users? Do we need this information to address product or business requirements?
To measure the click through rate and block rate of the search shortcut feature in Activity Stream. Which allows us to understand that whether or not this feature improves the user experience of Top Sites in Activity Stream.

3) What alternative methods did you consider to answer these questions? Why were they not sufficient?
There is no current instrumentation of this code.

4) Can current instrumentation answer these questions?
The current instrument only measures the total clicks on search shortcuts, we'd need to measure how many Activity Stream sessions with search shortcuts to compute the overall click through rate and block rate.   

5) List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox [data collection categories](https://wiki.mozilla.org/Firefox/Data_Collection) on the Mozilla wiki.
* (Category 2) A new field `topsites_search_shortcuts` in the Activity Stream session ping, tracked in bug 1483297

6) How long will this data be collected?  Choose one of the following:
I (:nanj) want to permanently monitor this data.

7) What populations will you measure?
All the channels for all the countries/locales.

8) If this data collection is default on, what is the opt-out mechanism for users?
The user can opt-out the data collection by either disabling the telemetry of Activity Stream or disabling the Firefox telemetry as a whole.

9) Please provide a general description of how you will analyze this data.
General reporting on Redash.

10) Where do you intend to share the results of your analysis?
This will be shared over Mozilla LDAP via Redash.

Comment 3

[François Marier [:francois]]

datareview+

(In reply to Nan Jiang [:nanj] from comment #2)

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in https://github.com/mozilla/activity-stream/blob/master/docs/v2-system-addon/data_dictionary.md and https://github.com/mozilla/activity-stream/blob/master/docs/v2-system-addon/data_events.md.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, either the usual telemetry setting, or the browser.newtabpage.activity-stream.telemetry pref to disable just the Activity Stream telemetry.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, :nanj.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1 because that count is based on geo/locale.

5) Is the data collection request for default-on or default-off?

Default ON.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.

Comment 4

[[github robot]]

Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/bff4635b97452a931361a095c50a83924ce5a7b3
Fix Bug 1483297 - Add search shortcut count to session stats (#4340)

Comment 5

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/13cca9e3e12d

Comment 6

[Ed Lee :Mardak]

https://hg.mozilla.org/releases/mozilla-beta/rev/9e752b54a20f425449e1f00bf3adefa903f8051a

Comment 7

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

I have verified that the issue is no longer reproducible in the latest Nightly (63.0a1, Build ID 20180816100035) on Windows 10, Mac 10.13 and Arch Linux x64.

Comment 8

[Cristian Fogel [:cfogel]]

Verified on Win 10x64, Ubuntu 18.04LTS, macOS 10.12 for 62.0b18. The counter is present as well on beta in the stats.