1397298 - Add search count metrics for experiment viewer to main_summary

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, enhancement, P2)

Description

[Ryan Harter [:harter]]

We need to add a derived column for whatever search volume metric we want to review in the experiments viewer. 

The first step will be defining what metrics we want to look at. Dave - what do you think? My initial thoughts are to create two new metrics: sap_searches and attributed_searches. sap_searches will be all chrome-sap searches. attributed_searches will be the sum of in-content-sap and follow-on searches.

See the search handbook [0] for a definition of these terms.

Once we have metrics defined, I will start a PR to get these into main_summary.

[0] https://github.com/harterrt/search-adhoc-analysis/tree/master/docs

Comment 1

[Dave Zeber [:dzeber]]

Sorry for the delay! Those sound like good general summaries. We may want to others in the future as we get a better understanding of the range of questions that will be answered about search using the experiment viewer.

Based on these aggregates, I recommend making comparisons in terms of total searches per profile and total searches per active hour (over some time period, eg. daily) in the experiment viewer.

Comment 2

[Ryan Harter [:harter]]

Let's start with just one search metric: SAP Searches. This metric can be calculated by taking the sum of the SEARCH_COUNTS histogram for all keys in this whitelist [1]. 

Eventually, we'll want to be able to slice this metric by engine, country, and search access point. For now, let's just get a safety metric implemented.


[1] https://github.com/mozilla/python_mozetl/blob/master/mozetl/constants.py#L5

Comment 3

[Ryan Harter [:harter]]

After discussing this more, I think this metric will probably end up in the experiment artifact generator before it is shown in the experiment viewer. Let's close this bug for now and track this in Bug 1426163.