1029776 - [Search experiment] Create experiment add-on and define user group (cell) on installation

Status: RESOLVED FIXED

(Firefox :: Search, defect)

Description

[:Felipe Gomes (needinfo for replies!)]

Create the basic structure for the experiment add-on, and on installation, define the user group (using the experiment branch mechanism from bug 1017806) as follows:

- Users who don't have Google as the default search provider: put it on group 6
- Users who have customized away the search box in the URL bar: put it on group 6

For all other users, make a random draw to produce an equal sampling between 5 groups, and put them in the chosen group. The meaning of each group is explained in bug 1029263.

If the experiment has been disabled and reenabled, the defined group for that user shouldn't change.

Comment 1

[:Felipe Gomes (needinfo for replies!)]

Additional requirement: Every member of every cell must have used firefox at least once a week for the past four weeks

Comment 2

[:Felipe Gomes (needinfo for replies!)]

(In reply to :Felipe Gomes from comment #1)
> Additional requirement: Every member of every cell must have used firefox at
> least once a week for the past four weeks

This function can be used as the experiment jsfilter function to determine if firefox was used at least once per week:

>function filter(a, healthReportPayload, c) {
>  if (healthReportPayload.version < 2 ||
>      !healthReportPayload.data.days) {
>    return false;
>  }
>
>  const ONE_DAY = 60 * 60 * 24 * 1000;
>  let today = new Date();
>
>  function daysAgo(n) {
>    // toISOString().substr(0, 10) is the same implementation
>    // that FHR uses
>    return (new Date(today - n * ONEDAY)).toISOString().substr(0, 10) 
>  }
>
>  function week(start, end) {
>    for (let i = start; i <= end; i++) {
>      yield daysAgo(i);
>    }
>  }
>
>  let days = healthReportPayload.data.days;
>  return
>    [day for each (day in week(1, 7))].some(day => day in days) &&
>    [day for each (day in week(8, 14))].some(day => day in days) &&
>    [day for each (day in week(15, 21))].some(day => day in days) &&
>    [day for each (day in week(22, 28))].some(day => day in days)
>}

It should be easy to convert it to run from inside the experiment if we want to tag these users in a specific cell. But it's slightly convenient to run it from the jsfilter because the FHR payload is already passed as a parameter (and also it can be reevaluated every day to see if the user becomes a candidate)

Comment 3

[:Felipe Gomes (needinfo for replies!)]

Start of work here:
https://hg.mozilla.org/users/felipc_gmail.com/search-experiment

Comment 4

[Marco Mucci [:MarcoM]]

Added to Iteration 33.2  Can you provide a point value and if the bug should be marked as [qa+] or [qa-] for verification.

Comment 5

[:Felipe Gomes (needinfo for replies!)]

I'm setting qa- because there's an overall bug about testing the add-on, bug 1029818, and it doesn't make much sense to test piecewise.

Comment 6

[:Felipe Gomes (needinfo for replies!)]

Untested, but should be close to what's necessary:
http://hg.mozilla.org/users/felipc_gmail.com/search-experiment/rev/7fb5fc0ac3e8

Comment 7

[Chad Weiner [:chadw]]

Amendment to Acceptance criteria:

*Each of the five cells should contain 10% of the EN-US Beta population.

Comment 8

[:Felipe Gomes (needinfo for replies!)]

Work done at https://hg.mozilla.org/users/felipc_gmail.com/search-experiment and to be reviewed through
bug 1029818