1500556 - Experiment Analysis for ROW Activity Stream Shortcuts

Status: RESOLVED FIXED

(Data Science :: Experiment Collaboration, task, P1)

Description

[Ryan Harter [:harter]]

We launched a second phase of experimentation for Activity Stream Shortcuts in Bug 1495565. This bug tracks the experiment analysis.

Comment 1

[Ryan Harter [:harter]]

I have an analysis framework put together here [0]. There are no results in the report yet, but I'd like to start getting feedback on the analysis.


[0] https://metrics.mozilla.com/~rharter/reports/experimentation/iterative_bootstrap/index.html

Comment 2

[Ryan Harter [:harter]]

I have some initial analysis here [0]. Keep in mind that these numbers will change as I get more data. This analysis still needs review.

[0] https://metrics.mozilla.com/~rharter/reports/as_shortcuts_row/index.html#Results

Comment 3

[Ryan Harter [:harter]]

The report is under review here: https://github.com/mozilla-services/mozilla-private-reports/pull/21

Comment 4

[Ryan Harter [:harter]]

Reviewed and submitted. You can find the report here [0]. We're going to let this run for another week to gather more data.

[0] https://reports-dev.telemetry.mozilla.org/as_shortcuts_row/index.html#Results

Comment 5

[Ryan Harter [:harter]]

Ben updated this analysis two weeks ago. I'm going to update the analysis with data through yesterday and add the remaining countries. Expect updated results this week.

Comment 6

[Ryan Harter [:harter]]

Initial results at [1]. The update is under review [2].

[1] https://metrics.mozilla.com/~rharter/reports/as_shortcuts_row/index.html
[2] https://github.com/mozilla-services/mozilla-private-reports/pull/24

Comment 7

[Ryan Harter [:harter]]

Initial report is done. We have a request to add organic search to the report, which will be tracked in bug 1511082. Closing this bug.