Closed Bug 1576277 Opened 5 years ago Closed 4 years ago

[Experiment] Add-On Experiment: Activity Stream Search TopSites Fx 69.0 Release

Tracking

(firefox69+ fixed)

Status:

RESOLVED FIXED

Tracking Flags:

Tracking

Status

firefox69

fixed

People

(Reporter: experimenter, Assigned: mconnor)

References

(
URL
)

Details

User Story

Experiment Type: Add-on experiment

    What are the branches of the experiment:

- Treatment treatment 50%:

primary branch displaying Top Sites before the user starts typing
        
- Treatment control 50%:

Standard address bar experience
        

    What version and channel do you intend to ship to?

0.9% of Release Firefox 69.0

    Are there specific criteria for participants?


Prefs: Exclude users with the following prefs:

browser.newtabpage.activity-stream.feeds.topsites = false
browser.privatebrowsing.autostart = true

Experiments:

Any additional filters:

Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Sep 16, 2019 - Nov 15, 2019 (60 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

Primarily we're looking to see if this increases usage of top sites across the experiment  users (combining Top Sites usage in New Tab + Top Sites usage through the awesomebar).  We're also looking to see if this drives increased usage of search shortcuts, and if it converts to searches more effectively.

Business impacts we're looking to understand include impacts to search (volume, ad clicks, organic search, and Pocket.

In the analysis, we are planning to look at: 
- search volumes between the two branches
- click volume on activity streams between the two branches

We have introduced new event telemetry collection and will be looking for the following events:
urlbar#engagement#click
urlbar#engagement#enter
urlbar#engagement#drop_go
urlbar#engagement#paste_go
urlbar#abandonment#blur

EVENT VALUE:
The value for the event is the type of initial interaction: "typed", "pasted", "dropped", "topsites"

EXTRA DATA:
- elapsed time
- number of characters
- only for engagement: 
  - selection index
  - selection type

    Who is the owner of the data analysis for this experiment?

Teon Brooks

    Will this experiment require uplift?

False

    QA Status of your code:

https://jira.mozilla.com/browse/PI-240

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/search-topsites/

Experimenter Bot

Reporter

Description

•

5 years ago

Search TopSites

To test integration of the Top Sites list with QuantumBar, with the list displayed on focus, before the user starts typing. The goals of this experiment are to see if the user value from Top Sites carries over to other contexts, and whether a pre-search experience will drive incremental engagement with Firefox.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

More information: https://experimenter.services.mozilla.com/experiments/search-topsites/

Mike Connor [:mconnor]

Assignee