1532374 - [Shield] Pref-Flip Study: Search in Private Browsing New Tab impact study, Fx66, Release

Status: RESOLVED FIXED

(Shield :: Shield Study, enhancement)

Description

[Experimenter Bot]

Search in Private Browsing New Tab impact study

In Firefox 66 we have the ability to show a new version of about:privatebrowsing that adds a search box, in order to better capture private browsing searches

More information: https://experimenter.services.mozilla.com/experiments/search-in-private-browsing-new-tab-impact-study/

Comment 1

[Bridget Kaluzny]

In Firefox 66 we have the ability to show a new version of about:privatebrowsing that adds a search box, in order to better capture private browsing searches

Comment 2

[Marius Coman [:mcoman], Ecosystem QA]

Search in Private Browsing New Tab impact study

Firefox Release 66

We have finished testing the Search in Private Browsing New Tab impact study experiment.

QA’s recommendation: GREEN - SHIP IT

Reasoning:

• No new issues have been found during testing.

• A single known issue has been encountered during testing, Bug 1530706, which is an intermittent issue only on Mac using a Japanese keyboard. We do not consider this issue damaging enough to block the experiment, but we would recommend fixing it as soon as possible.

Testing Summary:

• TestRail

Tested Platforms:

• Windows 10

• Mac 10.14

• Arch Linux 4.14

Tested Firefox versions:

• Firefox 66RC1

Comment 3

[Experimenter Bot]

    Experiment Type: Pref Flip Study

    What is the preference we will be changing

browser.privatebrowsing.searchUI

    What are the branches of the study and what values should
    each branch be set to?

- Treatment Variant (old UI) 50%:

Value: false

The variant group will see the old about:privatebrowsing UI without search (and a lot more text)
        
- Treatment Control 50%:

Value: true

The control group will see the new about:privatebrowsing search functionality launching in Firefox 66
        

    What version and channel do you intend to ship to?

0.9% of Release Firefox 66.0

    Are there specific criteria for participants?

No filtering required

    What is your intended go live date and how long will the study run?

Mar 19, 2019 - Apr 30, 2019 (42 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success

We want to see no negative effects on the following metrics:

SAP Search volume 
Organic search volume
Ad clicks
Ad displays

Do you plan on surveying users at the end of the study? No.
Strategy and Insights can help create surveys if needed

    Who is the owner of the data analysis for this study?

Ben Miroglio

    Will this experiment require uplift?

False

    QA Status of your code:



    Link to more information about this study:

https://experimenter.services.mozilla.com/experiments/search-in-private-browsing-new-tab-impact-study/

Comment 4

[Liz Henry (:lizzard) (relman/hg->git project)]

Untracking since we have this information easily findable now in Experimenter.

Comment 5

[:shell escalante]

experiment was ended on April 29th.