Closed Bug 1668861 Opened 5 years ago Closed 5 years ago

[Experiment] Pref-Flip Experiment: Measure "set to default" adoption impact of changing the prompt to set Firefox as default (Win10) Fx 81.0 to 83.0 Release

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: RT)

References

(
URL
)

Details

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

browser.defaultbrowser.notificationbar

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Infobar default browser notification 50%:

Value: true

New experience
        
- Control Modal default browser notification 50%:

Value: false

Current experience
        

    What version and channel do you intend to ship to?

3% of Release Firefox 81.0 to 83.0

    Are there specific criteria for participants?

Prefs: N/A

Experiments: N/A

Any additional filters: N/A
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Oct 27, 2020 - Dec 01, 2020 (35 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

1 Compare D1, D7 and D14 adoption of Firefox as a default browser between test and control
2 Breakdown 1 and 2 per  attribution status and Tier 1 country to identify potential segments where results deviate significantly from average

Update: Su
Pre-planning doc here: Set to Default UI Change Experiment Pre-Planning
https://docs.google.com/document/d/1ZxIcu3C7cQo67LXmh1n1pG1JZuqvuBLWO7fZEznmIrY/edit

    Who is the owner of the data analysis for this experiment?

shong@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

QA Status: GREEN. For further information, see: https://experimenter.services.mozilla.com/experiments/measure-new-user-retention-impact-of-changing-the-prompt-to-set-firefox-as-default/#comment1191.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/measure-new-user-retention-impact-of-changing-the-prompt-to-set-firefox-as-default/

Measure "set to default" adoption of changing the prompt to set Firefox as default

Measure "set to default" adoption of changing the prompt to set Firefox as default:

  • Adjustment to UI (infobar design replaces modal)
  • Adjustment to timing (infobar displayed upon opening new tab and about:home until user dismisses the message)

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-1046
More information: https://experimenter.services.mozilla.com/experiments/measure-new-user-retention-impact-of-changing-the-prompt-to-set-firefox-as-default/

Updated default browser prompt

Firefox Release 81, 82, 83

We have finished testing the [PI-818] Updated default browser prompt (Windows 10) experiment.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • Delivery mechanism:
    • No issues found.
  • Product specific:
    • The known issues we have encountered on the feature part do not affect user experience in a harmful way.

Testing Summary:

Tested Platforms:

  • Windows 10 x64

Tested Firefox versions:

  • Firefox Release 81.0.2 - Build ID: 20201012085804
  • Firefox Release 82.0 - Build ID: 20201014125134
  • Firefox Release 83.0b3 - Build ID: 20201022171613

Regards,

Marius Coman

Ecosystem QA Team

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Measure "set to default" adoption of changing the prompt to set Firefox as default → [Experiment] Pref-Flip Experiment: Measure "set to default" adoption impact of changing the prompt to set Firefox as default (Win10) Fx 81.0 to 83.0 Release
Start Date: 2020-10-27 End Date: 2020-12-01
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.