[Experiment] Pref-Flip Experiment: Credit Card Autofill A/B Test Fx 80.0 to 81.0 Release
Categories
(Shield :: Shield Study, task, P3)
Tracking
(firefox80+ fixed)
People
(Reporter: experimenter, Assigned: jimm)
References
()
Details
User Story
Delivery Type: Pref Flip Experiment
What is the preference we will be changing
extensions.formautofill.creditCards.enabled
What are the branches of the experiment and what values should
each branch be set to?
- Control Control 50%:
Value: None
CC autofill pref'd off
- Treatment Treatment 50%:
Value: None
CC autofill pref'd on
What version and channel do you intend to ship to?
5% of Release Firefox 80.0 to 81.0
Are there specific criteria for participants?
Prefs:
only enroll users with extensions.formautofill.creditCards.available = false
Shell: Need to make pref targeting sticky - as that is changed in branches.
Experiments:
exclude users enrolled in "Pref-Flip Experiment: Credit Card Autofill A/B Test Beta"
Countries: United States of America (US)
Locales: English (US) (en-US)
What is your intended go live date and how long will the experiment run?
Aug 25, 2020 - Sep 22, 2020 (28 days)
What is the main effect you are looking for and what data will you use to
make these decisions?
What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success
See possible outcomes in this section of this doc:
https://docs.google.com/document/d/1C_tANBDcQC_OMaGdhHDrMCqZRtDjPVfDxKJHfa2kCA0/edit#heading=h.9060bhu4dl8l
Do you plan on surveying users at the end of the delivery? NO
We are developing a survey, but it likely won't be deployed until version 81. Survey development being done in this doc
https://docs.google.com/document/d/1dBbPpw8G9wu-fOD5N6keEj7n2XHe_TPoKUzh-CbpMfU/edit?ts=5f17238e#heading=h.cw3wckhparab
Who is the owner of the data analysis for this experiment?
loines@mozilla.com
Will this experiment require uplift?
False
QA Status of your code:
Bug filed here:
https://jira.mozilla.com/browse/PI-751
Link to more information about this experiment:
https://experimenter.services.mozilla.com/experiments/credit-card-autofill-ab-test/
Credit Card Autofill A/B Test
Experiment testing the usability of a new Credit Card autofill feature.
Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.
Data Science Issue: https://jira.mozilla.com/browse/DS-737
More information: https://experimenter.services.mozilla.com/experiments/credit-card-autofill-ab-test/
|
||
Comment 1•4 years ago
|
||
[Tracking Requested - why for this release]: experimenting on this in 80 before releasing.
|
||
Updated•4 years ago
|
|
Assignee | |
Updated•4 years ago
|
|
||
Comment 2•4 years ago
|
||
Pref-Flip Experiment: Credit Card Autofill A/B Test Firefox 80
Firefox Release 80 to Firefox Release 81
We have finished testing the “[PI-772] Pref-Flip Experiment: Credit Card Autofill A/B Test” experiment.
Please take in consideration that for this request only the experiment enrollment/unenrollment and the feature telemetry were in scope.
We weren’t able to test the experiment on Nightly 81 due to the fact that the pref is set to “true” by default on the Nightly channel. However, we have verified the telemetry pings on Nightly 81 (see meta Bug 1653073).
Quality status: GREEN - SHIP IT
Why is this experiment green?
- We haven’t found any issues during testing.
Testing Summary:
Tested Platforms:
- Windows 10 x64;
- macOS 10.14.6;
- Linux Debian 9;
Tested Firefox versions:
- Firefox RC 80 en-US (Build ID: 20200817172600)
- Nightly 80 to Nightly 81 (for the update scenarios)
- Firefox Beta 80.0b8 to RC 80 (for update scenarios)
|
Reporter | |
Updated•4 years ago
|
|
|
Reporter | |
Updated•4 years ago
|
|
|
Reporter | |
Comment 3•4 years ago
|
||
Start Date: 2020-08-25 End Date: 2020-09-22
|
|
Reporter | |
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
Description
•