[Experiment] Pref-Flip Experiment: activity-stream-pocket-new-tab-popular-topics-smoke-test-rerun Fx 78.0 to 79.0 Beta
Categories
(Shield :: Shield Study, task, P3)
Tracking
(firefox79+ wontfix)
People
(Reporter: experimenter, Assigned: digitalnewslab)
References
()
Details
(Whiteboard: [no-nag])
User Story
Delivery Type: Pref Flip Experiment
What is the preference we will be changing
None
What are the branches of the experiment and what values should
each branch be set to?
- Treatment Variant 50%:
Value: None
Popular Topics above the fold
- Control Control 50%:
Value: None
Popular topics at the bottom of New Tab
What version and channel do you intend to ship to?
1% of Beta Firefox 78.0 to 79.0
Are there specific criteria for participants?
Additional Filtering
Prefs:
browser.newtabpage.enabled
OR (we want AT LEAST one of these to be default)
browser.startup.homepage
AND (we want ALL of the below to be default)
browser.newtabpage.activity-stream.showSearch
AND
browser.newtabpage.activity-stream.feeds.topsites
AND
browser.newtabpage.activity-stream.feeds.section.topstories
AND
browser.newtabpage.activity-stream.feeds.section.highlights
Experiments: None
Countries: United States of America (US)
Locales: English (US) (en-US)
What is your intended go live date and how long will the experiment run?
Jul 14, 2020 - Jul 28, 2020 (14 days)
What is the main effect you are looking for and what data will you use to
make these decisions?
The main effect we're looking for is whether Popular Topics positioned above the fold see a higher volume of clicks per user than when they are positioned below the fold. We will also be checking for any corresponding decline in engagement with stories and SPOCS.
Do you plan on surveying users at the end of the delivery? No. Strategy and Insights can help create surveys if needed
Is a Survey Required? No
Who is the owner of the data analysis for this experiment?
kirill@getpocket.com
Will this experiment require uplift?
False
QA Status of your code:
QA Status: GREEN. For more information, see: https://experimenter.services.mozilla.com/experiments/activity-stream-pocket-new-tab-popular-topics-smoke-test-rerun/#comment1006
Link to more information about this experiment:
https://experimenter.services.mozilla.com/experiments/activity-stream-pocket-new-tab-popular-topics-smoke-test-rerun/
activity-stream-pocket-new-tab-popular-topics-smoke-test-rerun
New Tab Popular Topics Smoke Test
Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.
Data Science Issue: https://jira.mozilla.com/browse/DO-352
More information: https://experimenter.services.mozilla.com/experiments/activity-stream-pocket-new-tab-popular-topics-smoke-test-rerun/
|
||
Comment 1•4 years ago
|
||
[Tracking Requested - why for this release]:
hoping to relaunch beta smoketest in Beta 79 to clear the path for this experiment to launch in Release 79.
|
||
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
|
||
Comment 2•4 years ago
|
||
New Tab Layout Study - Popular Topics - Firefox Beta 79 (7/7)
Firefox Beta 79.0bx
We have finished testing the [PI-694] New Tab Layout Study - Popular Topics - Firefox Beta 79 (7/7) experiment.
QA’s recommendation: GREEN-SHIP IT
Why is this experiment green?
- During testing, we have uncovered 2 known issues. However, the encountered issues are not considered of high impact for the user experience.
Testing Summary:
- Full Functional test suite: TestRail
Tested Platforms:
- Windows 10 x64
- macOS 10.15
- Ubuntu 18.04 x64
Tested Firefox versions:
- Firefox Beta 79.0b4 en-US - Build ID: 20200703150958
- Firefox Beta 79.0b5 en-US - Build ID: 20200708005253
|
Reporter | |
Updated•4 years ago
|
|
|
Reporter | |
Updated•4 years ago
|
|
||
Updated•4 years ago
|
Description
•