Closed
Bug 1549583
Opened 5 years ago
Closed 5 years ago
[Shield] Pref-Flip Study: activity-stream-pktnewtabH12019-release66-confirmation-test
Categories
(Shield :: Shield Study, defect)
Shield
Shield Study
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: experimenter, Assigned: nchapman)
References
Details
activity-stream-pktnewtabH12019-release66-confirmation-test
4 branch, large scale study to get high resolution tests on overall browser metrics for New Tab layout candidates (3 candidates + control) for releasing on FF68.
More information: https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh12019-release66-confirmation-test/
Comment 2•5 years ago
|
||
Jared, what versions of Firefox are affected by this study? 68 only?
Flags: needinfo?(jkerim)
Comment 3•5 years ago
|
||
:pascalc According to the experiment definition it says: Please target users on FF66 and above (so FF67, FF68, etc.).
Flags: needinfo?(jkerim)
Reporter | ||
Comment 4•5 years ago
|
||
Experiment Type: Pref Flip Study What is the preference we will be changing browser.newtabpage.activity-stream.discoverystream.config What are the branches of the study and what values should each branch be set to? - Treatment 3-col-1-row-octr-4-col-6-row-categories 25%: Value: "{\"api_key_pref\":\"extensions.pocket.oAuthConsumerKey\",\"collapsible\":true,\"enabled\":true,\"show_spocs\":true,\"hardcoded_layout\":false,\"personalized\":false,\"layout_endpoint\":\"https://getpocket.cdn.mozilla.net/v3/newtab/layout?version=1&consumer_key=$apiKey&layout_variant=3-col-1-row-octr-4-col-6-row-categories\"}" 3 column first row using the OCTR feed, followed by 6 rows of 4-column category feeds. - Treatment 3-col-7-row-octr 25%: Value: "{\"api_key_pref\":\"extensions.pocket.oAuthConsumerKey\",\"collapsible\":true,\"enabled\":true,\"show_spocs\":true,\"hardcoded_layout\":false,\"personalized\":false,\"layout_endpoint\":\"https://getpocket.cdn.mozilla.net/v3/newtab/layout?version=1&consumer_key=$apiKey&layout_variant=3-col-7-row-octr\"}" 3x7 grid of cards using the OCTR feed and followed by Popular Topic links. - Treatment basic-3-col-octr 25%: Value: "{\"api_key_pref\":\"extensions.pocket.oAuthConsumerKey\",\"collapsible\":true,\"enabled\":true,\"show_spocs\":true,\"hardcoded_layout\":false,\"personalized\":false,\"layout_endpoint\":\"https://getpocket.cdn.mozilla.net/v3/newtab/layout?version=1&consumer_key=$apiKey&layout_variant=basic-3-col-octr\"}" 3 stories from the OCTR feed with Popular Topic links below. - Treatment control 25%: Value: "{\"api_key_pref\":\"extensions.pocket.oAuthConsumerKey\",\"enabled\":false,\"show_spocs\":true,\"layout_endpoint\":\"https://getpocket.cdn.mozilla.net/v3/newtab/layout?version=1&consumer_key=$apiKey&layout_variant=basic\"}" This is the default experience without Discovery Stream enabled. What version and channel do you intend to ship to? 5% of Release Firefox 66.0 Are there specific criteria for participants? Prefs: browser.newtabpage.enable OR (we want AT LEAST one of these to be default) browser.startup.homepage AND (we want ALL of the below to be default) browser.newtabpage.activity-stream.showSearch AND browser.newtabpage.activity-stream.feeds.topsites AND browser.newtabpage.activity-stream.feeds.section.topstories AND browser.newtabpage.activity-stream.feeds.section.highlights Studies: Please exclude all subjects that were enrolled (at any point) in the following experiments: * https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh12019-release66layoutstest1/ * https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh12019-release66smoketest2/ Any additional filters: Please target users on FF66 and above (so FF67, FF68, etc.). Please exclude users on Dark Theme: ['environment']['addons']['activeAddons']['theme']['id'] = 'firefox-compact-dark@mozilla.org' What is your intended go live date and how long will the study run? May 14, 2019 - Aug 06, 2019 (84 days) What is the main effect you are looking for and what data will you use to make these decisions? What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success The success metrics are defined here: https://docs.google.com/document/d/1iwrKp3JT1WhULIlnv0r5ZahTrSdTSQ0RY2BrZv9_tss/edit?usp=sharing Do you plan on surveying users at the end of the study? Yes/No. Strategy and Insights can help create surveys if needed Yes. Who is the owner of the data analysis for this study? shong@mozilla.com Will this experiment require uplift? False QA Status of your code: This experiment will be launching at the end of FF66, and will continue through the release of FF67. Please test over both versions. QA guidelines here: https://docs.google.com/document/d/1iySmNA2tBl3ApLuJtP7UBlPwNx9sH1SmK-jG6exvMQ0/edit#heading=h.s177b01wzbij Link to more information about this study: https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh12019-release66-confirmation-test/
Reporter | ||
Updated•5 years ago
|
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•