Closed Bug 1676316 Opened 11 months ago Closed 5 months ago

[Experiment] Pref-Flip Experiment: TopSites Launch Phase 3 - DE Fx 83.0 to 85.0 Release

Categories

(Shield :: Shield Study, task, P3)

Tracking

(firefox83+ fixed, firefox84+ fixed)

RESOLVED FIXED
Tracking Status
firefox83 + fixed
firefox84 + fixed

People

(Reporter: experimenter, Assigned: bkaluzny)

References

(Blocks 1 open bug, )

Details

(Whiteboard: [no-nag])

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

browser.topsites.useRemoteSetting

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Treatment-admarketplace 50%:

Value: None

Changes Amazon tile to a URL and sends a background request to ad market place
        
- Control Control 50%:

Value: None

Don't use Remote Settings to power the Top Sites
        

    What version and channel do you intend to ship to?

40% of Release Firefox 83.0 to 85.0

    Are there specific criteria for participants?

Prefs:

Experiments:

Any additional filters:
Countries: Germany (DE) 

Locales: all

    What is your intended go live date and how long will the experiment run?

Nov 17, 2020 - Feb 25, 2021 (100 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

Our partner (AdMarketplace) is monitoring the traffic we send their way with the ‘amazon.attributionURL’.  
They’ve contacted us right away before when they saw something strange.
Their internal contacts are Mike DeBoer,  Dao, and mconnor.

    Who is the owner of the data analysis for this experiment?

tbrooks@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

https://jira.mozilla.com/browse/PI-860; 
QA Status: YELLOW. For more information see this comment https://experimenter.services.mozilla.com/experiments/topsites-launch-phase-3-de/#comment1227.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/topsites-launch-phase-3-de/

TopSites Launch Phase 3 - DE

TopSites Redirect Launch

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-673
More information: https://experimenter.services.mozilla.com/experiments/topsites-launch-phase-3-de/

[Tracking Requested - why for this release]:

TopSites Launch Phase 3 - DE
Release Firefox 83.0 to 85.0

We have finished testing the [PI-860] TopSites Launch Phase 3 - DE experiment.

Quality status: YELLOW - SHIP IT CONDITIONALLY

Why is this experiment yellow?

What needs to be done?

Testing Summary:

  • Test runs: TestRail - https://testrail.stage.mozaws.net/index.php?/plans/view/37171.
  • Exploratory testing performed on Linux Mint 20 related to enrollment, Sponsored Top Sites being displayed, them being dismissable and that visiting Amazon and eBay can still make the non-Sponsored version be displayed in Top Sites.
  • Testing has been performed on the Stage environment. Due to the nature of the experiment, care should be taken so that the Production version has been set up with the same content as the Stage version of the top-sites Remote Settings collection.

Tested Platforms:

  • Windows 10
  • Linux Mint 20 (exploratory and bug verification)
  • macOS 10.15 (exploratory and bug verification)

Tested Firefox versions:

  • Firefox 83.0 RC 2 (Build ID: 20201112153044)
  • Firefox 83.0 RC 1 (Build ID: 20201110140026)
  • Firefox Nightly 84.0a1 (Build ID: 20201111163850) (update scenario)

Regards,
Ciprian Muresan
Ecosystem QA Team

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: TopSites Launch Phase 3 - DE → [Experiment] Pref-Flip Experiment: TopSites Launch Phase 3 - DE Fx 83.0 to 85.0 Release
Start Date: 2020-11-16 End Date: 2021-02-24
User Story: (updated)
Whiteboard: [no-nag]
  • The YELLOW sign off for Fx 84 (PI-886) can be found here.
  • The GREEN sign off for Fx 85 (PI-905) can be found here.
  • The GREEN sign off for Fx 86 (PI-972) can be found here.
  • The YELLOW sign off for Fx 87 (PI-1005) can be found here.
  • We have finished testing the experiment for Firefox 88 and sent a GREEN sign off. You can find it here.
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED

We have finished testing the experiment on Firefox 90 (PI-1109) and sent a GREEN sign-off. You can find it here.

You need to log in before you can comment on or make changes to this bug.