Closed Bug 1654519 Opened 4 years ago Closed 4 years ago

[Experiment] Pref-Flip Experiment: Geocoding locally Fx 80.0 to 81.0 Release

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: daleharvey)

References

(
URL
)

Details

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

browser.region.local-geocoding

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Local Geocoding Enabled 50%:

Value: true

Local Geocoding Enabled
        
- Control Control 50%:

Value: false

Control branch
        

    What version and channel do you intend to ship to?

0.9% of Release Firefox 80.0 to 81.0

    Are there specific criteria for participants?

Prefs:

Experiments:

Any additional filters:
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Sep 01, 2020 - Oct 01, 2020 (30 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success

We will replicate some of our search reliability dashboards @ https://sql.telemetry.mozilla.org/dashboard/search-reliability-measurements, for each branch in the experiment. Specifically, the Revenue Efficiency query: https://sql.telemetry.mozilla.org/queries/61856/source#159124.

    Who is the owner of the data analysis for this experiment?

bmiroglio@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

Finished testing the experiment and sent a GREEN sign off: https://experimenter.services.mozilla.com/experiments/local-geocoding/#comment1106.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/local-geocoding/

Geocoding locally

We are changing where the Geo-locale is looked up at MLS, from the server to now locally (without a request to the server) . We want to ensure effectiveness and efficiency.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-896
More information: https://experimenter.services.mozilla.com/experiments/local-geocoding/

Experiment to test Local Geo coding
Firefox Release 80 to Firefox Release 81

We have finished testing the “[PI-726] Experiment to test Local Geo coding” experiment.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • We haven’t found any issues during testing.

Testing Summary:

Tested Platforms:

  • Windows 10 x64 (two runs);
  • macOS 10.14.6 (two runs);
  • Linux Debian 9 (one run);
  • Ubuntu 20.04 x64 (one run);

Tested Firefox versions:

  • Firefox Beta 80.0b8 DE (Build ID: 20200813191622)
  • Firefox Beta 80.0b8 FR (Build ID: 20200813191622)
  • Firefox RC 80 en-US (Build ID: 20200817172600)
  • Nightly 81.0a1 en-GB (Build ID: 20200818214031)
  • Nightly 81.0a1 en-CA (Build ID: 20200818214031)
  • Nightly 81.0a1 en-US (Build ID: 20200818214031)
  • Nightly 80 to Nightly 81 (for the update scenarios)
  • Firefox Beta 80.0b8 to RC 80 (for the update scenarios)
  • Firefox Beta 80.0b7 to Firefox Beta 80.0b8 (for the update scenarios)
User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Geocoding locally → [Experiment] Pref-Flip Experiment: Geocoding locally Fx 80.0 to 81.0 Release
Start Date: 2020-09-01 End Date: 2020-10-01
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.