1523780 - [SHIELD] Pref-Flip Study: Mid-65 Default-On Cookie Restrictions for New Users, v65, release

Status: RESOLVED FIXED

(Shield :: Shield Study, defect)

Description

[:shell escalante]

for half of all new users for 4 weeks. The ad-click probe and churn will be monitored during the rollout to inform a 2019/3/11 decision whether to scale up to 100% of users.

More information: https://experimenter.services.mozilla.com/experiments/mid-65-default-on-cookie-restrictions-for-new-users/

Comment 1

[Tony Cinotto [:tcinotto] (UTC-8)]

Hi Robert, I just wanted to see if you think this experiment needed Data Steward sign off?

Comment 2

[Robert (rrayborn, he/him)]

@Tony - are we introducing new Telemetry with this project? My limited understanding is that the Telemetry probes were using already exist and we're using them as outcome measures. If this is correct data review is not required. If we are adding/modifying probes you'll need to attach a data review request as an attachment and request a review: https://github.com/mozilla/data-review/raw/master/request.md

Comment 3

[Tony Cinotto [:tcinotto] (UTC-8)]

(In reply to Rob from comment #2)

@Tony - are we introducing new Telemetry with this project? My limited understanding is that the Telemetry probes were using already exist and we're using them as outcome measures. If this is correct data review is not required. If we are adding/modifying probes you'll need to attach a data review request as an attachment and request a review: https://github.com/mozilla/data-review/raw/master/request.md

Hi Rob, I think you're correct that the probes we're using already exist. Felix can you just kindly confirm?

Comment 4

[Felix Lawrence]

I can confirm that. Yes this is a simple pref-flip study; no probes are being added in association with this study, and the pref being flipped doesn't lead to a change in what data gets sent to us.

Comment 5

[Arthur Edelstein [:arthur]]

[Tracking Requested - why for this release]: We are pushing out this pref-flip study on 2019/2/11 (mid-65).

Comment 6

[Carmen Fat [:cfat] - Ecosystem QA]

Pref-Flip Study: Mid-65 Default-On Cookie Restrictions for New Users
Pref-Flip Study: Mid-65 Default-On Cookie Restrictions for Existing Users
Targeted: Firefox Release 65.x

We have finished testing the "Pref-Flip Study: Mid-65 Default-On Cookie Restrictions for New Users & Existing Users" experiment.

QA’s recommendation: GREEN - SHIP IT

Reasoning:

• We haven’t found any issues during testing.

Testing Summary:

• Full Functional test suite: TestRail (https://tinyurl.com/y9tyv5yl)
• Verified that the "browser.search.ad_clicks" probe registers click on SERP ads.

Tested Platforms:

• Windows 10 x64
• Ubuntu 16.04 x64
• Mac 10.14.2

Tested Firefox versions:

• Firefox Release 65 (en-US, de, fr)

Comment 7

[Josh Gaunt [:jgaunt]]

Recipe 683 has been disabled. This experiment has concluded.