Closed Bug 1595604 Opened 5 years ago Closed 4 years ago

[Experiment] Add-On Experiment: Fingerprinting Implementation Retention Study V1 Fx 69.0 to 71.0 Release

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(firefox71+ fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox71 + fixed

People

(Reporter: experimenter, Unassigned)

References

(
URL
)

Details

User Story

Delivery Type: Add-on experiment

    What are the branches of the experiment:

- Treatment treatmentScript 33%:

Fingerprinting is blocked by blocking list of fingerprinting scripts.
        
- Treatment treatmentDomain 33%:

Fingerprinting is blocked by pref flip for ETP domain blocking (replicate treatment in v2 study).
        
- Control control 34%:

Fingerprinting is not blocked
        

    What version and channel do you intend to ship to?

2% of Release Firefox 69.0 to 71.0

    Are there specific criteria for participants?


* Exclude people in https://experimenter.services.mozilla.com/experiments/fingerprinting-protections-retention-study-v2/
* Pref targeting `browser.contentblocking.category === "standard"` - do not unenroll people if that changes - addon will handle that appropriately in different branches.

Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Dec 09, 2019 - Jan 06, 2020 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

As this study is an explicit comparison to previous work, the analysis will be very similar study to the previous fingerprinting studies [1][2] - the main difference being that we will have an additional branch with an alternate blocking implementation.

Check for churn, reduced usage over time, or a reduction in the number of search ad clicks over time.

Specifically, for various values of `n`, analyze per-user data for between `n` and `n+1` weeks after the user enrolled, and estimate the relative uplift (with uncertainties) in the % of users with >x active hours, uri count, search count, or search ad click count, for enough values of x to cover the vast majority of users.

The data will be taken from the following probes, none of which are custom to this experiment:
browser.engagement.total_uri_count
browser.engagement.active_ticks

[1] https://experimenter.services.mozilla.com/experiments/fingerprinting-protections-retention-study/
[2] https://experimenter.services.mozilla.com/experiments/fingerprinting-protections-retention-study-v2/

    Who is the owner of the data analysis for this experiment?

flawrence@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

https://experimenter.services.mozilla.com/experiments/fingerprinting-implementation-retention-study-v1/#comment555

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/fingerprinting-implementation-retention-study-v1/

Attachments

(6 files, 31 obsolete files)

fpblock-control@mozilla.org-0.6.0.xpi
2.74 KB, application/x-xpinstall
Details
fpblock-treatmentDomain@mozilla.org-0.6.0.xpi
3.76 KB, application/x-xpinstall
Details
fpblock-treatmentScript@mozilla.org-0.6.0.xpi
215.69 KB, application/x-xpinstall
Details
FPBlock-control@mozilla.org-0.6.0-signed.xpi
6.87 KB, application/x-xpinstall
Details
FPBlock-treatmentDomain@mozilla.org-0.6.0-signed.xpi
7.95 KB, application/x-xpinstall
Details
FPBlock-treatmentScript@mozilla.org-0.6.0-signed.xpi
221.91 KB, application/x-xpinstall
Details

Fingerprinting Implementation Retention Study V1

We plan to study the effect of different implementations (domain - current, vs url and function) of fingerprinting blocking. Following a reduction in retention observed in Fingerprinting Protections Retention Study v1, this study will see if more fine grained blocking of fingerprinters affects retention. CUT second sentence and make diff implementations clear.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

More information: https://experimenter.services.mozilla.com/experiments/fingerprinting-implementation-retention-study-v1/

Update to description:

We plan to study the retention effect of a potential alternative implementation of fingerprinting blocking. Current ETP blocking, including fingerprinting blocking, is done at the domain level -- any domain that maintains fingerprinting code, according to the Disconnect list, will have all resources from that domain blocked. We will study retention when only blocking the specific resources from a domain that are performing fingerprinting.

Attached file fpblock-control@mozilla.org-0.1.0.xpi (obsolete) —
Flags: needinfo?(mcooper)
Flags: needinfo?(mcooper)
Attachment #9112111 - Attachment is obsolete: true
Attachment #9112126 - Attachment is obsolete: true
Flags: needinfo?(mcooper)
Attachment #9112161 - Attachment is obsolete: true
Attached file fpblock-control@mozilla.org-0.2.0.xpi (obsolete) —
Attachment #9112109 - Attachment is obsolete: true
Attachment #9112110 - Attachment is obsolete: true
Attachment #9112122 - Attachment is obsolete: true
Attachment #9112125 - Attachment is obsolete: true
Flags: needinfo?(mcooper)

Please sign with testing so most of QA can proceed.

Attachment #9112164 - Attachment is obsolete: true
Attachment #9112165 - Attachment is obsolete: true
Attachment #9112166 - Attachment is obsolete: true
Attachment #9112172 - Attachment is obsolete: true
Attachment #9112173 - Attachment is obsolete: true
Attachment #9112174 - Attachment is obsolete: true
Flags: needinfo?(mcooper)
Attached file fpblock-control@mozilla.org-0.3.0.xpi (obsolete) —

Need to tag a new version.

Flags: needinfo?(mcooper)
Attachment #9113078 - Attachment is obsolete: true
Attachment #9113079 - Attachment is obsolete: true
Attachment #9113080 - Attachment is obsolete: true
Attached file fpblock-control@mozilla.org-0.4.0.xpi (obsolete) —

:mythmon - a new batch for signing with test cert. peer review is in progress, will request production signing separately when that's done.

Flags: needinfo?(mcooper)
Flags: needinfo?(mcooper)

v0.5.0 for testing

Attachment #9113347 - Attachment is obsolete: true
Attachment #9113348 - Attachment is obsolete: true
Attachment #9113349 - Attachment is obsolete: true
Attachment #9113350 - Attachment is obsolete: true
Attachment #9113351 - Attachment is obsolete: true
Attachment #9113352 - Attachment is obsolete: true
Flags: needinfo?(mcooper)
Attached file fpblock-control@mozilla.org-0.5.0.xpi (obsolete) —

rhelmer, are you happy enough with the code you've seen so far for some kind of preliminary peer review notice so mythmon can sign and qa can test.

Flags: needinfo?(rhelmer)
Flags: needinfo?(mcooper)

(In reply to Sarah Bird from comment #32)

rhelmer, are you happy enough with the code you've seen so far for some kind of preliminary peer review notice so mythmon can sign and qa can test.

Yes, I don't see any blockers. Please proceed :)

Flags: needinfo?(rhelmer)

mythmon, can you please sign for production, so qa can continue.

Flags: needinfo?(mcooper)
Flags: needinfo?(mcooper)

Study targeting Firefox Desktop 71

Attachment #9113644 - Attachment is obsolete: true
Attachment #9113645 - Attachment is obsolete: true
Attachment #9113646 - Attachment is obsolete: true
Attachment #9113656 - Attachment is obsolete: true
Attachment #9113657 - Attachment is obsolete: true
Attachment #9113658 - Attachment is obsolete: true
Attachment #9113873 - Attachment is obsolete: true
Attachment #9113874 - Attachment is obsolete: true
Attachment #9113875 - Attachment is obsolete: true
Flags: needinfo?(mcooper)

Tagged: https://github.com/mozilla/fingerprinting-blocking-methods-study-addon/releases/tag/v0.6.0

Incorporates suggestions from mixedpuppy (#17, #18, #19). Incorporates bugfix noted by ublock author (#21).

:mythmon if you can sign for both testing and production so QA can proceed as soon as normandy problem is in fixed (https://bugzilla.mozilla.org/show_bug.cgi?id=1601901).

Depends on: 1601901
Flags: needinfo?(mcooper)

Add-On Experiment: Fingerprinting Implementation Retention Study
Firefox Release 69.x, 70.x, 71.x

We have finished testing the Add-On Experiment: Fingerprinting Implementation Retention Study.

QA’s recommendation: GREEN - SHIP IT

Reasoning:

  • We haven’t found any new issues during testing the latest version (0.6.0) of the experiment.

Testing Summary:

Tested Platforms:

  • Windows 10 x64;
  • Arch Linux 5.3.6;
  • macOS 10.15.1;

Tested Firefox versions:

  • Firefox Release 69.0.3 en-US;
  • Firefox Release 70.0 fr;
  • Firefox Release 71.0 de;
User Story: (updated)
Summary: [Experiment]: Add-On Experiment: Fingerprinting Implementation Retention Study V1 → [Experiment] Add-On Experiment: Fingerprinting Implementation Retention Study V1 Fx 69.0 to 71.0 Release
Start Date: 2019-12-11 End Date: 2020-01-08

The experiment is live and we have no reported issue, I am marking 71 as fixed.

status-firefox71: affected → fixed
User Story: (updated)
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: