Closed Bug 1655504 Opened 5 years ago Closed 4 years ago

[Experiment] Add-On Experiment: DNSSEC Interference Measurement Fx 82.0 to 83.0 Beta

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(firefox83+ fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox83 + fixed

People

(Reporter: experimenter, Assigned: ahounsel)

References

(
URL
)

Details

(Whiteboard: [no-nag])

User Story

Delivery Type: Add-on experiment

    What are the branches of the experiment:

- Control Observeration branch 100%:

Only need one observational branch to gather baseline success responses.
        

    What version and channel do you intend to ship to?

1% of Beta Firefox 82.0 to 83.0

    Are there specific criteria for participants?


Prefs:

Experiments:

Any additional filters: Please ignore clients that have disabled telemetry.

Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Nov 17, 2020 - Nov 24, 2020 (7 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

Concretely, there are two questions we intend to answer: 

1. How frequently do middleboxes between clients and recursive resolvers interfere with/drop DNSSEC record types? In other words, is it viable for Firefox to perform DNSSEC validation?
2. Do other record types (e.g. SMIMEA and an unallocated code point) get interfered with/dropped by middleboxes? In other words, is it viable to introduce new record types in a finite period of time?
3. Do certain populations of clients or ISPs experience higher failure rates?

At a high-level, both questions can be answered for each record type with the following analysis steps:

1. For each client, extract the DNS responses reported in each ping for a given record type
2. For a given record type, compare the bytes for the DNS response reported in each ping to the expected response across sessions
3. For a given record type, compute the percentage of DNS responses that do not match the expected response across sessions
4. For a given record type, compute the percentage of missing DNS responses across sessions

    Who is the owner of the data analysis for this experiment?

None

    Will this experiment require uplift?

False

    QA Status of your code:

https://jira.mozilla.com/browse/PI-744

Signed-off as GREEN as per comment 1125 - https://experimenter.services.mozilla.com/experiments/dsn-sec-interference-measurment/#comment1125

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/dsn-sec-interference-measurment/

DSN sec interference measurment

We want to measure the rates on interference with DNS sec requests.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DO-352
More information: https://experimenter.services.mozilla.com/experiments/dsn-sec-interference-measurment/

Summary: [Experiment]: Add-On Experiment: DSN sec interference measurment → [Experiment]: Add-On Experiment: DNSSEC Interference Measurement
See Also: → 1654059
Depends on: 1654023
Depends on: 1654059

[Tracking Requested - why for this release]: this is an experiment we are aiming to run on 78. just filing so someone sees.

tracking-firefox78: --- → ?
See Also: 1654059 →

[Tracking Requested - why for this release]: We will run the initial version of this experiment on Beta, not Release. We will probably do a second version on Release, but I suspect this bug should be updated to track for Firefox 80.

tracking-firefox78: + → ---
tracking-firefox80: --- → ?
tracking-firefox81: --- → ?
tracking-firefox81: ? → ---

DNSSEC Interference Measurement
Firefox Beta 80-81

We have finished testing the [PI-744] DNSSEC Interference Measurement experiment.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • We haven’t found any issues during testing.

Testing Summary:

  • Test Runs: Testrail
  • Tested several scenarios from the test run using a different nameserver on Windows and mac.

Tested Platforms:

  • Windows 10 x64
  • macOS 10.15

Tested Firefox versions:

  • Firefox Beta Unbranded 80.0b8, Build ID 20200813191622
  • Firefox Dev Edition 81.0b3, Build ID 20200827203325
User Story: (updated)
Summary: [Experiment]: Add-On Experiment: DNSSEC Interference Measurement → [Experiment] Add-On Experiment: DNSSEC Interference Measurement Fx 82.0 to 83.0 Beta
Start Date: 2020-11-17 End Date: 2020-11-24
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
status-firefox83: affected → fixed
User Story: (updated)
You need to log in before you can comment on or make changes to this bug.