Experiment: Measure and compare performance of TRR providers
Categories
(Firefox :: Security, task, P1)
Tracking
()
People
(Reporter: nhnt11, Assigned: johannh)
References
(Blocks 2 open bugs)
Details
Attachments
(3 files, 1 obsolete file)
47 bytes,
text/x-phabricator-request
|
pascalc
:
approval-mozilla-beta+
|
Details | Review |
47 bytes,
text/x-phabricator-request
|
pascalc
:
approval-mozilla-beta+
|
Details | Review |
3.04 KB,
text/plain
|
chutten
:
data-review+
|
Details |
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Comment 1•5 years ago
|
||
Reporter | ||
Comment 2•5 years ago
|
||
Depends on D62845
Reporter | ||
Comment 3•5 years ago
|
||
The only answer I haven't filled out is the one about where the analysis will be shared. Tim can probably help with that. I'd appreciate any corrections or clarifications for the other answers as well. Thanks!
Reporter | ||
Comment 4•5 years ago
|
||
Comment 5•5 years ago
|
||
I fleshed it out a little bit. I'll leave this for chutten since I've been involved in the telemetry design.
Reporter | ||
Comment 6•5 years ago
|
||
Here are the prefs that control this module:
doh-rollout.trrRace.trrList
: comma-separated list of TRR endpoints to test. e.g. "https://mozilla.cloudflare-dns.com/dns-query, https://trr.dns.nextdns.io/" (omit the quotes, obviously). Defaults to an empty list.
doh-rollout.trrRace.canonicalDomain
: the canonical domain that we control. Subdomains will be generated against this. Defaults to "firefox-dns-perf-test.net", the sole purpose of this pref is to allow remotely overriding this if we need to for some reason.
doh-rollout.trrRace.randomSubdomainCount
: Number of random subdomains to test per TRR. Defaults to 5.
doh-rollout.trrRace.popularDomains
: Comma-separated list of "popular" domains - for cached lookups. Defaults to "google.com, facebook.com, apple.com" (without the quotes, obviously)
doh-rollout.trrRace.enabled
: Causes the measurements to run when set, or if true at startup. Note: this has no effect if the complete
pref is set to true! See below.
doh-rollout.trrRace.complete
: This is set to true when we have completed our measurements and recorded telemetry. If this is set, the enabled
pref will not cause us to re-run measurements.
Reporter | ||
Comment 7•5 years ago
|
||
I believe the debug failures in the previous try push can be fixed by removing nsISupports
from the QueryInterface list. Pushed that idea to try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8bdc37df3ff0960eace7a2caa5764626fc074657
Comment 8•5 years ago
|
||
Comment 9•5 years ago
|
||
Thanks for the review. That works for me. I left a review comment in Phabricator.
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Updated•5 years ago
|
Comment 10•5 years ago
|
||
Comment 11•5 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/a53749907dee
https://hg.mozilla.org/mozilla-central/rev/e45f9aabdc9e
Assignee | ||
Comment 12•5 years ago
|
||
Comment on attachment 9126594 [details]
Bug 1613790 - Implement module to measure performance of TRRs and send telemetry. r=johannh!,dragana!,valentin!
Beta/Release Uplift Approval Request
- User impact if declined: This is the code for an experiment that is scheduled to run in 74 release, see https://experimenter.services.mozilla.com/experiments/doh-trr-selection-racing-experiment/
It should have no user-visible impact
- Is this code covered by automated tests?: Yes
- Has the fix been verified in Nightly?: No
- Needs manual test from QE?: Yes
- If yes, steps to reproduce: This experiment has its own QA team
- List of other uplifts needed: None
- Risk to taking this patch: Low
- Why is the change risky/not risky? (and alternatives if risky): The experiment itself should not impact the user experience at all, even when turned on. When turned off, it doesn't do anything except listen for the pref to turn it on. Any bugs in the code that's being uplifted should only impact the experiment, which we could delay in case of issues.
- String changes made/needed: None
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Updated•5 years ago
|
Updated•5 years ago
|
Updated•5 years ago
|
Comment 13•5 years ago
|
||
Comment on attachment 9126594 [details]
Bug 1613790 - Implement module to measure performance of TRRs and send telemetry. r=johannh!,dragana!,valentin!
Needed for 74, uplift approved for 74 beta 9, thanks.
Updated•5 years ago
|
Updated•5 years ago
|
Comment 14•5 years ago
|
||
@Johann, is this bug actionable at this point?
We manually added the experiment’s preference (doh-rollout.trrRace.enabled = true) in the latest Nightly build, but didn't notice any effect in the browser:
- the preferences mentioned in comment 6 were not added,
- the "security.doh.trrPerformance" telemetry event was not generated in the Events section from "about:telemetry".
Comment 15•5 years ago
|
||
bugherder uplift |
Assignee | ||
Comment 16•5 years ago
|
||
(In reply to Carmen Fat [:cfat] - Ecosystem QA from comment #14)
@Johann, is this bug actionable at this point?
We manually added the experiment’s preference (doh-rollout.trrRace.enabled = true) in the latest Nightly build, but didn't notice any effect in the browser:
- the preferences mentioned in comment 6 were not added,
- the "security.doh.trrPerformance" telemetry event was not generated in the Events section from "about:telemetry".
I replied to your email, let me know if you need anything else :)
Thanks!
Comment 17•5 years ago
|
||
Multiple default TRR racing
Firefox RC 74.0
We have finished testing the Multiple default TRR racing experiment for Firefox 74.
Quality status: GREEN - SHIP IT
Why is this feature green?
- During testing, we haven’t found any issues.
Testing Summary:
- Test Suite: Test Rail
Tested Platforms:
-
Windows 10 x64
-
macOS 10.15.1
-
Ubuntu 18.04 x64
Tested Firefox versions:
- Firefox RC 74.0 en-US
Regards,
Robert Martin
Updated•5 years ago
|
Description
•