Closed Bug 1631609 Opened 3 months ago Closed 1 month ago

DoH Rollout Extension: Steering heuristics


(Firefox :: Security, task, P1)




Firefox 79
Tracking Status
firefox78 --- fixed
firefox79 --- fixed


(Reporter: valentin, Assigned: nhnt11)


(Blocks 1 open bug)


(Whiteboard: [necko-triaged][trr])


(5 files, 4 obsolete files)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
2.13 KB, text/plain
: data-review+

+++ This bug was initially created as a clone of Bug #1626057 +++

We need the rollout addon to be able to query a DNS name and map the CNAME response to one of the TRR URIs in our list.
That URI should be used for that network, if no user choices are present that say it shouldn't be used.

Attachment #9141872 - Attachment description: Bug 1626057 - [WIP] doh-rollout steering detection r=dragana → Bug 1631609 - [WIP] doh-rollout steering detection r=dragana
Priority: P2 → P1

This ensures we can check gDNSService.currentTrrURI

This patch was created by rebasing the patches for central onto beta and resolving conflicts. Also some small modifications to accommodate for the lack of auto-trr selection committing.

Attached patch Folded patch for beta (obsolete) — Splinter Review

This patch is basically a folded version of the patches for central with some modifications to account for the lack of auto-trr-selection committing in 77. Attaching as a patch on the bug in addition to phabricator because we need different patches for beta and central and I don't know how to do that with Lando.

Attached patch Folded patch for beta v2 (obsolete) — Splinter Review

Same as comment 8 but added in the fix for removing the doorhanger tab progress listener.

Attachment #9151922 - Attachment is obsolete: true
Attachment #9150922 - Attachment description: Bug 1631609 - Steer to network-indicated DoH endpoint if detected. r=valentin!,dragana! → Bug 1631609 - Steer to network-indicated DoH endpoint if detected. r=valentin!,johannh!
Blocks: 1638131
Attachment #9141872 - Attachment is obsolete: true
Attachment #9151953 - Attachment is obsolete: true
Attachment #9151921 - Attachment is obsolete: true

Backed out for failures on browser_providerSteering.js.



failure log:

[task 2020-06-05T02:59:02.905Z] 02:59:02 INFO - TEST-PASS | browser/extensions/doh-rollout/test/browser/browser_providerSteering.js | TRR URI set to auto-selected -
[task 2020-06-05T02:59:02.905Z] 02:59:02 INFO - Leaving test bound testProviderSteering
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - Console message: [JavaScript Error: "WebExtension context not found!" {file: "resource://gre/modules/ExtensionParent.jsm" line: 994}]
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - getContextById@resource://gre/modules/ExtensionParent.jsm:994:13
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - recvAPICall@resource://gre/modules/ExtensionParent.jsm:849:24
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - _recv@resource://gre/modules/ConduitsChild.jsm:78:20
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - receiveMessage@resource://gre/modules/ConduitsParent.jsm:357:20
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - JSActor query*_send@resource://gre/modules/ConduitsChild.jsm:63:11
[task 2020-06-05T02:59:02.906Z] 02:59:02 INFO - _send@resource://gre/modules/ConduitsChild.jsm:111:18
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - callParentAsyncFunction@resource://gre/modules/ExtensionChild.jsm:826:18
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - callAsyncFunction@resource://gre/modules/ExtensionChild.jsm:590:33
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - stub@resource://gre/modules/Schemas.jsm:2679:30
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - setSetting@moz-extension://967ee500-aabb-4aa6-93af-c6e7cbca7754/background.js:346:47
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - rememberTRRMode@moz-extension://967ee500-aabb-4aa6-93af-c6e7cbca7754/background.js:103:19
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - asyncsetState@moz-extension://967ee500-aabb-4aa6-93af-c6e7cbca7754/background.js:94:24
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - async
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - asynconConnectionChanged@moz-extension://967ee500-aabb-4aa6-93af-c6e7cbca7754/background.js:547:19
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - Async
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - fire@resource://gre/modules/ExtensionChild.jsm:775:37
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - recvRunListener@resource://gre/modules/ExtensionChild.jsm:779:13
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - _recv@resource://gre/modules/ConduitsChild.jsm:78:20
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - receiveMessage@resource://gre/modules/ConduitsChild.jsm:169:20
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - JSActor query*_send@resource://gre/modules/ConduitsChild.jsm:61:20
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - _send@resource://gre/modules/ConduitsParent.jsm:228:18
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - listener@resource://gre/modules/ExtensionParent.jsm:920:39
[task 2020-06-05T02:59:02.907Z] 02:59:02 INFO - applySafeWithoutClone@resource://gre/modules/ExtensionCommon.jsm:614:24
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - applySafe@resource://gre/modules/ExtensionParent.jsm:502:17
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - addListener/async/<@resource://gre/modules/ExtensionCommon.jsm:2411:39
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - promise callbackasync@resource://gre/modules/ExtensionCommon.jsm:2409:34
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - observerStatus@chrome://extensions/content/parent/ext-networkStatus.js:63:20
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - simulateNetworkChange@chrome://mochitests/content/browser/browser/extensions/doh-rollout/test/browser/head.js:268:16
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - testProviderSteering@chrome://mochitests/content/browser/browser/extensions/doh-rollout/test/browser/browser_providerSteering.js:68:3
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - Async
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - async*Tester_execTest@chrome://mochikit/content/browser-test.js:1104:11
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - nextTest/<@chrome://mochikit/content/browser-test.js:927:14
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:918:23
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO -
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - Console message: [JavaScript Error: "NS_ERROR_NOT_AVAILABLE"]
[task 2020-06-05T02:59:02.908Z] 02:59:02 INFO - Buffered messages finished
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - TEST-UNEXPECTED-FAIL | browser/extensions/doh-rollout/test/browser/browser_providerSteering.js | A promise chain failed to handle a rejection: [Exception... "Component is not available" nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)" location: "<unknown>" data: no] - stack: (No stack available.)
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - Rejection date: Fri Jun 05 2020 02:59:02 GMT+0000 (Greenwich Mean Time) - false == true - JS frame :: resource://testing-common/PromiseTestUtils.jsm :: assertNoUncaughtRejections :: line 265
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - Stack trace:
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - resource://testing-common/PromiseTestUtils.jsm:assertNoUncaughtRejections:265
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - chrome://mochikit/content/browser-test.js:nextTest:610
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - chrome://mochikit/content/browser-test.js:testScope/test_finish/<:1464
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - chrome://mochikit/content/browser-test.js:run:1379
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - GECKO(10440) | MEMORY STAT | vsize 19406189MB | vsizeMaxContiguous 66966942MB | residentFast 1245MB
[task 2020-06-05T02:59:02.909Z] 02:59:02 INFO - TEST-OK | browser/extensions/doh-rollout/test/browser/browser_providerSteering.js | took 1708ms

Flags: needinfo?(nhnt11)

Comment on attachment 9150922 [details]
Bug 1631609 - Steer to network-indicated DoH endpoint if detected. r=valentin!,johannh!

Beta/Release Uplift Approval Request

  • User impact if declined: This patchset enables automatic usage of provider-specific DoH endpoints and advances our efforts to ship DNS over HTTPS to more users.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: We are working with QA already and a test plan is being created.
  • List of other uplifts needed: Bug 1640741
  • Risk to taking this patch: Medium
  • Why is the change risky/not risky? (and alternatives if risky): Not a trivial patch, but also not high-risk due to automated tests + QA support.
  • String changes made/needed: None
Attachment #9150922 - Flags: approval-mozilla-beta?
Flags: qe-verify+
Attachment #9151317 - Flags: approval-mozilla-beta?
Attachment #9151318 - Flags: approval-mozilla-beta?
Attachment #9151319 - Flags: approval-mozilla-beta?
QA Whiteboard: [qa-triaged]

Per our conversation on slack please request data-review on the addition to the telemetry event.

Flags: needinfo?(nhnt11)
Attached file Data review doc
Flags: needinfo?(nhnt11)
Attachment #9156073 - Flags: data-review?(tdsmith)
Group: mozilla-employee-confidential
Comment on attachment 9156073 [details]
Data review doc

1) Is there or will there be **documentation** that describes the schema for the ultimate data set in a public, complete, and accurate way?

They are currently documented in

The heuristics will move out of the system-addon and the definition will move into Events.yaml in the next few cycles and will be documented in the probe dictionary.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

Nhi Nguyen will monitor the telemetry.

4) Using the **[category system of data types](** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, techical data.

5) Is the data collection request for default-on or default-off?


6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?


7) Is the data collection covered by the existing Firefox privacy notice?


8) Does there need to be a check-in in the future to determine whether to renew the data?

No, this is a permanent collection.

9) Does the data collection use a third-party collection tool?

Attachment #9156073 - Flags: data-review?(tdsmith) → data-review+

Comment on attachment 9150922 [details]
Bug 1631609 - Steer to network-indicated DoH endpoint if detected. r=valentin!,johannh!

approved for 78.0b7

Attachment #9150922 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9151317 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9151318 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9151319 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
You need to log in before you can comment on or make changes to this bug.