1618042 - early DNS lookups fail with network.trr.mode=3 using network.trr.uri with hostname in it

Status: VERIFIED FIXED

(Core :: Networking: DNS, defect, P2)

Description

[Steven Noonan]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0

Steps to reproduce:

1. Run Firefox Nightly (this issue was authored against build ID 20200225094028)
2. Set home custom URL home page in Preferences>Home (e.g. https://start.duckduckgo.com)
3. Set network.trr.uri to a DoH URL with a hostname in it, instead of a bare IP address (e.g. https://example.com/dns-query), but do not set any network.trr.bootstrapAddress values.
4. Set network.trr.mode to 3
5. Close all Nightly windows and start a new Nightly process

Actual results:

On startup, the home page hostname fails to resolve, causing the browser to display "Hmm. We're having trouble finding that site." If a manual refresh is requested, it loads fine.

Expected results:

The home page should load on the first try.

What I think is happening:

The TRR resolver service is not using getaddrinfo() for the hostname in trr.uri early enough in browser startup.

If I specify the IP for the hostname in network.trr.uri via network.trr.bootstrapAddress, the homepage loads fine on startup without the need for a manual refresh. But setting the bootstrap address should not be necessary on Firefox 74+ from what I've read about network.trr.mode=3's intended behavior.

Evidence for this is that if I set MOZ_LOG=nsHostResolver:5 and run the reproduction steps, TrrLookup responds to all the early lookup requests with the message "service not enabled". Later in the startup it does the lookup for the TRR URI's hostname via getaddrinfo() and succeeds, which enables all the subsequent TRR requests to work properly.

Note also that this affects a lot of other lookups in early startup beyond what the homepage would have required (e.g. incoming.telemetry.mozilla.org, profile.accounts.firefox.com, etc).

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Dragana Damjanovic [:dragana]]

Valentin, can we make this better? Probably we can, we can also remember the last used TRR server's IP address. or is it the confirmation code the problem?

Comment 3

[Valentin Gosu [:valentin] (he/him)]

Yes, the problem is the confirmation code - however, for mode 3 I think we can just ignore the confirmation, and all resolutions would just wait for the TRR connection to be established. At worst it would just fail if the TRR connection returns an error or times out.

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Note that this probably also happens for a TRR uri that specifies an IP, but it's just faster if it doesn't have to wait for the TRR's DNS resolution to complete.
I'll try to write up a fix next week.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1618042 - Skip confirmation in TRR mode3 r=dragana

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed75364b23c3
Skip confirmation in TRR mode3 r=dragana

Comment 7

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b7d06bfdd0d5
Backed out changeset ed75364b23c3 for xpc shell failures complaining about test_trr.js CLOSED TREE

Comment 8

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset ed75364b23c3 (Bug 1618042) for xpc shell failures complaining about test_trr.js

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=22ff475bd4e4993a6e2647ee5f37f8394c565c22&tochange=b7d06bfdd0d5655fba6c53d7d86070b1f2f2ace6&searchStr=xpc

Backout link: https://hg.mozilla.org/integration/autoland/rev/b7d06bfdd0d5655fba6c53d7d86070b1f2f2ace6

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=292475475&repo=autoland&lineNumber=3329

[task 2020-03-10T15:24:40.681Z] 15:24:40     INFO -  TEST-START | netwerk/test/unit/test_trr.js
[task 2020-03-10T15:24:49.208Z] 15:24:49  WARNING -  TEST-UNEXPECTED-FAIL | netwerk/test/unit/test_trr.js | xpcshell return code: 0
[task 2020-03-10T15:24:49.209Z] 15:24:49     INFO -  TEST-INFO took 8525ms

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1c0953cf5410
Skip confirmation in TRR mode3 r=dragana

Comment 10

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/1c0953cf5410

Comment 12

[Catalin Sasca, Desktop Test Engineering [:csasca]]

Hi guys, tried verifying this issue, but by following the exact steps from Comment 0, the "Hmm. We’re having trouble finding that site." message is still displayed. Tried on 77.0a1 (2020-04-27) on Windows 7 (x64). Valentin, any idea about this? Thanks!

Comment 13

[Valentin Gosu [:valentin] (he/him)]

(In reply to Catalin Sasca, QA [:csasca] from comment #12)

Hi guys, tried verifying this issue, but by following the exact steps from Comment 0, the "Hmm. We’re having trouble finding that site." message is still displayed. Tried on 77.0a1 (2020-04-27) on Windows 7 (x64). Valentin, any idea about this? Thanks!

What value did you use for network.trr.uri ?

Comment 14

[Catalin Sasca, Desktop Test Engineering [:csasca]]

I used the example from Comment 0, step 3 - (https://example.com/dns-query).

Comment 15

[Valentin Gosu [:valentin] (he/him)]

(In reply to Catalin Sasca, QA [:csasca] from comment #14)

I used the example from Comment 0, step 3 - (https://example.com/dns-query).

Set network.trr.uri to a DoH URL with a hostname in it, instead of a bare IP address (e.g. https://example.com/dns-query), but do not set any network.trr.bootstrapAddress values.

https://example.com/dns-query doesn't host a DNS over HTTPS server, so the fact that it doesn't work is to be expected.
I think you can use the default value for the pref - https://mozilla.cloudflare-dns.com/dns-query

Comment 16

[Catalin Sasca, Desktop Test Engineering [:csasca]]

Yeah, thanks Valentin, that is only just an example indeed, got a little confused :)).

I was able to reproduce the issue on Firefox Nightly (2020-02-25) under Windows 7 (x64) following the STR from Comment 0.

The issue is no longer reproducible on latest Nightly 77.0a1 (2020-04-29) and Firefox 76.0. Tests were performed on Windows 7 (x64), Ubuntu 18.04 (x64) and macOS 10.15.4.

Comment 17

[Valentin Gosu [:valentin] (he/him)]

Thanks for confirming, Catalin!