1884762 - Add networking API to determine if domain is local

Status: NEW

(Core :: Networking, enhancement, P2)

Description

[(PTO until Jan 6) Valentin Gosu [:valentin] (he/him)]

Add API to query whether a domain is local or remote

(In reply to Marco Bonardo [:mak] from bug 1871359 comment #0)

While investigating Bug 1870484, we found that our functionality is overlapping with Domain-Over-Https (called trr in network code and later here).

The Address Bar tries to discern local from remote domains, using PSL to identify top-level domains, and preferences for potential local resources.
Though, once that's done, it's up to network, so even if the Address Bar evaluated something to be local, it may still go to the remote DNS because network decides so. Or even if network considers some resource local, the Address Bar may not know.

Trr is doing pretty much a similar discerning as the Address Bar, but with more sources:

• nsINetworkLinkService.dnsSuffixList
• Preferences network.trr.excluded-domains and network.trr.builtin-excluded-domains
• /etc/hosts

Due to this disconnection between the 2, the final user experience may not be the best.
We'd like to evaluate with the network team what's the best approach to unify this decision path.

For example, we could migrate our prefs to the network ones, but would network accept additional local domains indicated by https://tools.ietf.org/html/rfc2606 (.test, .example, .invalid, .localhost) + https://tools.ietf.org/html/draft-wkumari-dnsop-internal-00 (we have evidence of .internal being used by GCP and Digicert, possibly others).
Having a single pref that users can modify would be beneficial to both.
Otherwise, we could start observing both prefs (ours and network's).

Similarly, having a common souce of truth (API) including /etc/hosts and dnsSuffixList may be beneficial, so we don't have to duplicate all the checks, with risk of introducing differences and bugs.
Something like nsINetworkLinkService could expose an isResourceLocal(URI) (fantasy name) API that we could use.

As a first step, I'd like to hear from Network for thoughts and opinions about this, and whether we could put together a plan to reach convergence.

Requrements:

1. Interface that provides method to determine if domain is local: boolean hasLocalDomain(in nsIURI uri) or boolean isLocalDomain(in AUTF8String domain).
2. Make sure that all of rfc2606 domains are added to the list (.internal too) - builtin-excluded-domains - or maybe we want a different pref that is not TRR specific.
3. Make sure we parse also /etc/hosts when TRR is not enabled (watch out for performance regressions).
4. Consider watching for changes /etc/hosts - this can be a follow-up. I think Chrome does it, but I don't think it's very common for the file to change.
5. Make sure it's an appropriate replacement for domainsuffixwhitelist prefs.

Comment 1

[(PTO until Jan 6) Valentin Gosu [:valentin] (he/him)]

the DNS suffixes are already parsed here

Comment 2

[Kershaw Chang [:kershaw] PTO until Jan 2]

Hi Oskar, could you take a look at this bug?
This is a good opportunity to learn DNS code a bit.

Comment 3

[Oskar Mansfeld]

I started looking into this and have some questions/thoughts:

1. How should we structure the prefs?

   • I think it's a good idea to have a single point where the user would add additional custom local domains, right now they would have to be added to both browser.fixup.domainsuffixwhitelist.<suffix> and to network.trr.excluded-domains or network.trr.builtin-excluded-domains.
   • Suggestion: add a new pref (something like network.local-domains e.g.) that can replace both builtin-excluded-domains and the domainsuffixwhitelist prefs. excluded-domains should be kept because users might want to exclude domains from TRR that are not local. This new pref can then be used by the new API.

2. Where to implement method

   • should this be it's own interface that only implements the new API or can we add the method to an existing interface?
   • e.g. to nsINetworkLinkService where we are already parsing the DNS suffixes since we probably want to use that for the API anyways.

Comment 4

[(PTO until Jan 6) Valentin Gosu [:valentin] (he/him)]

(In reply to Oskar Mansfeld from comment #3)

I started looking into this and have some questions/thoughts:

1. How should we structure the prefs?
   • I think it's a good idea to have a single point where the user would add additional custom local domains, right now they would have to be added to both browser.fixup.domainsuffixwhitelist.<suffix> and to network.trr.excluded-domains or network.trr.builtin-excluded-domains.
   • Suggestion: add a new pref (something like network.local-domains e.g.) that can replace both builtin-excluded-domains and the domainsuffixwhitelist prefs. excluded-domains should be kept because users might want to exclude domains from TRR that are not local. This new pref can then be used by the new API.

I would leave existing network.trr.excluded-domains prefs untouched. We use them to save the user's excluded domains, and they're not exactly local domains.
We can replace network.trr.builtin-excluded-domains and browser.fixup.domainsuffixwhitelist.*

2. Where to implement method
   • should this be it's own interface that only implements the new API or can we add the method to an existing interface?
   • e.g. to nsINetworkLinkService where we are already parsing the DNS suffixes since we probably want to use that for the API anyways.

I'm OK with either. Probably best to create a new interface nsILocalDomainService. We need a few different methods here:

• hasLocalTLD(domain) - returns true if the domain is .internal, .local, etc
• isPublicTLD(domain) - not sure if we need this when we have nsIEffectiveTLDService::hasKnownPublicSuffix
• isLocallyResolved(domain) - returns true if the domain is on the TRR excluded-domain list, or in /etc/hosts, or DNS suffix list. We can split this into several methods (isExcludedFromTRR, isInEtcHosts, matchesDNSsuffixList). I think many consumers just want to know if something is locally resolved, so as to not use TRR / turn it into a google search, so this method is the one we want most.

Comment 5

[Oskar Mansfeld]

currently cleaning up my bugs and unassingning from this one as I'll focus on neqo/quic/http3 work.