1634592 - Add reserved domains to list of known TLDs for awesomebar

Status: RESOLVED DUPLICATE of bug 1634650

(Firefox :: Address Bar, enhancement, P2)

Description

[Tom S [:evilpie]]

Reserved TLDs (example, invalid, localhost, test) will never appear on the PSL, because they reserved. However these are still well known TLDs that are often used for local development. I think we should not do a search for these domains and make the life easier for developers using Firefox.

Comment 1

[Marco Bonardo [:mak]]

localhost is already in the domainwhitelist and it's likely the most widely used. What are common cases for example, invalid and test? Are there very widely used software/harnesses using them?
I'd be wary of adding more things to the whitelist that may surprise non-tech users, when tech users are more than able to add something to the whitelist.

We must also consider that when you type "example", we immediately search for it, but then we do a DNS query, if the DNS query resolves we show you a "Did you mean to go to http://example/?" and if you answer yes we add the whitelist for you. We also have a pref to always go to the dns first, for enterprise.

For these reasons I'd be prone to wontfix, because of a least surprise approach and because we already have a feature covering that case.

Comment 2

[Marco Bonardo [:mak]]

Additionally, I don't think bug 1080682 changed anything here? That bug was only about the public suffix and these are TLDs without a public suffix.

Comment 3

[Tom S [:evilpie]]

What are common cases for example, invalid and test

https://searchfox.org/mozilla-central/search?q=mochi.test

Comment 4

[Marco Bonardo [:mak]]

mochi.test is already whitelisted
https://searchfox.org/mozilla-central/rev/7fd1c1c34923ece7ad8c822bee062dd0491d64dc/testing/profiles/unittest-required/user.js#69
I don't think our own code is a good use case, we can easily add to the whitelist.

I still think this is a wontfix considered we have features covering it for users, and it's easy for enterprises to add whitelisted entries.

Comment 5

[Marco Bonardo [:mak]]

also bug 1634650 has been filed to whitelist suffixes

Comment 6

[Tom S [:evilpie]]

Enterprise/developers/sysadmins are users, too. We should not unnecessarily regress Firefox for them. Please reconsider this. I don't think adding 4 well known exceptions in addition to the already huge PSL list of TLDs should be controversial.
(See also some of the comments in https://www.reddit.com/r/linux/comments/gblty4/firefox_77_wont_connect_to_nondomain_address_bar/)

Comment 7

[Marco Bonardo [:mak]]

I still don't understand what's the deal here, it's matter of adding one or two entries to the policies or prefs in the worst case, and those words are perfectly valid searches for most users, we should not make the special case a default for everyone.
I also don't understand why you are speaking about reserved TLDs in regard to a change that only affects suffixes... is localhost a commonly used suffix?
Anyway we'd need a source we can rely on to establish if those are top 4, or others are.

Comment 8

[Asif Youssuff]

(In reply to Marco Bonardo [:mak] from comment #7)

I still don't understand what's the deal here, it's matter of adding one or two entries to the policies or prefs in the worst case, and those words are perfectly valid searches for most users, we should not make the special case a default for everyone.

I think the obvious problem is that while using the PSL to avoid lookups and help user experience, this will end up hurting user experience for another set of users.

For the subset of people that are typing in strings that look like domains that end in a reserved TLD, is it more likely that they are doing searches for these strings, or are they trying to access pages that use those TLDs? I would bet that most of those people are not searching for those strings, because unlike console.log, searching myproject.localhost is simply going to give terrible results on any search engine anyway.

That and the more obvious issue that we are still talking about the address bar here. Where people have typed URLs since Mosaic at least. It remains unexpected to type in a valid URL and be shunted to a search engine, no matter how "valid" the search might be (IP addresses can be valid searches too; there are no invalid searches, as far as I am aware).

The searching is the special case, not navigating to URLs. Navigating to URLs has always been the default case.

Comment 9

[Marco Bonardo [:mak]]

Fair enough, linking https://tools.ietf.org/html/rfc2606 would have been enough!
We first must be able to whitelist suffixes, so this depends on bug 1634650 and then it's just matter of adding a few default prefs.

Comment 10

[Filip Š]

(In reply to Marco Bonardo [:mak] from comment #9)

Fair enough, linking https://tools.ietf.org/html/rfc2606 would have been enough!

There are some other RFCs which specify additional suffixes: https://en.wikipedia.org/wiki/Top-level_domain#Reserved_domains

Comment 11

[Marco Bonardo [:mak]]

(In reply to Filip Š from comment #10)

There are some other RFCs which specify additional suffixes: https://en.wikipedia.org/wiki/Top-level_domain#Reserved_domains

The rfc says "Using '.local' as a private top-level domain conflicts with Multicast DNS and may cause problems for users.", as such I don't think we should be whitelisting .local, since it's suggested to not use it as a tld.
For .onion, it should probably depend on network.dns.blockDotOnion, it could be managed dinamically based on that pref without the need for a whitelisting pref.

Comment 12

[Asif Youssuff]

The rfc says "Using '.local' as a private top-level domain conflicts with Multicast DNS and may cause problems for users.", as such I don't think we should be whitelisting .local, since it's suggested to not use it as a tld.

I'm using the multicast DNS features on my Linux servers to make service discovery simpler on my local network. Some of those services have web frontends, like qBittorrent.

I never mean to make these services accessible outside of multicast DNS and it works fine with it, but I actually ran into an annoying problem today trying to load up ruTorrent today - I typed in the URL today, and I ended up here: https://www.google.com/search?q=butter-server.local%3A9987

Oddly, none of this stuff have to do with ruTorrent - searches for .local domains aren't going to result in anything useful for anyone.

If you have a better way of hosting my in network torrent client that doesn't use a .local domain, I am all ears - otherwise, this just looks like Firefox disrupting an already working use case for theoretical concerns that are actually not problems (Avahi is mDNS).

FWIW, the above string works fine in GNOME Web and Vivaldi on my machine -- it doesn't end up in a search.

Comment 13

[Marco Bonardo [:mak]]

I think the RFC should be respected when it says .local should not be used, moreover the page you linked points out .internal should instead be used in those cases, but that's still a draft. We should add .internal once it's confirmed, or even sooner. I looked around for some articles about .local domains and the general consensus I found seems to be that they should not be used. Let me be clear, I think it's WRONG, .local should have never been used to indicate a specific kind of service or protocol, but that's the world we must live with.
Once we have the suffix whitelist (soon) the workaround will be trivial, so I'd leave that out for now until we have more evidence of the impact.

Comment 14

[Asif Youssuff]

(In reply to Marco Bonardo [:mak] from comment #13)

I think the RFC should be respected when it says .local should not be used, moreover the page you linked points out .internal should instead be used in those cases, but that's still a draft. We should add .internal once it's confirmed, or even sooner.

I think you may have me confused with someone else - I haven't linked to much of anything in this bug.

In any case, have you seen RFC 6762: https://tools.ietf.org/html/rfc6762 ?

It is not a draft and it states:

Multicast DNS (mDNS) provides the ability to perform DNS-like operations on the local link in the absence of any conventional Unicast DNS server. In addition, Multicast DNS designates a portion of the DNS namespace to be free for local use, without the need to pay any annual fee, and without the need to set up delegations or otherwise configure a conventional DNS server to answer for those names.

I think this is unambiguous evidence showing that IETF believes that mDNS ought to be supported with .local.

Let me be clear, I think it's WRONG, .local should have never been used to indicate a specific kind of service or protocol, but that's the world we must live with.

Wrong like -webkit-grid and the rest of the properties here: https://developer.mozilla.org/docs/Web/CSS/WebKit_Extensions#Formerly_proprietary_properties_that_are_now_standard ?

It isn't even the case here that this is a pragmatic move made under pressure like the webkit properties becoming standard (even if WRONG) -- the mDNS RFC exists and is uncontroversial! It is probably stupid to use .local as a tld when you aren't actually using mDNS, but if you are it is perfectly valid and I expect it to work per the RFC.

Comment 15

[Aidan Harris]

Sorry for intruding but can we please get *.localhost whitelisted (this might require https://bugzilla.mozilla.org/show_bug.cgi?id=1220810 to be solved first).

With the latest changes if I go to http://en.wikipedia-on-ipfs.org.ipns.localhost:8080/wiki/Mozilla_Firefox.html and forget to add the http:// protocol at the beginning I end up here https://duckduckgo.com/?t=ffab&q=en.wikipedia-on-ipfs.org.ipns.localhost%3A8080%2Fwiki%2FMozilla_Firefox.html&ia=web.

Another issue, I open a new tab start typing Mozilla_Firefox.html which fills out the URL from the last page I visited and then edit the URL like this: en.wikipedia-on-ipfs.org.ipns.localhost:8080/wiki/Google_Chrome.html. I hit enter and now I end up here: https://duckduckgo.com/?t=ffab&q=en.wikipedia-on-ipfs.org.ipns.localhost%3A8080%2Fwiki%2FGoogle_Chrome.html&ia=web.

Comment 16

[Marco Bonardo [:mak]]

Yours seems a different question, afaict localhost is not a suffix there, it looks like a domain with subdomains, so question is "should subdomains be whitelisted if a domain is whitelisted?". Maybe! We should think about it, but I'd suggest to file its own separate bug.
Anyway once Bug 1634650 is fixed, you could workaround the problem by adding your own pref to whitelist localhost as a suffix, then any domain ending in localhost would be whitelisted.

Comment 17

[Mathew Hodson]

(In reply to Aidan Harris from comment #15)

Another issue, I open a new tab start typing Mozilla_Firefox.html which fills out the URL from the last page I visited and then edit the URL like this: en.wikipedia-on-ipfs.org.ipns.localhost:8080/wiki/Google_Chrome.html. I hit enter and now I end up here: https://duckduckgo.com/?t=ffab&q=en.wikipedia-on-ipfs.org.ipns.localhost%3A8080%2Fwiki%2FGoogle_Chrome.html&ia=web.

That's bug 1635033

Comment 18

[Marco Bonardo [:mak]]

Fixed by Bug 1634650