503171 - Core does not perform domain guessing before keyword search

Status: REOPENED

(SeaMonkey :: Location Bar, defect)

Description

[Johannes Leushuis]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20090223 SeaMonkey/2.0a3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20090223 SeaMonkey/2.0a3

When I type 'cnn' in the location bar, and (1) "www." and ".com" are added and (2) unknown locations are searched for by the search engine, then I expect (1) to happen before (2), so that the web page www.cnn.com is displayed. Not so, a Search result page for 'cnn' is displayed.

Reproducible: Always

Steps to Reproduce:
1. in Preferences in Category Browser, Location Bar in the field Unknown Locations check 'add "www." and ".com" to the location if a web page is not found' and check 'Perform a web search when entered text is not a web location'
2. type 'cnn' in the location bar
3. hit return
Actual Results:  
A search result page for 'cnn' is displayed

Expected Results:  
The web page 'www.cnn.com' is displayed

When either 'add "www." and ".com" to the location if a web page is not found' or 'Perform a web search when entered text is not a web location' is checked, the behaviour is alright. But when both are checked, the behaviour is strange.

When both are checked, now, the checking of the Search box always overrides the checking of the www-com box. Instead it should depend on it. If 'www.cnn.com' does not exits, then Search for 'cnn'. If 'www.cnn.com' does exist, then, according to the checking of the www-com box, display it.

Comment 1

[Robert Kaiser]

Neil, Frank, you have been working with location bar stuff, are we applying domain fixup and keyword search in a wrong order, is this bug invalid, or is there a bug somewhere in between those two extremes?

Comment 2

[[e-mail]]

The code for this is all in docshell, so Firefox will have the same bug.

The normal keyword code (in nsDefaultURIFixup.cpp) only triggers for strings either a) beginning with a "?" symbol or b) containing at least two words, the first of which cannot contain ".", ":" or "?" c) not guessable as a URI

However when the load of http://cnn/ fails there is code in nsWebShell.cpp (nsDocShell.cpp on trunk) that forces keyword fixup before domain guessing, because the original Netscape keyword server included domain guessing.

FYI the original keyword code was added 10 years ago (in bug 11869).

Comment 3

[Johannes Leushuis]

Thanks Neil for replying. Does the code in nsWebShell.cpp, that forces keyword fixup (I assume this results in a search result page) before domain guessing, ever execute the domain guessing part, since a search query always succeeds?

I assume the intention of the makers is to let the domain guessing happen before the keyword search, otherwise the domain guessing will never happen, if both options are checked.

Does this mean that the code in nsWebShell.cpp has been wrong all the time?

Comment 4

[[e-mail]]

As I mentioned before, the code was designed for search keywords to override domain guessing, since the Netscape keyword server already did domain guessing. I can't tell whether this makes the code wrong or not.

Note that you can't try both domain guessing and keywords on the same word, since both features destroy the original input word.

Comment 5

[Johannes Leushuis]

Firefox has a location bar and a search bar, typing 'cnn' in the location bar displays the web page www.cnn.com, typing 'cnn' in the search bar displays a search result page for 'cnn'. That works alright. However it has always been a strong feature of Seamonkey that you could use the location bar for web addresses and search queries. In Firefox one often finds oneself typing the entry in the wrong bar...

Can the design of the code that was designed for search keywords to override
domain guessing, be turned around, so that domain guessing comes first and if succeeds, displays the web page (like www.cnn.com)? If domain guessing and keywords can't be used on the same word since the original input word is destroyed, can the original input word then be saved in a variable or so?

variable=originalinputword
if www.variable.com exists then display www.variable.com
else search variable and display search results

Comment 6

[[e-mail]]

No, Firefox does keyword lookup too. Its keyword lookup for cnn is
http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=cnn
which redirects to www.cnn.com in any browser.

SeaMonkey's keyword lookup for cnn is
http://www.google.com/search?ie=UTF-8&oe=utf-8&q=cnn
which is just a search page.

Note: I don't know whether we're allowed to use Firefox's keyword lookup URL.

Comment 7

[Johannes Leushuis]

Ok.
But if I change
http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=cnn
in for instance
http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=google
then I get a search page again, so it is not a way to trick the Search engine into domain guessing. Perhaps it also works only for Google.

Is there anyone who can tell whether the expected unknown location behaviour of the location bar can be realized?

Otherwise it is no use checking both 'add "www." and ".com" to the location if a web page is not found' as well as 'Perform a web search when entered text is not a web location', since adding www. and .com will never be done.

Radio-buttons should be made then out of the checkboxes.

Comment 8

[[e-mail]]

(In reply to comment #7)
> Is there anyone who can tell whether the expected unknown location behaviour of
> the location bar can be realized?
> 
> Otherwise it is no use checking both 'add "www." and ".com" to the location if
> a web page is not found' as well as 'Perform a web search when entered text is
> not a web location', since adding www. and .com will never be done.
Though surely the same is true in reverse, since if you add the www. and .com and that still fails then the web search won't be performed since by this point it won't know if you originally typed in the full domain name.

Comment 9

[[e-mail]]

(In reply to comment #5)
> variable=originalinputword
> if www.variable.com exists then display www.variable.com
> else search variable and display search results
I overlooked this part of your question, but from the SeaMonkey point of view, we just tell the internal browser to open "cnn" and it then decides (based on preferences) whether to do keyword search or domain guessing (but not both).

Comment 10

[Johannes Leushuis]

OK, that means if we want (already for many years) the internal browser to behave for unknown locations in the expected way , we should redirect our quest for changing the code to another department, to the people who make the code for the internal browser. To which department should we redirect our question then? To the department/component "General" or "Preferences"? I am missing a department/component "internal browser".

Comment 11

[[e-mail]]

Well, I guess we could start with the component where the feature is actually implemented, which in this case is Core:Document Navigation.

Comment 12

[Johannes Leushuis]

Thanks Neil. I rephrased the summary. Should I pose the question again or will people take up the matter here automatically?

Comment 13

[[e-mail]]

Attachment: Possible patch

This makes it so that domain guessing (for supported words) takes priority over keyword search when they are both enabled. Since keyword search used to supersede domain guessing, also requesting review to turn domain guessing "back" off for Firefox, in case they don't want it to start domain guessing.

Comment 14

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 440464 [details] [diff] [review]
Possible patch

r=bzbarsky

Comment 15

[:Gavin Sharp [email: [e-mail]]]

I think this would impact Firefox's behavior when loading things from drags (where we currently don't allow keyword search) - domain guessing would be disabled so we'd just fail to load most single words as "invalid hosts". We probably want to revise the decision to not allow keyword search in that case, so that might not matter, though.

Comment 16

[:Gavin Sharp [email: [e-mail]]]

Comment on attachment 440464 [details] [diff] [review]
Possible patch

I don't think we can take this until comment 15 is addressed somehow.

Comment 17

[[e-mail]]

Well, we could always leave domain guessing on, if you'd be happy with it taking precedence over keyword search...

Comment 18

[:Gavin Sharp [email: [e-mail]]]

I don't think that we would want that behavior change either.

Comment 19

[Ben Bucksch (:BenB)]

> We probably want to revise the decision to not allow keyword search in that case

Drag&drop often happens unintentionally. Feeding the word or sentence to a search engine is an information leak. Sure, DNS currently also sees it, but DNS is less logged and analyzed than search engines. That's probably why it currently is the way you described.

Comment 20

[Andrei Purice]

Hey Johannes,
Can you still reproduce this issue or should we close it?

Comment 21

[Johannes Leushuis]

I can still reproduce it: if you type "cnn" it does a keyword search, not a domain guessing, even if that is required by the preferences
Firefox shows the same behaviour.

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Comment 23

[Sean Feng [:sefeng211]]

This is not a web-platform issue, this is more like the desired behaviour of the address bar. I am moving this to the right component.

Comment 24

[James Teow [:jteow]]

Hi Johannes, you should able to get this behaviour by setting the following preferences manually in about:config:

• browser.fixup.alternate.enabled: true
• keyword.enabled: false

If that doesn't work, please let us know.

Comment 25

[Johannes Leushuis]

Moved to Linux in the meantime.

With the above changes in config. 'cnn' directs correctly to www.cnn.com but the query 'cnn website' gives now an error message. So it is not a good solution. Anybody has a solution or is this quest already moved to another department?

Comment 26

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:adw, could you have a look please?

For more information, please visit BugBot documentation.

Comment 27

[Marco Bonardo [:mak]]

There's no plan to change the current behavior atm, and I think it's good enough to cover most use cases reported here:

1. if browser.urlbar.ctrlCanonizesURLs is true, pressing CTRL+Enter after typing cnn will go to www.cnn.com
2. if keyword.enabled is false it will load cnn, fail but there should be a suggested link to www.cnn.com on the error page
3. if you visit the site often enough, typing cnn should autofill to cnn.com
4. otherwise cnn is searched, that will still be useful to some less technical users to reach their destination. Once the site is visited enough, or bookmarked, case 3 will serve directly the url.

Comment 28

[Johannes Leushuis]

if browser.urlbar.ctrlCanonizesURLs is true, pressing CTRL+Enter after typing cnn will not go to www.cnn.com, but still presents a search page with cnn.com as search result
for this I introduced browser.urlbar.ctrlCanonizesURLs as config-item, with value true.

this happened in seamonkey 2.53.17.1 and in Firefox 117.0.1, firefox already having such a config-item, pressing return and ctrl-return giving same wrong result.
Opera behaves alright according to marco's scheme: pressing enter produces the search page, pressing ctrl-enter goes directly to cnn.com.
But I don't want this difference between enter and ctrl-enter: a browser should do what its settings specify: my settings are Preferences <- Browser <- Location Bar <- Unknown Locations <- Add "www." and ".com"to the location if a web page is not found is checked, so it should browse to cnn.com, and it does not, which is still wrong behaviour.

Comment 29

[:Gijs (he/him)]

(In reply to Johannes Leushuis from comment #28)

if browser.urlbar.ctrlCanonizesURLs is true, pressing CTRL+Enter after typing cnn will not go to www.cnn.com, but still presents a search page with cnn.com as search result
for this I introduced browser.urlbar.ctrlCanonizesURLs as config-item, with value true.

this happened in seamonkey 2.53.17.1 and in Firefox 117.0.1, firefox already having such a config-item, pressing return and ctrl-return giving same wrong result.

I can't reproduce on Firefox 118 or current Nightly - ctrl-enter correctly opens cnn.com (which for me redirects to edition.cnn.com).

Can you clarify on what OS you tested Firefox, and if you can reproduce with 118 (released last week) on a clean profile?

Comment 30

[Marco Bonardo [:mak]]

(In reply to Johannes Leushuis from comment #28)

if browser.urlbar.ctrlCanonizesURLs is true, pressing CTRL+Enter after typing cnn will not go to www.cnn.com, but still presents a search page with cnn.com as search result
for this I introduced browser.urlbar.ctrlCanonizesURLs as config-item, with value true.

There should be no need to introduce the pref in Firefox as it's already there by default, you should just ensure it's set to true.
IF the problem is another product (e.g. Seamonkey) you should report the bug to them.

But I don't want this difference between enter and ctrl-enter: a browser should do what its settings specify: my settings are Preferences <- Browser <- Location Bar <- Unknown Locations <- Add "www." and ".com"to the location if a web page is not found is checked

We have no such preferences. That must be Seamonkey or something else.

Comment 31

[Johannes Leushuis]

The behaviour under Firefox with cnn in the address bar displays a search query, if cnn is deleted from the all autocomplete menu entries with shift-del. With cnn in the autocomplete menu, then autocomplete autocompletes it with ".com" so that an enter as well as shift-enter already direct the browser to www.cnn.com. A general Add "www." and ".com"to the location if a web page is not found -behaviour is not avaiiable in Firefox.

the preference Preferences <- Browser <- Location Bar <- Unknown Locations <- Add "www." and ".com"to the location if a web page is not found , is not present in Firefox, which explains the resulting search query behaviour in Firefox.

Somebody moved this bug from Seamonkey to FIrefox, see above.

Following James Teow's solution:
Hi Johannes, you should able to get this behaviour by setting the following preferences manually in about:config:

    browser.fixup.alternate.enabled: true
    keyword.enabled: false

If that doesn't work, please let us know.

keyword.enabled was true in my case, setting it to false gave the required behaviour, when Add "www." and ".com"to the location if a web page is not found, was checked!
"www." and ".com" were added, and the browser immediately went to www.cnn.com, regardless whether enter or ctrl-enter was pressed!

checking my preferences showed
Add "www." and ".com" to the location if a web page is not found was still checked
Perform a web search when entered text is not a web location was now unckecked, after setting the keyword.enabled config-item to false!

Apparently keyword.enabled is connected to Perform a web search when entered text is not a web location.
This behaviour is satisfying for me, although what I really want is that if the adding of "www." and ".com" does not result in a valid web site, a search query would then be carried out and displayed, hence the checking of the two preferences above. This was what this bug was all about, see my entry at the top of this bug.

Somebody can move this back to Seamonkey and implement this?