940685 - Load about:home instead of the engine's searchForm when the search bar is customized away

Status: RESOLVED FIXED

(Firefox :: Search, defect)

Description

[Mike Connor [:mconnor]]

We initially implemented support for this in bug 325913, and it's been used partly to support a feature in the Sherlock plugin format, partly to better handle the case where a user had hidden their search bar but still looked to invoke it through the menu or keyboard shortcut.  Prior to using the searchForm value, we would use a hardcoded Google URL, so this was a step forward at a time where we did not have something like about:home as an alternative.

Now that we have a page-based implementation that uses the value from the search service, I would propose that:

a) we remove all support for searchForm (Sherlock is long dead)
b) Where a user invokes search without a search bar, we should use about:home as the fallback in all cases.  This will continue to use the same provider for search as currently selected.

I discussed this briefly with Gavin and he seemed in favour of moving forward.  This should be straightforward.

Comment 1

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Perhaps I am confused, but why not treat it like a Google search on empty string, and append the partner code?

(I can look through our data to see how rare this behaviour is, but I know I personally have used this side behaviour.  I might be weird!)

Comment 2

[Mike Connor [:mconnor]]

Basically, I'm aiming to do something that is minimally different from what we currently do.  Doing a "blank" search isn't a defined experience in all cases, but loading a clean (and fast) search UI seems like a win.

Comment 3

[:Gavin Sharp [email: [e-mail]]]

Switching the "Cmd+K with hidden search bar" case to about:home seems reasonable.

I'm inclined to suggest that for blank searches we should just do as Gregg suggests and actually submit blank searches.

I don't think we need to actively rip out searchForm support, it still seems potentially useful in certain cases, and doesn't hurt much to leave in.

Comment 4

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

I think if you cc Bryan Clark, he will suggest that `about:home` isn't a clean search experience, at the moment :)  (no suggestions, no history, except form history, etc).  Not to bikeshed, but he might have an idea that is better than any of ours, that is also trivial to implement.

Comment 5

[Mike Connor [:mconnor]]

I'd rather fix search suggestions in about:home and streamline, tbh.  It's _just_ a matter of hooking up the search autocomplete to a datalist element... it shouldn't be that hard. [1]

Do we need to do anything in the case of blank searches?  In most places enter with a blank field is a silent no-op.  I'd actually expect (on gut, not on data, alas) that the majority of the blank "searches" are user errors that should be quietly ignored.  Thus the proposal would reshape to:

* hitting Enter on a blank field is a no-op (like about:home, address bar, and the major search engines I spot-checked)
* clicking the magnifying glass is treated as a focus event for the search bar
** Modifier+click on search could still open $searchPage in a new tab/window, but I'd be inclined to deprecate this as an obscure power user feature unless we have any data (heatmap study?) that users actually modifier-click this button.  (On one hand I'm happy that we made modifier-click consistently available in many contexts, on the other I'm pretty sure no one uses it except for a handful of us.)
* WebSearch command opens about:home using the currently selected engine.

As for searchForm, it feels more like an attractive nuisance than a useful feature.  We could stop using it for shipped engines, but then you violate principle of least surprise if you install/switch to an engine and a chrome shortcut opens a semi-arbitrary web URL instead of invoking a Firefox feature.  (It also makes it much harder to introduce search engine switching into about:home if we decided to implement that.)

[1] Yes Gregg, I am trolling you.

Comment 6

[:Gavin Sharp [email: [e-mail]]]

(In reply to Mike Connor [:mconnor] from comment #5)
> I'd rather fix search suggestions in about:home and streamline, tbh.

I concur. If about:home search is currently bad, that's a separate problem and not one we're making significantly worse with any of these proposed changes.

> Do we need to do anything in the case of blank searches?  In most places
> enter with a blank field is a silent no-op.  I'd actually expect (on gut,
> not on data, alas) that the majority of the blank "searches" are user errors
> that should be quietly ignored.

I suppose this makes sense, though it's a loss of functionality compared to the status quo, and leaves people without a way to effectively "load google.com" (which has features that about:home doesn't, as Gregg mentions). Not that it's that discoverable of a way...

> As for searchForm, it feels more like an attractive nuisance than a useful
> feature.  We could stop using it for shipped engines, but then you violate
> principle of least surprise if you install/switch to an engine and a chrome
> shortcut opens a semi-arbitrary web URL instead of invoking a Firefox
> feature.  (It also makes it much harder to introduce search engine switching
> into about:home if we decided to implement that.)

I'm not sure what you're talking about here. I'm suggesting keeping the code behind it but not using it for anything in Firefox.

Comment 7

[Mike Connor [:mconnor]]

(In reply to :Gavin Sharp (email [e-mail]) from comment #6)
> (In reply to Mike Connor [:mconnor] from comment #5)
> > Do we need to do anything in the case of blank searches?  In most places
> > enter with a blank field is a silent no-op.  I'd actually expect (on gut,
> > not on data, alas) that the majority of the blank "searches" are user errors
> > that should be quietly ignored.
> 
> I suppose this makes sense, though it's a loss of functionality compared to
> the status quo, and leaves people without a way to effectively "load
> google.com" (which has features that about:home doesn't, as Gregg mentions).
> Not that it's that discoverable of a way...

I suppose it's removing functionality, technically.  I don't think it's an especially compelling case to preserve, though I'm sure there will be a handful of people who've decided that's a good optimization.  I think the cost outweighs the benefit.

> > As for searchForm, it feels more like an attractive nuisance than a useful
> > feature.  We could stop using it for shipped engines, but then you violate
> > principle of least surprise if you install/switch to an engine and a chrome
> > shortcut opens a semi-arbitrary web URL instead of invoking a Firefox
> > feature.  (It also makes it much harder to introduce search engine switching
> > into about:home if we decided to implement that.)
> 
> I'm not sure what you're talking about here. I'm suggesting keeping the code
> behind it but not using it for anything in Firefox.

Oh, so you're saying to leave it in the service, but never make use of it?  I was thinking you meant to just stop using it for default engines.  (This way is certainly less work!)

So the patch for this would:

1) Make clicks on the search button focus the field.

That's handled at http://mxr.mozilla.org/mozilla-central/source/browser/components/search/content/search.xml#452 and should just require calling this.focus()

2) Make enter without a value a no-op.
te
In handleSearchCommand (after the focus check mentioned above) return false if textValue is an empty string.

3) Open about:home instead of .searchForm at http://mxr.mozilla.org/mozilla-central/source/browser/base/content/browser.js#2901

Pretty simple, really.  If I find cycles I'll attach a patch this cycle.

Comment 8

[:Gavin Sharp [email: [e-mail]]]

That sounds right. I think we'll also want to just remove the block at http://hg.mozilla.org/mozilla-central/annotate/c335eaa4494a/toolkit/components/search/nsSearchService.js#l2696.

Comment 9

[Bryan Clark (DevTools PM) [:clarkbw]]

I'd prefer breaking up to two items I'm seeing here.  The original comment was going after this behaviour of not having a search input but using the keyboard command to focus it.  I would assume there aren't many users in that described situation but if the proposed change were made I think you're going with a good option.  The UX team should get a consult here since I'm not sure if they would rather the URL entry focused, either seems right to me.

As for the empty searches I'd like to split that out into something separate and pull it into my "Get people to their search engine faster" group of items.  I want data on how many times this is happening or at least instrument it as we make changes.  In general I'd rather we apply our partner code whenever we send people to their search engine.  And if people are inclined to certain behaviours of clicking UI buttons to get to google then I'd rather they do that than need to type in 'google.com' directly.

Comment 10

[Matthew N. [:MattN]]

(In reply to :Gavin Sharp (email [e-mail]) from comment #8)
> That sounds right. I think we'll also want to just remove the block at
> http://hg.mozilla.org/mozilla-central/annotate/c335eaa4494a/toolkit/
> components/search/nsSearchService.js#l2696.

That will break the empty keyword search feature e.g. "g" in the awesome bar loads https://www.google.com/ rather than searching with an empty search term. I use this feature regularly (about once per day).

Comment 11

[:Gavin Sharp [email: [e-mail]]]

(In reply to Matthew N. [:MattN] from comment #10)
> That will break the empty keyword search feature e.g. "g" in the awesome bar
> loads https://www.google.com/ rather than searching with an empty search
> term. I use this feature regularly (about once per day).

That scenario would end up loading a URL like:
https://www.google.com/search?q=&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:unofficial&client=firefox-nightly&channel=sb

which seems to redirect to a search box page that's equivalent to google.com anyhow, so that shouldn't really be a problem in practice.

Comment 12

[Mike Connor [:mconnor]]

I'd argue that comment 10 is a use-case already covered by bookmark keywords, so bouncing through the search service feels like a weird corner case we don't need to support.

Comment 13

[Matthew N. [:MattN]]

(In reply to :Gavin Sharp (email [e-mail]) from comment #11)
> …so that shouldn't really be a problem in practice.

Not for Google maybe, but redirecting empty searches is not that common in the wild (especially on smaller sites) AFAIK. I actually use this for bugzilla (to take me to the homepage) most of all and that relies on searchForm with the prePath fallback[1]. An empty search there is not nice: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=

Also, not using the searchForm is slower because they rely on server-side redirects.

en-US defaults plus 10 "search plugins for popular domains" from mycroft behaviour with an empty search:
* Google - Good - https://www.google.com/search?q=&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:unofficial&client=firefox-a
* Yahoo - Good
* Bing - Good
* Amazon - Good - http://www.amazon.com/ref=nb_sb_noss_null
* eBay - Bad - http://www.ebay.com/sch/allcategories/all-categories/?_rdc=1
* Twitter - Bad - https://twitter.com/search-home
* Wikipedia - Bad - http://en.wikipedia.org/wiki/Special:Search?search=&sourceid=Mozilla-search
* Facebook - Bad - https://www.facebook.com/search/results.php?q=
* Youtube - Good - http://www.youtube.com/
* Baidu - Good - http://www.baidu.com/
* IMDB - Bad - http://www.imdb.com/find?s=all&q=

"Good" for a homepage and "Bad" otherwise.

(In reply to Mike Connor [:mconnor] from comment #12)
> I'd argue that comment 10 is a use-case already covered by bookmark
> keywords, so bouncing through the search service feels like a weird corner
> case we don't need to support.

I disagree since you can't have one keyword take you to the homepage and do a search with places keywords but I can now with search engine keywords.

I still haven't seen a convincing reason to remove the searchForm feature yet so the cost of losing some functionality is bigger than the benefit I can see.

[1] https://mxr.mozilla.org/mozilla-central/source/toolkit/components/search/nsSearchService.js?rev=d1e4fed327fe#2624

Comment 14

[Mike Connor [:mconnor]]

After a long conversation with Bryan, my gut is still that we should move forward as outlined in comment 7.

Right now, the search bar acts in a way that isn't consistent with other text fields in the browser, including about:home, or search fields on major search engines.  In all other cases hitting enter or clicking the search button is a no-op if the field is empty.  I don't think that's a strong enough use case to continue to break that consistency, and in the absence of data that it's a real use case I'm always going to argue for implementing what we think is correct (and predictable) behaviour.  Especially if it means we get proper credit for searches.  We can always roll the change back if we get enough negative feedback.

Comment 15

[Matthew N. [:MattN]]

(In reply to Mike Connor [:mconnor] from comment #14)
> Right now, the search bar acts in a way that isn't consistent with other
> text fields in the browser, including about:home

To be honest, Jared and I were discussing "fixing" about:home to do a search at our last work week so opinions differ on what is the right behaviour. Google has changed behaviour on this in recent years IIRC.

Comment 16

[Mike Connor [:mconnor]]

Attachment: deprecateSearchForm

* Focus the search field if the search button is clicked with no value present
* make empty search in the search bar a no-op to match about:home (and most search engines that I looked at)
* open about:home instead of searchForm when search is invoked as a command without the search box visible.

Comment 17

[Bryan Clark (DevTools PM) [:clarkbw]]

(In reply to Mike Connor [:mconnor] from comment #16)
> Created attachment 8385725 [details] [diff] [review]
> deprecateSearchForm
> 
> * Focus the search field if the search button is clicked with no value
> present
> * make empty search in the search bar a no-op to match about:home (and most
> search engines that I looked at)

Please, do not commit this change.  I understand the desire for consistency but without data to help us understand the behavior behind the search input we're just taking a costly stab in the dark.  For any other system of Firefox I would agree with comment 14 that moving in the right direction in the absence of data isn't a great idea but still acceptable; for search I hold a higher standard.  We need to be much more calculated.

The only data we really have is that the search input is used more than any of the other search access points yet this change assumes that the behavior of the other inputs is more correct and is trying to change the search input to be consistent with them.  You can argue that numbers for the address bar are close but that only segments the audience, it doesn't change the search input numbers.

I'm not advocating for consistency in the opposite direction, I just want a data probe for FHR that shows people are using the about:home style behavior and not using the search input style behavior.  It will take much longer but I don't think we want to make mistakes we don't have to.

> * open about:home instead of searchForm when search is invoked as a command
> without the search box visible.

Comment 18

[:Gavin Sharp [email: [e-mail]]]

(In reply to Bryan Clark (Firefox Search PM) [:clarkbw] from comment #17)
> I'm not advocating for consistency in the opposite direction, I just want a
> data probe for FHR that shows people are using the about:home style behavior
> and not using the search input style behavior.

I don't know what this would even mean. You can't "use the about:home style behavior" because it just doesn't do anything in the scenario we're discussing (blank searches).

I'm not really in favor of the empty search changes, but that's mostly because I don't see much benefit to breaking this use case (or at least not any that couldn't be achieved in other ways).

The browser.js part of mconnor's patch (load about:home when trying to search with the search bar completely removed) is uncontroversial, so let's land that and discuss empty searches in a followup.

Comment 19

[:Gavin Sharp [email: [e-mail]]]

Comment on attachment 8385725 [details] [diff] [review]
deprecateSearchForm

browser.js change looks good, other changes less clear.

Comment 20

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Don't land +1.

I agree with Bryan here.  'Empty' searchbox searches do exist in the wild.  I don't know what people are expecting their either, but "nothing" is probably not it.

Comment 21

[Marco Bonardo [:mak]]

I was sure to have posted a comment here, but sounds like either I didn't or I posted it to the wrong bug.
in bug 959576 I need a searchForm url with affiliate code, due to comment 13 I can't expose just the submission url with an empty string, since while here it's for a less disoverable feature, my use-case is the autocomplete.
Since this bug is aiming to also remove searchForm, I'm a little bit lost on what to use, and I wouldn't want to reintroduce searchForm+affiliate one minute after we removed searchForm...

Comment 22

[Mike Connor [:mconnor]]

Attachment: useAboutHome

For now, let's use about:home instead of searchForm if the search bar isn't accessible.  One-liner.

Comment 23

[Mike Connor [:mconnor]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/2d7ce67f5874

If this doesn't cause major problems I'll likely request uplift to Aurora to get it out sooner.

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Backed out for mochitest-bc failures.
https://hg.mozilla.org/integration/mozilla-inbound/rev/53d44383015c

https://tbpl.mozilla.org/php/getParsedLog.php?id=37011794&tree=Mozilla-Inbound

Comment 25

[Mike Connor [:mconnor]]

Attachment: testFixSearchForm

Waiting on a test run to ensure this works.

Comment 26

[Mike Connor [:mconnor]]

Comment on attachment 8399571 [details] [diff] [review]
testFixSearchForm

Pretty much rs=, not sure how I missed this test.

Comment 27

[Mike Connor [:mconnor]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/03314fb9df16

Comment 28

[Mike Connor [:mconnor]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/365dad594390

Comment 29

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/03314fb9df16
https://hg.mozilla.org/mozilla-central/rev/365dad594390

Comment 30

[Marc Auslander]

I had no idea you planned this till it happened.

Please please give me a way to go back to visiting https://www.google.com when I use ctrl-k!

Why.

1.  There is no way to verify that about:home is using an https access.
2.  The search just isn't as good.  The real google search starts showing tentative results as you type and shows the type-ahead correctly.  Your version is a pale imitation.
3.  I don't need distracting firefox stuff when I'm searching.

Comment 31

[Igor Mackeev]

And please please give me a way to go to https://www.gooogle.com/ncr when I press Ctrl-K!!!

Comment 32

[Igor Mackeev]

Actually... I've found out that my problem (with  https://www.gooogle.com/ncr)can be resolved by creating searchplugins directory in user profile and placing my modified xml file in there.

But I still don't get why Alt+Home, Ctrl-K and Ctrl-E shortcuts now all result in one about:home page!