Closed Bug 923656 Opened 6 years ago Closed 5 years ago

Global search box dropdown options: Having "Search Bing for: foo" web search between local searches is "surprising" (violating Mozilla's Privacy Principles?)

Categories

(Thunderbird :: Search, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 38.0

People

(Reporter: mozdev, Assigned: squib)

References

Details

Attachments

(2 files, 1 obsolete file)

Thunderbird global search is violating some Mozilla's Privacy Principles https://www.mozilla.org/en-US/privacy/ :

* Thunderbird global search is violating "No Surprises", because

  - It's surprizing to mix mailbox search (inside the user's mailbox) and Web search with a Web Search Engine (outside the user's mailbox)

  - When one starts a web search an alert should ask to confirm sending request outside, otherwise it's very easy to leak private search on the network (it happened to many users as collected at Mozilla Summit 2013)

* Thunderbird global search is violating "User Control", because it's not possible to disable the "Search the Web with this or this Search Engine" feature

The fact that it uses Microsoft "Bing" search engine is not the point of this ticket.


Recommended fix: 

* Add a preference to totally disable Web search in global search
* Add an alert each time the user starts a Web search
Marc, thanks for sharing your privacy concern; privacy is an important topic.
However, I'm mostly failing to see the problem from your description.
So pls help us by providing a new description which follows the prescribed structure for bug reports:

Steps to reproduce (STR)
1. start here
2. do this
3. do that

Actual results
- this happens
- that happens

Expected results
- this should (not) happen
- that should (not) happen

Here's my analysis:

* Global search is the [Search...<Ctrl+K>] searchbox on mail toolbar in main 3pane view (and elsewhere)
* typing searchwords "foo" into that box and pressing enter to start the search will search the user's local sql-lite database, iow it will NOT send the search request to the web
* From the search-box autocomplete dropdown, user has a *choice* to click on "Search Bing for: foo" which will trigger web search "foo" using Bing online search engine and return results in a TB tab.

So user needs to actively click on that option to send data to web search, we don't do that secretly. 
Bing is a widely-known internet search engine; even if you don't know actively know that, you can easily arrive at that conclusion because "Bing" certainly does not resemble anything on your local data (e.g. a local folder or such).
So basically, it's on opt-in procedure where the user's consent can be assumed because there's no way we could search with an internet search engine without sending data to the web.

(In reply to Marc-Aurèle DARCHE from comment #0)
> Thunderbird global search is violating some Mozilla's Privacy Principles
> https://www.mozilla.org/en-US/privacy/ :
> 
> * Thunderbird global search is violating "No Surprises", because
> 
>   - It's surprizing to mix mailbox search (inside the user's mailbox) and
> Web search with a Web Search Engine (outside the user's mailbox)

"mix" is a fuzzy term which imho does not adequately describe a separate web search option at the top of a dropdown of local search options. But I see your point, it's a web search option "in between" local search option. Not sure if that's a problem for many users or not, but it's a bit surprising indeed (not what it does, but the fact that it's there in the UI). We'd need to find the bug where this was introduced to find out about the intended scenarios/benefits of this feature in this place.

>   - When one starts a web search an alert should ask to confirm sending
> request outside, otherwise it's very easy to leak private search on the
> network (it happened to many users as collected at Mozilla Summit 2013)
> 
> * Thunderbird global search is violating "User Control", because it's not
> possible to disable the "Search the Web with this or this Search Engine"
> feature
>
> Recommended fix: 
> 
> * Add a preference to totally disable Web search in global search

Users who don't want to use web search can just ignore this option by not clicking on it. So I'm not sure if this really presents an issue of ux-control. But having a pref to disable this might still be reasonable. We should find out if such pref already exists.

> * Add an alert each time the user starts a Web search

Showing an alert for *each* web search will definitely violate ux-efficiency and ux-interruption (see https://bugzilla.mozilla.org/describekeywords.cgi) and is hence wontfix. Especially if we implement the preference for switching web search off, I don't think this is necessary. At the maximum, we could have a "first-time" alert with "never show this again" checkbox, but we are trying to avoid these as they add a lot of complexity to code and in this case, I think it's overkill - just don't click on it and you're safe...

I think if anything, the question to consider here is the potential element of surprise (or *perhaps* ux-error-prevention) of having a web search offered between local search options in a local desktop email application.

OT: It's a pity we didn't meet to talk about this on the Summit ;)
Summary: Thunderbird global search violates some Mozilla's Privacy Principles → Global search box dropdown options: Having "Search Bing for: foo" web search between local searches is "surprising" (violating Mozilla's Privacy Principles?)
(In reply to Thomas D. (away till 23rd Oct) from comment #1)
>
> OT: It's a pity we didn't meet to talk about this on the Summit ;)
>

Yes :'( And as you have guessed, we did discuss this problem, as a group, in a security+privacy session a the Summit.


> So pls help us by providing a new description which follows the prescribed
> structure for bug reports:
> 
> Steps to reproduce (STR)
> [...]

Your analysis is perfect.

> I think if anything, the question to consider here is the potential element
> of surprise (or *perhaps* ux-error-prevention) of having a web search
> offered between local search options in a local desktop email application.
> 

The group in the security+privacy session agreed that the current UI is error prone: the persons in the group had all made web searches by error while they only intended to do search in their mailboxes. And they felt bad about it afterward because private words had leaked out of their system to an external entity (the search engine). Nobody in the group ever had, nor wanted to use the Thunderbird Web Search. So it's really a shared feeling, not a one-person remark.

The main point is: how to prevent the users to inadvertently make a web search. Be it with the mouse or with keyboard navigation, we all, at a time or another, hit the "Search Bing for: foo" choice-list item. Users can't just ignore this option by not clicking on it, when they navigate quickly around the UI controls.

After carefully reading your analysis, here is a new proposal :-)

Either : 
* Completely remove the Web Search from Thunderbird. The decision could be done based on metrics (but again maybe the metrics are flawed because of the inadvertent Web Search) and the bug where the Web Search was introduced. It will really help a lot wrt to code complexity ;-)
* Add a "first-time" alert when the user starts a Web Search, asking the user if she wants to a) Never be notified again (opt-in) b) Disable Web Search (opt-in)

What do you think?
(In reply to Thomas D. (away till 23rd Oct) from comment #1)
> 
> But I see
> your point, it's a web search option "in between" local search option. Not
> sure if that's a problem for many users or not, but it's a bit surprising
> indeed (not what it does, but the fact that it's there in the UI). We'd need
> to find the bug where this was introduced to find out about the intended
> scenarios/benefits of this feature in this place.
> 

Here is the bug that introduced the Web Search (OpenSearch) in Thunderbird: bug 677421
I'd like to add my voice to Marc-Aurele's. 

I *never* want to search the web (Whether its Bing or not I agree is a different issue) from Thunderbird,
But about once a week I end up doing an unwanted websearch, while trying to hit either the "Messages mentioning" or the first choice email address. 

I agree the best solution would be a first-time warning that you could turn it off, but this is something I don't think can even be configured by advanced users through Config editor.
I don't think we should remove opensearch, but I wouldn't mind seeing us remove it from the global search box. The interesting part of opensearch to me has always been that you can select text *in a message* and search for that. In fact, you can even open that search in your browser by default.

Of course, I'm biased since I landed the patch.
The contextual opensearch (that is selecting a text in a message) has never been in my way, never did it make me do an unwanted action. This ticket is only aimed at the global search box including a web search/opensearch. And I'm very glad if everybody feels the same, so let's remove it and close this ticket, shall we?
I think the correct course of action here would be 1) remove the gloda search completer for web searches, 2) finish and land bug 335781, 3) completely remove the websearch tab and push all the searches out to the user's default browser.
How about, as a first (zeroth?) step, we do as I suggested in bug 724434? (i.e. turn off the gloda completer if you have searches set to open in your browser.) This would make life easier for security-minded folks, and would be pretty easy to do. That way, we're not held up by bug 335781, which is needed to let users change the search engine they use.
Taking this. I plan to do essentially what I said in comment 7. However, since bug 335781 is WONTFIX, I'll be moving the websearch engine selector into Options somewhere instead (that'll be done in a different bug, though).
Assignee: nobody → squibblyflabbetydoo
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Josiah: What are your thoughts on completely removing the gloda autocomplete result for web searches? It's removing a feature, so some users might be upset, but it'd probably be relatively easy to add back in as an add-on. However, it also carries the risk of violating users' privacy, and frankly, it's always seemed rather weird that you'd want to do a web search from a mail client*, to say nothing of the fact that Thunderbird's web search tab is pretty crappy. (I should know; I wrote it!)

* Selecting text in a mail and searching for that makes sense to me, though.
Flags: needinfo?(josiah)
+1 to completely removing web searches from the gloda search bar. I suspect that was added originally as part of the hope to generate a search-related income stream, but it proved to be a bust financially, plus we expect to move to a donation-based financial model in any case.

I've also had anecdotal comments that web-based search there is completely unexpected.

When you remove this feature, it would probably be pretty trivial to create an addon to add it back. Would you be able to also do that? I suspect that this feature is not widely used, but we have been surprised before (think Folder columns).
(In reply to Kent James (:rkent) from comment #11)
> When you remove this feature, it would probably be pretty trivial to create
> an addon to add it back. Would you be able to also do that? I suspect that
> this feature is not widely used, but we have been surprised before (think
> Folder columns).

I don't think an independent add-on would be the appropriate place for this, especially since I plan to remove the websearch tab type entirely from Thunderbird. The end result is that, with the add-on, you'd just enter a search and the results would immediately appear in your web browser, at which point it would make more sense to use your web browser in the first place. I could move the websearch tab type into the add-on too, but that code is so awful that I'd rather just purge it from existence.

However, it *would* make sense for the Thunderbrowse developers to add something like this, if they haven't already. At least Thunderbrowse's browser isn't completely terrible. (Our websearch browser tab won't even let you navigate history with the keyboard, as far as I know. I take at least 50% of the blame for this.)
But I thought that you were planning on leaving the context search, which opens as a tab. Doesn't that use the websearch tab type?
(In reply to Kent James (:rkent) from comment #13)
> But I thought that you were planning on leaving the context search, which
> opens as a tab. Doesn't that use the websearch tab type?

I plan on having context searches open in the user's default web browser (which is actually possible now via a hidden pref). There's really no way we can expect to make the web search tab a good experience; it's hard enough for Firefox desktop, and they have *considerably* more resources than we do.

Using the default web browser also means that users with web proxies set up on their web browser (but not Thunderbird) won't accidentally leak requests over a non-proxied connection. I believe there's a bug on this somewhere...
(In reply to Jim Porter (:squib) from comment #10)
> Josiah: What are your thoughts on completely removing the gloda autocomplete
> result for web searches? It's removing a feature, so some users might be
> upset, but it'd probably be relatively easy to add back in as an add-on.
> However, it also carries the risk of violating users' privacy, and frankly,
> it's always seemed rather weird that you'd want to do a web search from a
> mail client*, to say nothing of the fact that Thunderbird's web search tab
> is pretty crappy. (I should know; I wrote it!)
> 
> * Selecting text in a mail and searching for that makes sense to me, though.

So, why would we completely remove it from there, as opposed to hiding it by default? Seems to me the safe thing to do here would be to allow it to become available if they toggle a setting (Assuming we're keeping the context searching, which makes sense to me), since we're keeping the implementation anyway. But if there is a valid technical reason, then sure.

I personally agree that it doesn't make much sense to have it show up where it does, and have never used it myself. But that latter point is what makes me hesitant to say "Go ahead and kill it". If I did use it, I can imagine it would be very helpful (granted, helpful at an odd location, but something I would expect).

So, unless there's a huge technical benefit, I suggest we simply hide it by default, but keep it around if people would like it. (Maybe in the Privacy tab of Preferences under Web Content).
Flags: needinfo?(josiah)
(In reply to Josiah Bruner [:JosiahOne] (needinfo > CC) from comment #15)
> So, unless there's a huge technical benefit, I suggest we simply hide it by
> default, but keep it around if people would like it. (Maybe in the Privacy
> tab of Preferences under Web Content).

I'd argue that if you want to do web searches from the Gloda box, you probably want something more full-featured than we can provide, like Thunderbrowse. It would be totally reasonable for the Thunderbrowse developers to hook into the Gloda box, but it seems weird for us to do it when (eventually) it'll just be redirecting you to your web browser.

I'm also not a huge fan of keeping around disabled-by-default features when the reason we're disabling them is that they're dangerous.
(In reply to Jim Porter (:squib) from comment #16)
> (In reply to Josiah Bruner [:JosiahOne] (needinfo > CC) from comment #15)
> > So, unless there's a huge technical benefit, I suggest we simply hide it by
> > default, but keep it around if people would like it. (Maybe in the Privacy
> > tab of Preferences under Web Content).
> 
> I'd argue that if you want to do web searches from the Gloda box, you
> probably want something more full-featured than we can provide, like
> Thunderbrowse. It would be totally reasonable for the Thunderbrowse
> developers to hook into the Gloda box, but it seems weird for us to do it
> when (eventually) it'll just be redirecting you to your web browser.

Yeah, that's reasonable.

> 
> I'm also not a huge fan of keeping around disabled-by-default features when
> the reason we're disabling them is that they're dangerous.

Because people like to live on the edge. :) Also because they just don't care.

Anyway, I guess I'm fine with completely removing it, but we should be prepared for some backlash.
I can always make an add-on to restore it if we're really worried, although I don't expect it'll get much use.
We did discuss this problem, as a group, in a security+privacy session a the Mozilla Summit 2013 (Europe location). Nobody in the session had ever done web searches from the Gloda box and everyone feel it was dangerous.

SUMO doesn't even mention the possibility of web search in gloda : 
https://support.mozilla.org/en-US/kb/global-search

So I don't see any backlash possibly coming.

Also keeping around this web search in gloda, but disabled, would be contracting technical debt. Let's get rid of technical debt when we can.

Thank you
So in summary - a feature which:
* has security/privacy features; 
* is a negative in that it gets triggered by accident a lot (while looking for messages containing, or the first name); 
* and nobody appears to use (because we all switch to the browser to do searches). 

Must be a hard decision :-) 
Happy New Year.
Ok, this removes the Gloda completer for web searches.
Attachment #8543190 - Flags: ui-review?(josiah)
Attachment #8543190 - Flags: review?(mkmelin+mozilla)
Oops, there was more stuff I needed to remove. I also changed the context menu's string to match Firefox's exactly. It's a tiny difference though.
Attachment #8543190 - Attachment is obsolete: true
Attachment #8543190 - Flags: ui-review?(josiah)
Attachment #8543190 - Flags: review?(mkmelin+mozilla)
Attachment #8543192 - Flags: ui-review?(josiah)
Attachment #8543192 - Flags: review?(mkmelin+mozilla)
Comment on attachment 8543192 [details] [diff] [review]
Remove web searches from the Gloda box (v2)

Review of attachment 8543192 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, r=mkmelin

::: mail/locales/en-US/chrome/messenger/messenger.properties
@@ +780,5 @@
> +# LOCALIZATION NOTE (openSearch.label): The label used in the autocomplete
> +# widget to refer to a search on the web for a short string containing at most
> +# 15 characters. %1$S is the search provider to use. %2$S is the string to
> +# search for.
> +openSearch.label=Search %1$S for "%2$S"

I think I prefer it with the "for: " like it was before
Attachment #8543192 - Flags: review?(mkmelin+mozilla) → review+
(In reply to Magnus Melin from comment #23)
> ::: mail/locales/en-US/chrome/messenger/messenger.properties
> @@ +780,5 @@
> > +# LOCALIZATION NOTE (openSearch.label): The label used in the autocomplete
> > +# widget to refer to a search on the web for a short string containing at most
> > +# 15 characters. %1$S is the search provider to use. %2$S is the string to
> > +# search for.
> > +openSearch.label=Search %1$S for "%2$S"
> 
> I think I prefer it with the "for: " like it was before

I'll let Josiah chime in on that, but I wanted to match what Firefox does, since this feature is practically identical to the Firefox version of it.
Ah, I guess that makes sense
Comment on attachment 8543192 [details] [diff] [review]
Remove web searches from the Gloda box (v2)

Review of attachment 8543192 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/locales/en-US/chrome/messenger/messenger.properties
@@ +780,5 @@
> +# LOCALIZATION NOTE (openSearch.label): The label used in the autocomplete
> +# widget to refer to a search on the web for a short string containing at most
> +# 15 characters. %1$S is the search provider to use. %2$S is the string to
> +# search for.
> +openSearch.label=Search %1$S for "%2$S"

I prefer this slightly, the colon isn't really necessary since we have the quotation marks.
Attachment #8543192 - Flags: ui-review?(josiah) → ui-review+
The patch didn't apply cleanly FYI. This is the updated patch.
Landed: https://hg.mozilla.org/comm-central/rev/f5c8e7536911
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 38.0
Awesome! Thank you very much to you all!
Duplicate of this bug: 724434
You need to log in before you can comment on or make changes to this bug.