522985 - (Power_Filter) [TB] Quick Search should filter for all-words-as-substrings, not phrase (aka Filter Revolution, faceted filter, better/smarter handling of multi-word search, multiple words)

Status: RESOLVED DUPLICATE of bug 552576

(Thunderbird :: Search, enhancement)

Description

[Thomas D. (:thomas8)]

Attachment: Screenshot: Power Filter in action ("Peter holiday" example)

Power Filter: Quick Search should filter for all-words-as-substrings, not phrase (aka Filter Revolution, faceted filter)

This bug proposes a small, yet ingenious change to the current quick search filter logic. This changes the way "multiple search words" are interpreted by quick search filters. It will actually work best with the newly-added "Subject, From or Recipient filter" (my bug 462578):

Instead of filtering for "*search phrase*" (exact string - as we currently do),
filter for "*search*" AND "*phrase*" (all words as substrings - enhancement suggested by this bug).

"That way, if I type the name of a sender and get too many hits, I can quickly append a subject word to the search, and vice-versa" (quoted from Seamonkey equivalent bug 123788).

This is no less than a revolution of how we are currently using filters. In fact, it's like a "faceted filter", only way more easy and efficient to use than "faceted search". To avoid any confusions, I'll just call it "Power Filter". Oh, and it's basically the same thing bugzilla quicksearch uses to find that single bug out of thousands:
Try https://bugzilla.mozilla.org/buglist.cgi?quicksearch=substring%20words
to see what I mean (will find this bug).

Example (Steps to reproduce)

1 You are looking for that mail from Peter about his holiday (From contains peter, Subject contains holiday).
2 You type "Peter" into "Subject, From, or Recipient filter" (quicksearch)
-> 100 mails from and to Peter filtered out (too many)
3 You add "holiday" as a subject word, so that the filter string is now "Peter holiday"

Actual behaviour
- TB will not find the message (finds nothing)

Expected behaviour:
- TB should find the message 
- TB should allow user to narrow down the search by just appending another relevant word to the current filter (i. e. adding "must include" facets for different headers like person or subject, without specifying where exactly the word must be)

So what's needed to implement "Power Filter"? It's not even difficult...

1) Current quick filter logic (e.g. for the "Subject, Sender or Recipient filter"):

Find messages WHERE
(Subject contains "Peter holiday") OR (Sender contains "Peter holiday") OR (Recipient contains "Peter holiday")

(This will never find anything, as we are searching for multiple words from multiple fields as an exact phrase in single fields. Iow, you cannot narrow down your search results by adding relevant words.)

2) New quick filter logic proposed by this bug

Find messages WHERE (
  (Subject contains "Peter") Or (Sender contains "Peter") Or (Recipient contains "Peter")
  AND
  (Subject contains "holiday") Or (Sender contains "holiday") Or (Recipient contains "holiday")
)

(This will allow the user to narrow down search results by adding relevant words from /any/ field.)

More Benefits of "Power Filter":

a) Implementing this RFE would dramatically increase Thunderbird's search power for those users who want to avoid the current complexity and clumsiness of "search all messages" (and that's gonna be a lot of users, e.g. a lot of existing tb2 users who are used to the comfort of visual quick filtering "as you type"!)

b) Implementing this for TB3 would also make the quick filters behaviour more similar/consistent with "Search all messages" behaviour, where you can also add relevant words from any field to narrow down your results (find mail from Peter with subject "holiday" by searching for "Peter holiday"). Currently, we have two competing and totally unrelated behaviours in the same quick search box, depending on which option you choose, which is likely to confuse people.

c) For added search power, yet slower speed (due to old architecture) and less precision, imagine this combined with "Entire Message filter" which I implemented in bug 271222... If we get "Entire Message filter" hooked up with gloda, this will really rock...

Code-wise, we'll probably want to look at quickSearchManager.js and work our way backwards from there.

Comment 2

[Thomas D. (:thomas8)]

5 months after this bug was filed and never received any response from developers, bug 552576 comes along, asking for the same thing, getting immediate attention and displaying an appetite for dupes. However, I propose duping bug 552576 to this one, as this bug is in a much more advanced state, covering the details of everything needed: concept/benefits, use cases, screenshots, implementation details, draft documentation (in the screenshot attachment 406950 [details]).

Comment 3

[Thomas D. (:thomas8)]

Having advocated for this enhancement since 2008 (bug 123788, comment 9), I'm glad that this is now getting attention at newer bug 552576 where Andrew Sutherland promises (in bug 552576, comment 2) to take care of this as part of Bug 545955. In the same comment, bug 552576 is also declared the [duptome] "winner" over any existing bug of which it might be a dupe.
-> duping forward against bug 55276.

Comment 4

[Thomas D. (:thomas8)]

FTR: The search algorithm, exactly as requested by this bug, was implemented with the arrival of quick filter bar in TB 3.1 (Bug 545955), providing a much more powerful and versatile search experience in message list.

4 years later, users are still struggling and campaigning to bring the very same search power to recipient autocomplete and AB quicksearch... (Bug 558931, Bug 529584, Bug 298438...)