462578 - Implement Quick Search option "Subject, Sender or Recipient" (Subject, From, To, CC or BCC) to allow filtering for full conversations with a contact easily

Status: RESOLVED FIXED

(Thunderbird :: Search, enhancement)

Description

[Thomas D. (:thomas8)]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1pre) Gecko/20081006 Shredder/3.0a3

A lot of people are keeping copies of their sent messages in their inbox folder, or any other folder, so as to have full conversations / message threads in one folder. Which is also a common practice for some well-known webmail applications. Therefore, it would be highly beneficial to be able to search for both sender and recipient simultaneously using quicksearch. Right now, believe it or not in the year 2008, there is no option in quicksearch that allows returning complete communication with (i.e. from and to) one person/contact. And since we all love comfort, and "Subject, From" has been there for a reason, I think "Subject, From, To or CC" should be the new option. Which will spare me from changing my quicksearch options each time I am looking for a subject rather than a person's communication and vice versa. I don't mind having "To or CC or From" in addition, but some people might think it makes the list too long.
A more friendly title for the requested new option might be
"Subject, Sender or Recipient" or even just
"Subject, Person"

Reproducible: Always

Steps to Reproduce:
1. Your inbox folder holds copies of sent messages; you are searching for all mails to and from a certain person (complete communication with that contact)
2. Look at quick search options to accomplish your search
3.
Actual Results:  
Be shocked that it is not possible to search for full communications (From and to)

Expected Results:  
Find a default quicksearch option in the quicksearch dropdown:
"Subject, From, To or CC"
or, perhaps friendlier to read:
"Subject, Sender or Recipient" or "Subject, Person"





Thinking about it in terms of the length of the options list, would it do much harm to *replace* "Subject or From" with "Subject, From, To or CC"?

Comment 1

[Thomas D. (:thomas8)]

Combine this RFE with the RFE of bug 123788 (quicksearch should be all-words-as-substrings, not phrase), and Thunderbird's quicksearch power will increase dramatically:
> That way, if I type the name of a sender and get too many hits,
> I can quickly append a subject word to the search, and vice-versa.
Combined with this RFE:
If I want to find a full thread of communication with a certain person about a certain topic, all I need to to is type the name of the person and add a subject word...

Example:
Quicksearch Option: Subject, Person
Search words: Peter Business Plan
will find all correspondence from and to Peter where the subject contains Business, or Plan, or both.

Comment 2

[rsx11m]

Thomas has a point here - I'm frequently trying to find messages where I'm searching for e-mails both send to and received from a specific person, so adding this as a quick search option would certainly make sense. On the other hand, there is bug 372068, allowing to customize for multiple search criteria. Thus, this may be a special case of that RFE, where as a difference such a combined search is proposed as a default option here.

Comment 3

[rsx11m]

Confirming per bug 372068 comment #20.

Comment 5

[Kent James (:rkent)]

With the new AllAddresses field, I assume that the request here should be logically extended to be "Subject, From, To, Cc, or Bcc"

Comment 6

[Thomas D. (:thomas8)]

Thanks, Kent, I have updated the summary to reflect your very obvious suggestion (I believe I originally referred to CC under the assumption that it includes BCC).

As the row of header fields to be searched is now getting rather long, I believe that "Subject or contact" would be a fair and plausible abbreviated caption for what this search actually does: Searching for all correspondence involving a certain subject or contact (or both), regardless of where that contact occurs (from, to, cc or bcc). I am open for alternative captions, my thinking is that users will familiarize with that caption easily since iirc "contact" is now the official term for address book entries whose main properties are names and email addresses.

Comment 7

[Kent James (:rkent)]

(In reply to comment #6)
> 
> As the row of header fields to be searched is now getting rather long, I
> believe that "Subject or contact" would be a fair and plausible abbreviated
> caption

We had a bit of discussion on abbreviated captions in bug 310359. At the time, it seemed like Bryan wanted the longer description (though reading over the comments again he was not that firm about it.) But that's what we went with in the string that landed in bug 310359, and was updated to include BCC in bug 483629. I don't have a strong opinion, but without additional official input I will stick with the longer description in any patches that I do.

Comment 8

[:Dossy Shiobara]

What's the use case for searching subject _or_ contact?  I personally want a Quick Search that pulls up all messages to/from a contact, but including subject in that search seems unnecessary.

I'm attaching a patch that adds "From, To, Cc or Bcc" which is consistent with the UI change from Bug 310359.

Comment 9

[:Dossy Shiobara]

Attachment: proposed patch

The search actually working depends on work done from bug 310359.  This exposes that functionality through changing these files:

mail/locales/en-US/chrome/messenger/messenger.dtd
mail/base/content/extraCustomizeItems.xul
mail/base/content/searchBar.js

Comment 10

[Magnus Melin [:mkmelin]]

Dossy: you'll want to ask review and ui-review. 
See https://developer.mozilla.org/en/Getting_your_patch_in_the_tree

Comment 11

[Kent James (:rkent)]

(In reply to comment #8)
> What's the use case for searching subject _or_ contact?  I personally want a
> Quick Search that pulls up all messages to/from a contact, but including
> subject in that search seems unnecessary.
> 
Although I fully agree with your statement "I personally want a Quick Search that pulls up all messages to/from a contact" (which is why Wayne encouraged me to do bug 310359 in the first place), the intent of the current bug is to combine subject and contact together. Although sometimes it is appropriate to revise the meaning of a bug (as I did in comment 5), removing the subject probably goes against the express intentions of the bug filer in this case. So I really think the correct thing to do would be to file a new bug, and mark it dependent on bug 310359, which is still open because the online IMAP part has not been done yet for BCC.

Comment 12

[Andrew Sutherland [:asuth] (he/him)]

As a heads-up, bug 474701's folder display layer patch (which ideally will land very soon) makes changes to quick-search.  While the actual bit-rotting should be fairly easy to resolve, the patch provides for unit tests for quick search, and those would need to be updated to include a test for any new search modes.

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to comment #8)
> What's the use case for searching subject _or_ contact?  I personally want a
> Quick Search that pulls up all messages to/from a contact, but including
> subject in that search seems unnecessary.

I would guess it was decided long ago that the most common searches were that precise combination - from OR subject. And that it would be nice not to have to frequently switch between the two.

Comment 14

[:Dossy Shiobara]

(In reply to comment #11)
> [...] the intent of the current bug is to
> combine subject and contact together. [...]

Understood - I think the rationale is to eliminate the need to actually switch the quicksearch mode between uses, where the common UX is to search for either subject OR sender/recipient - combining the two in one quicksearch eliminates the need to switch back and forth using the UI.

I'd rather not file a separate bug as having the two options ("Subject, From, To, Cc or Bcc" and "From, To, Cc or Bcc") in the quicksearch feels redundant to me.  I'll just revise my patch to include Subject in addition to Sender and Recipient.

Comment 15

[:Dossy Shiobara]

Attachment: proposed patch

This revised patch includes "Subject" to the quicksearch entry, making it "Subject, From, To, Cc or Bcc" now.

Comment 16

[Bryan Clark (DevTools PM) [:clarkbw]]

Comment on attachment 382955 [details] [diff] [review]
proposed patch

Can we come up with an alternate text for expressing this search.

"Subject, From, To, Cc or Bcc" is too long for English, I wouldn't want to see it for something like the Lithuanian locale.

I'm not sure what a good alternative would be.

Subject or contacts?
Subject or addresses?

Ideas?

Comment 17

[Phil Ringnalda (:philor)]

Subject or people?

Comment 18

[rsx11m]

Subject, name or address? (you can search for both real name or an e-mail...)

Comment 19

[:Dossy Shiobara]

Ultimately, "Message Headers" would be nice but different in purpose and intent than this bug originally describes.

It's really frustrating how quicksearch of "Entire Message" doesn't include the headers (which, IMHO it should, otherwise it should be renamed to "Message Body").

Comment 20

[Thomas D. (:thomas8)]

(In reply to comment #19)
> Ultimately, "Message Headers" would be nice but different in purpose and intent
> than this bug originally describes.

Indeed, searching ALL message headers (everything except the body) is a different RFE that is not covered/intended by this bug 462578.
Such an RFE to "Find in all message header" hides somewhere in the mess of bug 271222 (e.g. bug 271222 Comment #1).

> It's really frustrating how quicksearch of "Entire Message" doesn't include the
> headers (which, IMHO it should, otherwise it should be renamed to "Message
> Body").

The renaming issue has been spun off from bug 271222 into Bug 353347 (Quick Search "Entire Message" should be changed to "Body").

Comment 21

[Thomas D. (:thomas8)]

(In reply to comment #18)
> Subject, name or address? (you can search for both real name or an e-mail...)
not sure... but maybe yes...

If you really want it short, I think that "Subject or contact" will be good label to express that you can search for both real name or an email address, because that's the concept of a contact.

"Subject, Sender or Recipient"?

Comment 22

[bronius motekaitis]

I like the brevity of Thomas D's suggestions. However, "contact" implies the thread participant is a saved contact. What about "Subject or email" or "Subject or address"

Comment 23

[Thomas D. (:thomas8)]

(In reply to comment #22)
> I like the brevity of Thomas D's suggestions. However, "contact" implies the
> thread participant is a saved contact.

I am aware that "contact" is a term used in the address book for saved contacts. What I find interesting is that you use "/saved/" contact to clarify. That's what makes me wonder if we could get away with using "contact" here in a broader sense (saved or unsaved contact). After all, any thread participant (saved/unsaved) is someone you are in contact with, so perhaps his contact details (name, email address) can be called a "contact" even when they are not (yet) saved?

> What about "Subject or email" or
> "Subject or address"
Will people associate that "email" (or "address") allows for friendly names as well? When I see the term "email", I am immediately thinking of an asdf@jklö type of thing, not necessarily of names.
Furthermore, "address" sounds ambigous to me, as it could be mistaken for "residential address" (street and place) which is another part of the properties of a contact's card in the address book (Private > Address).

"Subject or person"?
"Subject or Involves"?

Comment 24

[bronius motekaitis]

Good point.

It appears the goal here is to convey specifically what this quick search filter in as little space as needed.  What about: "Subj, From, To, CC, BCC"

Comment 25

[Wayne Mery (:wsmwk)]

Exact would be nice but, as Bryan says in comment 16, exact is too long.
People is short, but doesn't encompass addresses which are not people.
Address is not ambiguous given the context.

However, what if you want to search by name (which does work)? In that case "address" implies one cannot specify a name, which is not accurate.

I think that brings us back to Contact. Or Address/Name

Comment 26

[Magnus Melin [:mkmelin]]

I think i like the "Subject, name or address" suggestion best.

Comment 27

[Bryan Clark (DevTools PM) [:clarkbw]]

Comment on attachment 382955 [details] [diff] [review]
proposed patch

Agreed.  I think this is the best of the suggestions.

"Subject, name or address"

ui-r+ with that change.

Thanks for doing all this work and pushing this patch through!

Comment 29

[Phil Ringnalda (:philor)]

Comment on attachment 382955 [details] [diff] [review]
proposed patch

Actually, asuth will be both vastly faster than me, and also vastly better than me at reviewing this.

Comment 30

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 382955 [details] [diff] [review]
proposed patch

I bit-rotted this patch rather badly some time ago :(

There is a new abstraction on the scene that affects the updated patch: QuickSearchManager:
http://mxr.mozilla.org/comm-central/source/mailnews/base/src/quickSearchManager.js

You will want to both introduce your new constant in QuickSearchConstants as well as propagating it still in searchBar.js.

Please name the constant along the lines of the accepted text (so kQuickSearchSubjectOrNameOrAddress, I guess?)

Quick search gets unit tested now; a new test should be added.  The block starts here:
http://mxr.mozilla.org/comm-central/source/mailnews/base/test/unit/test_viewWrapper_realFolder.js#475

If you are willing to update the patch (sorry about the delay; my bit-rotting refactoring put a lot of things up in the air), please do so and request review from me (":asuth").  You will not need to re-request ui-review and can just mark that + unless you make UI changes not already discussed.

Thanks!

Comment 31

[Thomas D. (:thomas8)]

(In reply to comment #9 and comment #30)
> Created an attachment (id=382914) [details]
> proposed patch

Dossy, your patch looked good before the changes of surrounding structures came into place as per comment #30.
Do you think you could try again with comment #30 in mind?
It would be great to have "Subject, Name or Address" quicksearch option.
Faceted search is nice, but it's too complex for everyday use.
Your patch could add a lot of value to quicksearch.

Comment 32

[Andrew Sutherland [:asuth] (he/him)]

We have actually changed the quick search implementation again a little more.  Now the quickSearchManager is actually the source of the entries it the quick-search widget (which dynamically builds the options).

I think we still have some latitude for string changes in the immediate future for rc1, but not a lot... September 29th is the current planned string freeze date.

Comment 33

[Thomas D. (:thomas8)]

(In reply to comment #32)
> We have actually changed the quick search implementation again a little more. 
> Now the quickSearchManager is actually the source of the entries it the
> quick-search widget (which dynamically builds the options).
> I think we still have some latitude for string changes in the immediate future
> for rc1, but not a lot... September 29th is the current planned string freeze
> date.

I filed Bug 517187 (Need new, consistent, and reordered set of strings for Quick Search Filters). It might be a good idea to set up all the refined strings and new QuickSearchConstants in QuickSearchManager.js NOW and do that systematically so that we're breaking things only once. We can (and should) implement the strings and constants even if some features (like this bug) are'nt ready yet. It'll actually make it easier to fix this bug if the string labels, names and number codes of QuickSearchConstants are prescribed, and we get better code because developers can define consistent constants and labels.

Comment 34

[Kent James (:rkent)]

I'd really like to see this pushed through before the string freeze (since I did the original All Addresses implementation, which this relies on), and I'm willing to do it unless Dossy objects. Since we haven't heard from him in 3 months, I'm going to change myself to assigned, but Dossy feel free to reverse that if you really want to push this through before Sept 29.

Comment 35

[:Dossy Shiobara]

I'd love to see this completed but do not have the time at the moment, so I really do appreciate you taking it over, Kent.  Thank you, and I'm sorry.

Comment 36

[Kent James (:rkent)]

Attachment: Screenshot of new search term in first position.

I propose that the new search term, as the most general search, will probably be the most common quick search term, and should be in the first position. That also makes sense because all of the other quick searches (with the exception of body) are subsets of this one.

Bryan, could you comment on this?

Comment 37

[Peter Lairo]

(In reply to comment #36)
> Screenshot of new search term in first position.

Nice addition!

The word "filter" at the end of every entry makes the list too crowded and harder to parse.

The menu-list seems too long and confusing with all those too-similar combinations.

Comment 38

[Wayne Mery (:wsmwk)]

I like it as is, including that the new item is first in the list.

The "addition" of filter simply retains what was added by the gloda patch. I think it just takes some getting accustomed to.

Comment 39

[Bryan Clark (DevTools PM) [:clarkbw]]

Comment on attachment 402141 [details]
Screenshot of new search term in first position.

Agreed.  That one is the broadest and most likely to work so it makes sense to have it at the top.

Comment 40

[Bryan Clark (DevTools PM) [:clarkbw]]

Having "filter" appear every time isn't optimal and I don't think we can have a "Filters" heading in the menu.  Any ideas to fix this?

Comment 41

[rsx11m]

In analogy to "Search everywhere" maybe "Search by Subject..." would be better, but still potentially lengthy and with that phrase showing up on each item.

Comment 42

[Thomas D. (:thomas8)]

Attachment: Screenshot with new, consistent and reordered labels (as per bug 517187)

It's great to see this is getting attention. I had the same idea as Kent at the same time and I've actually worked my way through the code for the first time ever. I implemented the label changes as proposed in my comment #33. Please read bug 517187 for the details. It's all up and running on my Trunk installation.

So here's an alternative screenshot of how this could look like.

(In reply to comment #37)
> (In reply to comment #36)
> > Screenshot of new search term in first position.

Alternatively, I ordered the quicksearch options from small scope to large scope.

> Nice addition!
> The word "filter" at the end of every entry makes the list too crowded and
> harder to parse.
> The menu-list seems too long and confusing with all those too-similar
> combinations.

It's confusing ("hard to parse") especially because auf the visual, logical, and terminological disorder of the labels /before/ "filter". After reordering and re-labeling (see my screenshot), it's much easier to understand (for details, see bug 517187).

I'll attach the code for Trunk below.

Comment 43

[Peter Lairo]

Everybody knows it's a filter. No need to label the obvious. <-- my opinion

Or add a title above the "filters": 

Search everywhere
-----------------
Search for:                    <-- new title
 Subject, Sender, or Recipient
 Subject or Sender
 Subject or Recipient
 Sender
 Recipient                     <-- that's almost always me (omit?)
 Subject
 Entire message                <-- "message body" is too limited - no header?

Can a menu-item be made non-selectable (no highlighting)?

I think this (reversed) order is better because it follows from "search everywhere" better (more inclusive to less inclusive - with the exception of "entire message", ahem).

Comment 44

[Thomas D. (:thomas8)]

Attachment: quickSearchManager.js (patched for trunk)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20090917 Shredder/3.1a1pre

Here's file 1 of 3 with code for the screenshot of attachment 402165 [details]. The patched the trunk code. Unfortunately, I don't have a development environment set up yet, so I can't generate actual patches. It works on my trunk build as above. Perhaps someone can create a diff that will include the 3 files that I've patched.

That's the full set of changes I'd like to see, including tooltips for each option. Because labels are now using natural language, user can find the details in the tooltips. E.g. for "Recipient filter", the tooltip is "To or CC filter for current view", so user will know exactly what this filter does, while we're keeping the UI options natural, short and clean.

I'll happily change anything if you let me know what (e.g. less UI changes, less internal code changes, provide same changes for Branch build etc.).

Comment 45

[Thomas D. (:thomas8)]

Attachment: quickSearch.properties (labels patched for Trunk, including tooltips)

Here's file 2 of 3 with patched code for the screenshot of attachment 402165 [details]. See comment #44.

http://mxr.mozilla.org/comm-central/source/mail/locales/en-US/chrome/messenger/quickSearch.properties

Comment 46

[Thomas D. (:thomas8)]

Attachment: search.xml (patched for trunk, implements tooltips for quicksearch options)

Here's file 3 of 3 with patched code for the screenshot of attachment 402165 [details]. See comment #44.

http://mxr.mozilla.org/comm-central/source/mail/base/content/search.xml

The first file (attachment 402170 [details] [diff] [review]) was
http://mxr.mozilla.org/comm-central/source/mailnews/base/src/quickSearchManager.js

Comment 47

[Wayne Mery (:wsmwk)]

Consider, what's obvious to us may not be obvious a new user.

Perhaps asuth's use of "filter" was to distinguish gloda's search everywhere from "search folder".  Remember too, these show up as hints in the search widget and with the verb, they are rather more action oriented which I suspect is a good thing.

That said, if each line doesn't have the word search or filter, I like your idea of doing a heading. But suggest adding the word folder, as in "Search folder for:"

Comment 48

[David Ascher (:davida)]

Right, I specifically picked filter vs. search to reflect the reality that "quicksearch" is actually filtering the existing view, not searching the way say a search engine searches.

Comment 49

[Thomas D. (:thomas8)]

Attachment: Sample Screenshot of new QuickSearch Menu Tooltip (implemented by previous attachments)

Here's a sample screenshot of the tooltips that I implemented (see my previous attachments). Tooltips might also help new users, while sparing everyone else (including new users) from UI clutter. I can adapt the tooltips to incorporate Bryan's ideas (comment #47), or variations thereof.

(In reply to comment #47)
> Consider, what's obvious to us may not be obvious a new user.
Well, yes, but we also show a magnifier search icon next to the quicksearch box, and even new users will probably be familiar with this approach from Firefox, Windows Vista etc.
> Perhaps asuth's use of "filter" was to distinguish gloda's search everywhere
> from "search folder".  Remember too, these show up as hints in the search
> widget and with the verb, they are rather more action oriented which I suspect
> is a good thing.
Perhaps yes.
> That said, if each line doesn't have the word search or filter, I like your
> idea of doing a heading. But suggest adding the word folder, as in "Search
> folder for:"
Should it be "Search folder for...", or perhaps "Filter folder for...", so as to be more distinct from "Search everywhere" behaviour?

Alternatively, we might add this into the tooltip as seen in the attached screenshots. So e.g. tooltip for new search option of this bug could be:
"Subject, From, To, CC or BCC filter for current folder"
I guess folder is a broad enough term to include virtual search folders, which can also be quicksearched.

Comment 50

[Thomas D. (:thomas8)]

I really like the way our thoughts seem to converge on this whole issue.

(In reply to Peter's comment #43)
> Everybody knows it's a filter. No need to label the obvious. <-- my opinion
It might help in the main UI, but it obfuscates the search options. In the long run, even for newbies, this word will become meaningless.
> Or add a title above the "filters": 
> 
> Search everywhere
> -----------------
> Search for:                    <-- new title

If so, should be "Filter for:" as per David's comment #48.

>  Subject, Sender, or Recipient
>  Subject or Sender
>  Subject or Recipient
>  Sender
>  Recipient                     <-- that's almost always me (omit?)

No, that's currently To or CC, and both of them can be anyone, including you.
In the long run, it'll probably be To, CC, or BCC.

>  Subject
>  Entire message                <-- "message body" is too limited - no header?

Well, currently it's Bug 353347 (Message Body Search has wrong label). It only searches the body. Unfortunately, full text search including header isn't implemented yet. I tried to patch this, too, but it caused errors so I left it out. 

> I think this (reversed) order is better because it follows from "search
> everywhere" better (more inclusive to less inclusive - with the exception of
> "entire message", ahem).

I think both small "scope -> large scope" and "large scope -> small scope" have their advantages. Above all, I'm very happy we agree options need to be re-ordered and re-labeled!

There are feature requests to complete the missing options, I hope in the long run we'll have a set of three in addition to the traditional ones:
- Message header filter (future)
- Message body filter (currently implemented with wrong label)
- Entire Message filter (future)
These three should be considered when ordering the options. Please have a look at Bug 517187.

Comment 51

[Kent James (:rkent)]

Hmmm, I thought I was just agreeing to do some final coding on a bug where all of the key issues had been decided, but it seems like those issues are being reopened. So I'll add in my comments, though this may just be a rehash of some of Thomas D's stuff.

There are several issues trying to be accomplished in these wordings, and as Thomas D points out, not in completely consistent ways.

1) What is the scope of the search? Original proposal was "search everywhere" for gloda, versus "filter" for the folder view. I don't like "filter" because elsewhere that always refers to operations that take actions on emails, which is not what is occurring here. "View" is a more consistent term, though is not perfect either.

2) What terms are being searched? "Search everywhere" is just scope and gives no clue what is actually being searched (and I have no idea, I sort of assume it is subject and address but I've been meaning to experiment to discover how it works.) All other text gives a list of what is being searched.

3) What exactly do we mean by a search of a sender or recipient? The new term "Subject, Name or Address" hints to the user that they can use someone's name as well as their email address, but the older terms have the same exact issue but not the same hint. That sort of implies that the new term is a different kind of animal, when in fact it is not.

So here is what I would propose.

1)  For the non-gloda terms, we basically take  Thomas D's suggestions, and use the terms "Subject", "Recipient", and "Sender". And also as he says, in the tooltips show the actual headers used, "To", "CC", etc. We get rid of the "filter" but use the expanded text in the tooltip.

2) For the gloda term, we add an expression of what we are searching, using the same terms, but add (everywhere) at the end to denote the expanded folder scope.

3) We ignore the issue of Name versus Email address, which has always been with us.

4) The first non-gloda term is the new term.

With that, we end up with the following (assumming that gloda searches subject, recipient, and sender):

Subject, Sender, or Recipient (everywhere)
--------
Subject, Sender, or Recipient
Subject
Sender
Recipient
Message Body

That makes a shorter, clearer list.

Two possible variations:

1)  There is one minor issue, and that is BCC. This is included in the AllAddresses search term (which would be used for Sender or Recipient) but there is no equivalent To, CC, or BCC search term (which would be just the recipient).  So as an option we could implement quickly which would be technically more accurate, though slightly less consistent, would replace "Recipient" with "Sender or Recipient".

2) To introduce the concept of name OR address, we add to some of the terms (name or address).

Combining these two options, would give (call this options 1&2):

Subject, Sender, or Recipient (everywhere)
--------
Subject, Sender, or Recipient
Subject
Sender (name or address)
Sender or Recipient (name or address)
Message Body

Comments? Since time is short, and Thomas does not have a development environment, I can combine all of this into the current bug myself.

Comment 52

[Peter Lairo]

(In reply to comment #51)
> Subject, Sender, or Recipient (everywhere)

I'd be really disappointed if it didn't also search the body (and header?). i.e., Search *everything* *everywhere*.

> Subject, Sender, or Recipient
> Subject
> Sender (name or address)
> Sender or Recipient (name or address)
> Message Body

I'd leave the "(name or address)" out. It's clutter, and the filter should "just work" for both name and address anyhow. And technically, you would also have to add the text to the first line ("Subject, Sender, or Recipient (name or address)"), which would be silly.

BCC should just be included in the filter for Sender and/or Recipient. No need to mention it explicitly or even give it an extra menu-item. It's a rare case, and it too should "just work".

> Comments? Since time is short, and Thomas does not have a development
> environment, I can combine all of this into the current bug myself.

Comment 53

[Kent James (:rkent)]

(In reply to comment #52)
> 
> BCC should just be included in the filter for Sender and/or Recipient. No need
> to mention it explicitly or even give it an extra menu-item.
>

BCC makes no sense alone with Sender. For Recipient, perhaps it should "just work", but at the moment it does not, with the bare Recipient. It would not be a big project to just tack it on silently to the ToOrCC search term, but that does require a change to the backend search code.

I think you are supporting my proposal without options 1&2. We could decide post-string-freeze whether to add BCC silently to the ToOrCC. Whatever gloda searches, I would change the first term to reflect that. I suspect it would then be:

Subject, Sender, Recipient or Body (everywhere)

I don't know if it includes all headers, but I doubt it.

Comment 54

[Peter Lairo]

(In reply to comment #53)
> Subject, Sender, Recipient or Body (everywhere)

Although technically more accurate, I think brevity might have more value here.

- Search all messages

Comment 55

[Kent James (:rkent)]

(In reply to comment #54)
> 
> Although technically more accurate, I think brevity might have more value here.
> 
> - Search all messages
My issue with that, as with the current "search everywhere" is that it gives no clue at all of what is being searched, nor does it match the pattern of the other terms which describe what is searched. If brevity is what you want, I would suggest:

- Search everything (everywhere)

though I still prefer my original suggestion.

Comment 56

[Magnus Melin [:mkmelin]]

Please don't use "Sender" to represent "From" - they are different!

Anyway, i'd like to filter Everywhere. If we can do that (can we?) we could have 

 [x] Filter View Mode
 --------------------
 Everywhere
 Subject
 From
 ...

(Whatever the order.) If filter mode is unchecked the search would open in a tab.


(Also note that bug 353347 is changing the "Entire Message" label to be "Message Body")

Comment 57

[Kent James (:rkent)]

(In reply to comment #56)
> Please don't use "Sender" to represent "From" - they are different!

If this is important to you, then you should be arguing for "Sender" not "From", as the message header field that is used is not necessarily "From" - see http://mxr.mozilla.org/comm-central/source/mailnews/local/src/nsParseMailbox.cpp#1284 (It's also not the header "Sender", instead it is the best guess as to the sender given the information available).

> 
> Anyway, i'd like to filter Everywhere. If we can do that (can we?) we could
> have

Do you mean everywhere (all folders) or everything (the entire message content) at least in the wording I am encouraging. In either case, adding a quick search for that would be beyond the scope of this bug, which is just trying to get new AllAddresses search attribute available to quick search.

> 
>  [x] Filter View Mode
>  --------------------
>  Everywhere
>  Subject
>  From
>  ...
> 
> (Whatever the order.) If filter mode is unchecked the search would open in a
> tab.

Once again, that seems to me to be a scope expansion for this bug.

> 
> 
> (Also note that bug 353347 is changing the "Entire Message" label to be
> "Message Body")

True - but it is relatively trivial to tack it onto this bug as part of trying to get some consistency to the wording. OK I'm a hypocrite, this is also a scope expansion, just a trivial one.

Comment 58

[Magnus Melin [:mkmelin]]

(In reply to comment #57)
> "From", as the message header field that is used is not necessarily "From" -

Yes, but Sender should always be "Sender: " (it's very confusing othwerwise when you have mailnews.headers.showSender set).

> Do you mean everywhere (all folders) or everything (the entire message content)

Ah yes, my bad :(

Agreed this bug shouldn't expand. The screen shot looks good to me, except there shouldn't be any comma after "Name".

Comment 59

[Magnus Melin [:mkmelin]]

Comment on attachment 402170 [details] [diff] [review]
quickSearchManager.js (patched for trunk)

FTR, only patches are of interest, not complete files.

Comment 60

[Kent James (:rkent)]

(In reply to comment #58)
> 
> Yes, but Sender should always be "Sender: " (it's very confusing othwerwise
> when you have mailnews.headers.showSender set).
>
I'm really confused what position you are promoting. You obviously understand all of the issues, but do you want "From" or "Sender"? I don't see how "Sender:" makes any sense here, since there are no ":" anywhere else. I really don't care, I just want to get this landed before the freeze.
 
> The screen shot looks good to me
Which one do you mean? There are many.

Comment 61

[Magnus Melin [:mkmelin]]

My position was that it currently reads From and should stay that way.

The screen shot with ui-r+ https://bugzilla.mozilla.org/attachment.cgi?id=402141

Comment 62

[Bryan Clark (DevTools PM) [:clarkbw]]

It's a bit hard to follow this bug as the comments keep climbing. :)  I'll just try to state some goals here.

Search vs. Filter - as David pointed out in Comment #48 we specifically choice the word "filter" for the quick search options because that is more descriptive of the behavior vs. a search.  There are lots of different perspectives on this but I think it's a clean separation that we want to make between the two types.  I'm open to new ways of making this happen visually but if all we can easy do is keep using the "$x filter" then I think that's what we have to do for now.

I like most of Thomas' proposals in bug 517187 in terms of text and added tooltips for extra information.  I think the top down order of least specific to most specific filter mechanism has value and likely more value that ordering by text length (which won't persist in localizations)

And in some further iterations today I'd like to change the "Search everywhere" to "Search all messages".  So if you're inclined to include that in your patch that would be appreciated.

A combination of attachment 402141 [details] with the tooltips ideas in Thomas' proposal is what I see as the best path forward.  I'm leery about trying to fix and close all the separate quick search string bugs in one swoop here; I had assumed when this bug started up again that the dust had settled already.

Comment 63

[Peter Lairo]

(In reply to comment #58)
> there shouldn't be any comma after "Name".

Yes there should be a comma:

http://www.drgrammar.org/faqs/#26

  2) Separate three or more items in a series with a  comma.
     Example: "We want to protect cats, dogs, and horses."

Comment 64

[Peter Lairo]

(In reply to comment #62)
> I think the top down order of least specific
> to most specific filter mechanism has value

I hope you mean "most inclusive to least inclusive":

  Search all messages
  -----------------
  Filter current folder on:
   Subject, Sender, or Recipient
   Subject or Sender
   Subject or Recipient
   Subject
   Sender
   Recipient
   Entire message

The term "filter" just doesn't seem right. I prefer:

  Search current folder for:

I'd also maybe change (i.e., add "criteria"): 

 "Save search as virtual folder" 
to 
 "Save search criteria as virtual folder".

Comment 65

[Andrew Sutherland [:asuth] (he/him)]

(In reply to comment #64)
> The term "filter" just doesn't seem right. I prefer:
> 
>   Search current folder for:

Keep in mind that the quick-search is being performed on top of:
- any mail views in effect (messages by tag, people I know, etc.)
- any view flags/crazy views in effect: unread messages, unread threads, unread watched threads
- potentially a virtual folder and its constraints.  (could be single folder or multiple folder)
- potentially gloda search results

From a technical perspective, filter is the most apt description of the mechanism in use.

Of course, the major issue is helping the user build a mental model of what the search box does that helps them find the messages they are looking for.  I would argue that 'filter' is still the right choice because it does not suggest you are going to find something you can't already see.  The filter is just cutting down on what you can already see, not searching in places you can't see.

In contrast, the gloda search is looking places you can't see, which makes "search" significant.

Comment 66

[Wayne Mery (:wsmwk)]

I quite agree with the filter/search contrast goal. In which case "Save search criteria as virtual folder" should become "Save filter criteria as virtual folder".

unfortunately, this goes against the grain of history where everywhere it has been referred to as "quick search". But I'm not sure there's any way to go forward, but to break this mold.

Comment 67

[Wayne Mery (:wsmwk)]

(In reply to comment #66)
> I quite agree with the filter/search contrast goal. In which case "Save search
> criteria as virtual folder" should become "Save filter criteria as virtual
> folder".
> 
> unfortunately, this goes against the grain of history where everywhere it has
> been referred to as "quick search". But I'm not sure there's any way to go
> forward, but to break this mold.

not to complicate matters, but one of the two ways we refer to the saved results is "saved search", i.e. not saved filter.

Comment 68

[Magnus Melin [:mkmelin]]

(In reply to comment #63)
> (In reply to comment #58)
> > there shouldn't be any comma after "Name".
> 
> Yes there should be a comma:
> 
> http://www.drgrammar.org/faqs/#26
> 
>   2) Separate three or more items in a series with a  comma.
>      Example: "We want to protect cats, dogs, and horses."

Generally true of course; the point is that in this case it's not really three items, only two: Subject or Name-and-Address - conceptually. But you couldn't use "and" to separate them because then it would sound like both items must contain the criteria.

Comment 69

[Peter Lairo]

(In reply to comment #68)
> in this case it's not really three items, only two: Subject or Name-and-Address 

No, not really. They are three separate list items: I can enter either the Subject, the person's name, or the person's e-mail address (which might be completely dissimilar to his name). You could write "Subject or Name or Address", but that would be very awkward (even dis-grammatical). The best and logical and grammatically correct formulation is "Subject, Name, or Address".

Also "Subject, Name or Address" (no comma) implies that (irrespective of the Subject) it must be either the name *or* the address, but cannot be both.

Comment 70

[Magnus Melin [:mkmelin]]

As Name-and-address is one, i don't think you can put it quite like that. It's really all "Address". The Name part has only been tagged on there so people would know the actual name can be used (but what is searched is "Joe <[e-mail]>" which all is the address).

Either way, i guess i could live with both versions ;)

Comment 71

[Kent James (:rkent)]

Attachment: Can we have just a little more consistency?

I'm going to try one more time for a small change, because I really find the huge wording change for the new term disturbing in the currently approved variant. It misleads the user that this is a different sort of filter, when it really is just another variation of the other combos. But I used "Recipient" instead of "To, CC, or BCC" in the interest of brevity. I don't think that completely changing the terminology on one particular term ("Name or Address") is the right way to tell the user that she can search for name as well as address.

I really don't think we have time for the tooltips, even though I liked a lot of that work. Perhaps that can be done in another bug, but I really want to get this one finished.

The code patch will follow this view. I can quickly switch to my previous version though if you really hate this.

Comment 72

[Kent James (:rkent)]

Attachment: The code

Comment 73

[Peter Lairo]

(In reply to comment #71)
> I used "Recipient" instead of "To, CC, or BCC" in the interest of brevity. 

I think that the two "To or CC" menu-items should also use "Recipient" - for the same sake of brevity, for the sake of clarity, and for the sake of consistency.

BTW: The word "filter" everywhere is really bothersome. I don't think it adds much value to real-world use. And after using the filter once or twice, users will quickly understand what it's doing (after all, it's a search-box, and it has a magnifying glass icon) - and remain bothered about the clutter forever.

PS: Does "To or CC" (aka "Recipient") include BCC?

Comment 74

[Kent James (:rkent)]

"To or CC" does not include BCC.

Peter, I might agree with you, but we really, really need to close this off and get something out. The appetite for change is very low.

Comment 75

[Bryan Clark (DevTools PM) [:clarkbw]]

Comment on attachment 402635 [details]
Can we have just a little more consistency?

looks good. thanks.

Comment 76

[Bryan Clark (DevTools PM) [:clarkbw]]

Maybe Thomas D wants to use bug 517187 to get the tooltips done.

I would suggest that some future bug actually work to reduce the number of filters present in favor of a bit more adaptive method.  The number of times filter shows up is bothersome mostly because there are 7 different filter types and so we see that word 7 times.

In actuality there are really only 2 types of filters but the reality is we list all the variations and permutations possible.  If we had just two filters, the most inclusive: Subject, From, Recipient and Message Body we could then help people prune the filter types after they have entered something; somewhat similar to how the gloda faceting search works.

Auto-complete can be used to match addresses of from or to,cc,bcc that address.  After completing on an address we could turn off subject matching as a filter option.

Auto-complete can also be used to match subject names which would turn off address matching.

Then I'd suggest adding a "filter type" bar just above the thread list that shows the different options for used in this filtering.

+------------------------------------------------+
| [x] To  [x] Cc  [x] Bcc  [x] From  [x] Subject |
+------------------------------------------------+
|             / thread list /                    |

You might also use the auto-complete to add / remove these options as you type so there is faster keyboard access to changing the type of filter you are using.

[ to clarkbw ]____________.
| to: [e-mail] |
'-------------------------'

Comment 77

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 402637 [details] [diff] [review]
The code

Danke!

Comment 78

[Kent James (:rkent)]

Comment on attachment 402637 [details] [diff] [review]
The code

I guess this does not need sr since it is mail only. Requesting a-tb3

Comment 79

[Kent James (:rkent)]

Comment on attachment 402637 [details] [diff] [review]
The code

Checked in http://hg.mozilla.org/comm-central/rev/f87adfe8b990

Comment 80

[Thomas D. (:thomas8)]

Thanks a lot to everyone for their contributions, and especially to Kent and Peter for gently pushing this into the right direction and ultimately making it happen. I was somewhat frustrated when Magnus annihilated all the effort I had put into my first-ever real code change at one radical swoop, and with it much of the ideas therein :( But then if there's something to learn from the story of this bug, it's probably "good ideas don't die easy" and "together, we are better". Thanks Bryan for mediating, accepting new ideas, and calmly taking all the restless little rascals under Dad's leading wings ;)

(In reply to comment #76)
> Maybe Thomas D wants to use bug 517187 to get the tooltips done.

Well, my code is there (tooltip text suggestions in attachment 402173 [details] [diff] [review], and one line to add the tooltips in attachment 402175 [details]), beyond that there's nothing I can do, as I said when posting the code, I can't provide patches in the technical sense as I don't have the needed tools set up.
Also, since most of the current set of QuickSearch labels haven't been converted to natural languages terms, plus we still have the bothersome "filter" word: having the tooltips NOW wouldn't help that much yet as all the details are still in the labels. I tend to agree with Peter that
> after using the filter once or twice, users
> will quickly understand what it's doing (after all, it's a search-box, and it
> has a magnifying glass icon) - and remain bothered about the clutter forever.

BTW, if we were to implement tooltips like "Subject, To or CC filter for current view" (cf. attachment 402173 [details] [diff] [review]), newbies could easily find out about the "filter" nature of these options by simply hovering over them, while all of us (including newbies) could be spared from the useless clutter of the "filter" word. While I appreciate the somewhat fuzzy difference of concept between "searching" and "filtering", I'm not sure if it really matters for the average user, who just wants to "find the message I'm looking for". Obviously, you can't miss the difference with two extra tabs to get to your "search everywhere" results.

> I would suggest that some future bug actually work to reduce the number of
> filters present in favor of a bit more adaptive method.  The number of times
> filter shows up is bothersome mostly because there are 7 different filter
> types and so we see that word 7 times.

"Subject or Recipient" and "Subject or Sender" might be likely candidates for reducing the options, as they really don't add much value now that this bug has been fixed. Let's keep in mind that unless Bug 123788 gets implemented (QuickSearch should be all-words-as-substrings, not phrase), having "Subject" combined with to other headers just adds the comfort of /not having to switch/ between options, but you're really searching /either/ subject /or/ the other headers. Iow, you can't use it for refined ("faceted") searches like "Peter Holiday" to find Peter's mail with that subject. Search everywhere has it, but that's two extra tabs on the way to results.

> In actuality there are really only 2 types of filters...

Yes, but some important ones are missing, especially  Bug 271222 ("Entire message" quick search option), but also "Message header" for /everything/ in the header (not just AllAddresses, which btw is still a subset of all addresses).
 
> the most inclusive: "Subject, From, Recipient" and "Message Body" we could
> then help people prune the filter types after they have entered something; 
> somewhat similar to how the gloda faceting search works. ...

That sounds like promising ideas to explore.

> You might also use the auto-complete to add / remove these options as you type
> so there is faster keyboard access to changing the type of filter
> [ to clarkbw ]____________.
> | to: [e-mail] |
> '-------------------------'

Good stuff! GMail UI addon does something like this very nicely (https://addons.mozilla.org/en-US/thunderbird/addon/1339): It provides a quicksearch option labelled "Expression", which combines the "Subject, Sender or Recipient" (=all standard headers) option added by this bug with refined gmail quickfilter options like typing "from:Peter subject:Holiday" (or "f:Peter s:Holiday") to find that very mail with ease and very fast (more examples on http://sites.google.com/site/kmixter/gmailui).

Comment 81

[Thomas D. (:thomas8)]

Kent, I understand you introduced nsMsgSearchAttrib.AllAddresses in http://mxr.mozilla.org/comm-central/source/mailnews/base/search/public/nsMsgSearchCore.idl#96. Would it be very difficult to introduce nsMsgSearchAttrib.Bcc, which I suppose will be needed to fix Bug 213318?
If nsMsgSearchAttrib.Bcc was there, bug 213318 could be fixed in very much the same way as this bug, so the patch for that is probably very quick and easy.
We could then swap "To or CC" option against a fully-functional "Recipient" option, which would add to the functional and UI consistency improvements you already delivered by fixing this bug.

Comment 82

[Kent James (:rkent)]

Adding or modifying backend search terms is not difficult for me now that I have gone through the pain to understand it all. But, Thomas, you have unfortunately become active at a very difficult time, when everyone is locking down their code bases in preparation for releases. Usually we have more time to discuss these kinds of things. So please do not be discouraged, you have good ideas and I wish we had more time to implement some of this.

My answer to this class of issues though is custom search terms, which I also implemented a few months ago. I should be able to implement a custom "BCC" or "To, CC, or BCC" custom search term, and I am likely to do that in my FiltaQuilla extension. I just have not been able to get to extension improvements for many months now, as I have been actively trying to squeeze as much as possible into the core before release.

If you want to try development, you might consider doing an extension that would add what you want as a search term. It is not very difficult.

Comment 84

[Thomas D. (:thomas8)]

(In reply to comment #82)
> I have been actively trying to squeeze as much as possible into the core
> before release.

That's a very good thing to do ;)

> Adding or modifying backend search terms is not difficult for me now that I
> have gone through the pain to understand it all. [...]
> I wish we had more time to implement some of this.

Dear Kent ("how can I bribe him into doing this..." 8-)), looking at https://wiki.mozilla.org/Thunderbird/StatusMeetings/2009-10-06, it seems your wish has come true: there's still time to squeeze some things in.
So this could be a deal:
- With the benefit of your experience gained from going through the pain, perhaps you could implement nsMsgSearchAttrib.Bcc in the backend (nsMsgSearchCore.idl#96)
- With the benefit of my newbie experience from patch to add "Entire Message filter" (bug 271222), I could then provide a patch that delivers BCC to the respective QuickSearch Options (bug 213318).

Considering that quicksearch filters are vital for handling mail, we would build on your achievements in this bug. This would go a long way towards improving functionality and UI of quick filters (in the line of the improvements proposed by this bug and bug 517187).

Mental note: Eventually, we also need nsMsgSearchAttrib.MessageHeader.

Comment 85

[Kent James (:rkent)]

(In reply to comment #84)
>
> Dear Kent ("how can I bribe him into doing this..." 8-)), looking at
> https://wiki.mozilla.org/Thunderbird/StatusMeetings/2009-10-06, it seems your
> wish has come true: there's still time to squeeze some things in.
> So this could be a deal:
> - With the benefit of your experience gained from going through the pain,
> perhaps you could implement nsMsgSearchAttrib.Bcc in the backend
> (nsMsgSearchCore.idl#96)

As I said in comment 82, this is easily achievable in a custom search term, so there is little motivation to add it to the backend. The last few weeks, I have been improving the server infrastucture for my MesQuilla extension site, and I will be updating all of my extensions for TB 3.0 Real Soon Now. I'll try to get this added to my FiltaQuilla extension before the next release.

But you should know Thomas that custom search terms are very easy to add in an extension, particularly in cases like BCC where the property exists in the message header. If you are interested, you could easily do an extension yourself that adds just that term.

There are already many search terms of dubious value in the backend. I think in tht future, the first place that search terms should be added is in extensions. Only when their worth and interest are clearly demonstrated there, should we consider adding them to the backend.

Comment 86

[Thomas D. (:thomas8)]

Thanks Kent for immediate feedback. I accept you are not up to do it, because you are doing valuable work in your extensions that have priority.

Could you point us to the bug containing the patch where you implemented nsMsgSearchAttrib.AllAddresses in backend?

(In reply to comment #85)
> (In reply to comment #84)
> As I said in comment 82, [...]
> custom search terms are very easy to add in an
> extension, particularly in cases like BCC where the property exists in the
> message header. If you are interested, you could easily do an extension
> yourself that adds just that term.

Unfortunately not. Extensions aren't easy, and I wouldn't know how. I'm also not really interested in loosing even more time to provide mockups for basic functionality.

> There are already many search terms of dubious value in the backend. I think > in the future, the first place that search terms should be added is in
> extensions. Only when their worth and interest are clearly demonstrated
> there, should we consider adding them to the backend.

Given the current code design, dubious as it maybe, the nsMsgSearchAttrib.Bcc is clearly missing. Thanks to your help, we have included BCC in the "Subject, From, or Recipient filter" (this bug). Including BCC into the other two existing filters that currently involve "recipient" ("To or CC filter" and "Subject, To, or CC filter") is not a luxury that should be reserved for addons. It's basic functionality as long as we have those filters. And the UI for the filters could become somewhat more natural for people if we just had "Recipient" instead of "To, CC or BCC".

Maybe one day we'll have customizable quick filters like in your extension implemented by default (which would be nice). Maybe one day we'll have a different way of filtering, e.g. along the lines of Bryan's comment #76, but we all know that's not going to happen anytime soon. So IMO right now there's much more benefit in fixing the functionality of existing filters. Including BCC in a recipient's filter is not to much to ask from a full grown mail app.

Comment 87

[Kent James (:rkent)]

> Could you point us to the bug containing the patch where you implemented
> nsMsgSearchAttrib.AllAddresses in backend?

If you look at bug 513102, you will see a recent example of a minimal add of a new backend search term. That is a better example for simply adding a new term than the AllAddresses patch.

Comment 88

[Kent James (:rkent)]

Just as  FYI, I have now added BCC as a custom search term to FiltaQuilla, though it will be a few weeks before I post that to AMO.