Open Bug 519006 Opened 15 years ago Updated 2 years ago

Provide a way of refining query based searches (tag and people based) using word matches. Search All Messages / [faceted search]

Categories

(Thunderbird :: Search, defect)

defect

Tracking

(blocking-thunderbird3.1 -)

Tracking Status
blocking-thunderbird3.1 --- -

People

(Reporter: davida, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [has patch][needs new assignee][has l10n impact])

Attachments

(3 files, 3 obsolete files)

Attached patch patch v1 (obsolete) — Splinter Review
If you do a query-based search (e.g. a "messages tagged Foo" or "messages involving bob"), you end up with a search result page which can only be narrowed down by faceting.  Unfortunately, there's no way to say "yes, but only show me messages which mention "warts"!  

That seems like a significant problem, in that people who use the affordances like autocomplete and tag-based searches can end up "stuck".

Bryan and I, and then later Andrew and I, discussed this.  My current proposal from those conversations:

1) all search results pages include a text field (a la Bing, and yes, Google)

2) for full-text search results, that text field is pre-filled with the search terms.  Hitting enter in the search field then causes a new full-text search with those terms.

3) for query searches, that text field is cleared.  A status area underneath the text field explains the current query ("messages tagged: important", "messages involving: bob@dole.com").  Typing in some words and hitting enter in the search field _filters_ the search results based on finding (in a case insensitive manner) the messages among the (up to) 400 hits which mention those words.  The filtering is in-place in the tab, so clearing the search field and hitting enter restores the original faceted search.  Facets choices are preserved.

A couple of other notes:

1) ideally we'd build a compound SQL query which did relevance scoring based on both FTS and facet values.  We don't have time for that for TB3.

2) I currently do a totally ad-hoc scoring of the importance of search hits in  GlodaMessage.fulltextMatchEverything, which should be informed by what the FTS search does at some point, except that I'm not actually using those scores in the display because of some weird interactions w/ the faceting code which I don't have the time to fix -- so we're maintaining date based ordering, which seems fine for now.

TODO: deal with multi-word search scoring in GlodaMessage.fulltextMatchEverything().

I'm putting this up so that Bryan can play with it and the UI, and Andrew can maybe put his 2c on the code.  It's not formally review ready yet.

I think bryan agrees this is blocking. Unfortunately it has some strings.
Flags: blocking-thunderbird3+
Looks good and headed in the right direction to me.

"tabmail.setTabTitle(tab)" is how you tell it a tab's title (needs to)change(/d).

Your localization note is missing a paren.

It appears that the scoring logic problem for the query case is that it's currently doing the wrong thing.  In fact, it looks like the searcher case is also sorta doing the wrong thing too.

In the searcher case, it's just using the order that the results are returned by the SQLite query.  The searcher puts the scores in searcher.scoresById, but nothing is ever done with them.  We should probably change the code to use those scores.  If they turn out to be worse than our SQL query's approximation, we can always twiddle the constants.  It's debatable whether the searcher should be doing this internally.

In the query case, Gloda.scoreNounItems returns a list of score values that correspond to the list of input items.  These numbers are obviously not messages, so that will never turn out well.  Clicking on the "relevance" button never does anything much because the code dies in an exception really early.  It needs to do what msg_search.js does and build a map and then sort on it.  This could be shared code if the sorting is not moved into the searcher.
Bryan: I was wondering if this thing actually needs string changes, once we move from "Searching for" to "Search", which I think one of your other patches already does.  I still we do want to land it soon to get experience, but I'm not sure it actually needs new strings.  (Oh, one reason: I implemented emptyText, but it looked really odd to me, no matter what words I put in there)
(In reply to comment #2)
> Bryan: I was wondering if this thing actually needs string changes, once we
> move from "Searching for" to "Search", which I think one of your other patches
> already does.  I still we do want to land it soon to get experience, but I'm
> not sure it actually needs new strings.  (Oh, one reason: I implemented
> emptyText, but it looked really odd to me, no matter what words I put in there)

Right, I was thinking the same thing when I tried it.  I think by using different styling on the input area we can avoid using the emptytext.  And that Search: string is part of my strings patch, don't try to take that away from me! :)
Whiteboard: [no l10n impact]
Assignee: nobody → david.ascher
Attached patch WIP, v2 (obsolete) — Splinter Review
This addresses a few things:

- fix bitrot
- the title updates
- the title is set even for query searches, and filtered query searches
- the icon in the tab works for pinstripe
- no more commas are inserted

I don't yet deal w/ the sorting bugs (which predate this patch anyway)
Attachment #403011 - Attachment is obsolete: true
Oh, and after discussions w/ clarkbw and aza I want to add an (x) to remove query descriptions under the search field if there is a refining text, so that people can widen their search if their initial guess about tags or people involved turn out wrong (but not let them end up with a "show me EVERYTHING!", useless, result)
Attached patch WIP, v3 (obsolete) — Splinter Review
Updated patch that allows of the clearing of the facet once a word-based filtering has occured.  works nicely, too.
Attachment #403588 - Attachment is obsolete: true
Note to self: 

move close icon to pinstripe specific theme.

gnomestripe should use: list-style-image: url("moz-icon://stock/gtk-close?size=menu");

winstripe should use: list-style-image: url("chrome://global/skin/icons/closeSidebar.png");
Attached two screenshots showing the planned layout, for l10n folks in particular
Does it need a judiciously placed "and"? And perhaps having the Search box below, to show more clearly it's an addition rather than a replacement?

Search: involves clarkbw@gmail.com and contains
        [                ]

Search: tagged: Personal and contains
        [                ]

Gerv
We can't add strings for TB3, so I'm hoping we can get the intent across w/ no new words.
Hmm. :-| I guess I'd still recommend putting the box under the existing conditions rather than above.

       involves clarkbw@gmail.com
Search [                ]

       tagged: Personal
Search [                ]

Gerv
Whiteboard: [no l10n impact] → [no l10n impact][has patch, needs work]
Attached patch WIP, v4Splinter Review
Putting on asuth's queue for after he's done w/ his mega-patch.

I'm deferring the sorting issues for now, as they're unrelated to this issue.

Bryan, if you can look at this, and opine as to what you want the layout to be like, that'd be great.
Attachment #403616 - Attachment is obsolete: true
Attachment #404699 - Flags: ui-review?(clarkbw)
Attachment #404699 - Flags: review?(bugmail)
Comment on attachment 404699 [details] [diff] [review]
WIP, v4

so far the close icon doesn't work on linux;  likely for the reasons you mentioned in comment 8.  I'm going to try it on my mac now.
Attachment #404699 - Flags: ui-review?(clarkbw) → ui-review-
Comment on attachment 404699 [details] [diff] [review]
WIP, v4

I like the current arrangement of the search entry on top and pre-filter on the bottom.  It allows our search to stay in a similar position when the pre-facet is removed.

I'm minusing this because I don't like the fact that we only show the (x) button after the person enters some text.  We have a nice "No messages match your search" empty page that could be used so people don't have to jump through the hoop of searching for some text and then removing the original facet.  I'm not sure why I didn't suggest that before.  If we always show the (x) button on the pre-facet and when the person removes it we display the empty search I think that will be a lot better.

Also we should file a bug about adding the auto-complete to this inline text search as it should match the functionality of the toolbar - at least eventually.  Right now if I search for david@ascher and then realize I wanted davida@gmail instead I have to go back up into the toolbar.
We're having a discussion about this bug in mozilla.dev.l10n. Francesco (Italian localizer) makes the point that using a different string here 
changes the semantics significantly, which would mean that localizations 
would have to adjust here, which should not happen now that we're in 
string freeze.

I do not fully concur with Francesco, that this will break localizations, 
but the quality of the localization will definitely suffer.

Here's the discussion (slightly edited for readability):

Il 14/10/09 10:55, Simon Paquet ha scritto:
> Can you elaborate more here or (even better) in the bug itself? From 
> looking at https://bugzilla.mozilla.org/show_bug.cgi?id=517449#c29 my 
> impression was, that there is no problem on the l10n side. If that's 
> no longer the case we need to flag this immediately, so that our 
> developers can work on this. 

Hi Simon,
consider for example this screenshot (searching messages involving a 
specific e-mail address): attachment 404270 [details]

On the current nightlies (Gecko/20091013 Shredder/3.0pre) the situation is 
like this: http://www.pseudotecnico.org/mozilla/screenshot/gloda.png

Which, in English, sounds like "Search that involve X" (and it doesn't make
sense).

In summary:
* without bug 519006, the right translation should be something like 
  "Search messages" + "that involve X"
* with bug 519006 landed (and the consequent change of semantics), the 
  right translation should be "Search" +"that involve X"

So, when bug 519006 lands, I think that it will probably break a lot of
localizations.
Per comment 17 this has l10n impact and should be treated accordingly.
Whiteboard: [no l10n impact][has patch, needs work] → [has l10n impact][has patch, needs work]
Perhaps this will make a nice feature for the planned quick-turnaround 3.1...
David, any input on comment 17?
Given where we are in the release, the l10n impact, I think we should push this out to 3.1.  clarkbw agrees.
Flags: blocking-thunderbird3.1+
Flags: blocking-thunderbird3-
Flags: blocking-thunderbird3+
Comment on attachment 404699 [details] [diff] [review]
WIP, v4

I'm canceling the review request because this has probably physically bit-rotted even if there isn't conceptual rot, which there may be.  Also, I fear the metrics.
Attachment #404699 - Flags: review?(bugmail)
Given current 3.1 priorities, I don't think we'd hold for this if it were the last bug standing.  Adding wanted-thunderbird+, however.
blocking-thunderbird3.1: --- → -
Flags: blocking-thunderbird3.1+ → wanted-thunderbird+
Blocks: 596212
Summary: Provide a way of refining query based searches (tag and people based) using word matches → Provide a way of refining query based searches (tag and people based) using word matches. Search All Messages / [faceted search]
Whiteboard: [has l10n impact][has patch, needs work] → [has patch][needs new assignee][has l10n impact]
David A. (assignee), are you still working on this?
(In reply to Thomas D. from comment #24)
> David A. (assignee), are you still working on this?

Nope.  Unassigning.
Assignee: dascher → nobody
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: