479767 - Allow filtering/searching for folders in folder pane by substring search (Implement quick filter for folder pane)

Status: NEW

(Thunderbird :: Folder and Message Lists, enhancement)

Description

[Jens Müller (:tessarakt)]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1b3pre) Gecko/20081204 Thunderbird/3.0b1

It should be possible to filter the folder view based on a substring search. This could be done by a searchbox at the bottom of the folder pane. This searchbox might only appear when starting typing, or when pressing "/", but this is a usability issue that should be carefully evaluated.

The search result should retain the folder hierarchy as described in bug 370737.

Reproducible: Always

Comment 1

[burak]

I think this is related with https://bugzilla.mozilla.org/show_bug.cgi?id=387423

Comment 2

[Mark Banner (:standard8)]

It is already possible to filter the folder view based on a substring search - use the quick search box on the toolbar and enter a string.

How is this bug different to that?

Comment 3

[Jens Müller (:tessarakt)]

I'm not familiar with the names of the parts of the 3-pane-window.

This bug is about filtering the _list_ of folders, not the list of _messages_ _in_ a folder.

Comment 4

[Jens Müller (:tessarakt)]

For those interested: I recycled this idea in connection with the gloda facet, see bug 515779.

Comment 5

[Thomas D. (:thomas8)]

For users with an elaborate folder hierarchy, this sounds like a useful and valid request for enhancement - implement a quick filter for the folder pane.
-> recommending "NEW" for this bug (although it's unlikely to get priority attention)

(In reply to comment #0)
> The search result should retain the folder hierarchy as described in bug
> 370737.

I think the envisioned UI will cause some debate, we've seen battles about hierarchical search results in the firefox product and developers decided that non-hierarchical search results are the order of the day
(hierarchical results: similar to Netscape; look at the bookmarks panel in Firefox without searching for something to picture the idea; searching will flatten the hierarchy)

Comment 8

[Thomas D. (:thomas8)]

In the long run, we need something to filter/search for folders (there's similar requests to filter for FF bookmark folders).

Duplicate bug 516346 has another interesting UI proposal:

> When dealing with lots of folders it's very hard to quickly browse among
> folders...
> 
> There is 4 views in the folders pane : all folders, recent folders, prefered
> folders and unread folders.
> 
> Adding a 5th view "folder finder" with a the ability to search/filter among
> folders.
> 
> It could be a simple text input in wich you can tip a text criteria to filter
> on the fly the folders

Comment 9

[Thomas D. (:thomas8)]

Try this to find duplicate bugs of this:

https://bugzilla.mozilla.org/buglist.cgi?quicksearch=:Thunderbird%20sum:filter,search,find,%22look%20for%22,%22looking%20for%22%20[folder%20-sum:virtual,junk,delet,automatic,%22message%20filter%22,tag,imap,import,%22smart%20folder%22,%22saved%20search%22,rss,message,mov,filters,crash,wrong,remember,compacting,%22Account%20central%22,%22filter%20rules%22%20sev:enh

Remove "%20sev:enh" from the end of that to include unconfirmed bugs that have wrong severity: [some_bug-severity] instead of severity: enhancement

Comment 10

[Wayne Mery (:wsmwk)]

wanted-thunderbird3 flag is obsolete, so clearing

Comment 11

[Michelle]

I'm confused, what is a TB flag & how does that relate to this feature?

Thanks


Michelle

Comment 12

[Jens Müller (:tessarakt)]

Apparently https://addons.mozilla.org/en-us/thunderbird/addon/quicker-filer/ allows something like this ...

Comment 13

[Jens Müller (:tessarakt)]

assign to me, please

Comment 14

[Thomas D. (:thomas8)]

(In reply to Jens Müller from comment #13)
> assign to me, please

Very welcome.
Jens, please ensure to present your ideas for UI/behaviour early enough so we can agree on that and get ui-review, to avoid frustrations when you code up everything and it turns out the UI doesn't fly. E.g. mockup screenshots or an addon to try.

Comment 17

[Jens Müller (:tessarakt)]

Just some quick thoughts:

1. Concerning the "static" filtering behaviour:
 - Treat each element of the tree as a list of strings (the labels of all the ancestors and its own label)
 - The query is a list of strings (search terms) as well
 - Define different kind of matches:
  a) Normal match: At least one search term is a substring of the element's own label (i.e., the last
     of the strings by which we represent it for purposes of the filtering).
  b) Child of match: None of the search terms is a substring of the element's own label,
     but all all substrings of a label of one of the parents.
  c) Parent of match: Neither a) or b), but one of the descendants is a "normal match".
 - Ideas for display:
   - Display c) in a very light gray only, just so that the structure of the tree is still recognizable.
   - By default, display all "normal matches", with some hightlighting applied compared to non-filtered view.
     - An open question is whether to allow collapsing tree nodes containing normal matches.
   - Allow access to children of matches somehow.
     - Note: This would require two different "expanded" states for tree nodes:
       Showing only "normal matches", or showing all children.
     - If we also allow to collapse nodes (see above), there would be three different collapsed/expanded states.
       Quite a strange thing ...

Next thing to consider would be the transitions between different filterings (or from unfiltered to filtered), i.e., the "dynamic" behaviour of the filter. When the user has automatically expanded a tree node, it should not be collapsed automatically ... In addition, changing the focussed/selected element(s) and the scroll position of the tree view have to be considered.

Comment 18

[Mike Conley (:mconley) (:⚙️)]

Needinfo'ing myself so I remember to give feedback here.

Comment 19

[:aceman]

Jens, are you actually working on this?

The feature as filed in comment 0 could be doable, but comment 17 would need quite some work.

Comment 20

[:aceman]

Attachment: WIP patch

So this is should implement the requested functionality from comment 0.
However, instead of somehow filtering the current view on the fly, I have done it as a new folder view (View -> Folders -> Filtered) as that is how I am able to do it.
I have sprinkled the code with some TODOs to indicate what could be done and we should decide if/how to do it. So I'd like your feedback now.

Comment 21

[Thomas D. (:thomas8)]

Aceman, could you add a screenshot including filter, with some result folders which in their original location are a nested subfolder 3rd level or such?

Comment 22

[:aceman]

Attachment: mode "All" - normal hierarchical view

Comment 23

[:aceman]

Attachment: new mode "Filtered" - but no search term yet

Comment 24

[:aceman]

Attachment: new mode "Filtered" - showing folders containing any of the searched word

Comment 25

[:aceman]

(In reply to Thomas D. from comment #21)
> Aceman, could you add a screenshot including filter, with some result
> folders which in their original location are a nested subfolder 3rd level or
> such?

OK, added some screenshots. As the resulting view is flat (no hierarchy) for now, 3rd level folders appear the same way as any others. They can be seen in the "no search terms" image.

Comment 26

[Josiah Bruner [:jsbruner]]

Comment on attachment 8404225 [details] [diff] [review]
WIP patch

Review of attachment 8404225 [details] [diff] [review]:
-----------------------------------------------------------------

Hmm... So I definitely like what this adds to Thunderbird, but I think we may want to approach this from a different direction. But first, let me talk about the current patch.

1) I noticed that although you can filter multiple folders (E.G. Inbox and Drafts), you can only do it by separating with a space, I think we should add support for a comma as well. If I search for "Inbox, Drafts", only "Drafts" shows up. (I think in most cases, simply removing the comma before you filter is enough to add this ability)

2) As a filter mode, I feel this isn't as practical as it could be, and without a search term it looks quite messy, which brings me to my suggestion.

=== Suggestion for new approach ===

Instead of making this a folders mode like the others, I think it would be better to remove the mode from the View > Folders list, and instead use it when the user toggles some kind of "filter" button. In fact, I think we could just use the existing Quick Filter Bar Button. If that filter is checked, bring up the *bar* (not the view mode yet). Then, if the user types something, switch to the filtered mode. When unchecked, go to the previous view mode.

Does that make sense?

Comment 27

[Magnus Melin [:mkmelin]]

A view like this would be useful only in special cases, like if you have a large amount of folders and can't find that one folder you're seaching for. So I agree it shouldn't be special view.

What would make sense to me is if something similar to the find bar would appear on the bottom of the folder pane allowing you to find the folder you want. Involving the QFB will only be confusing though IMHO.

Comment 28

[:aceman]

As long as it uses the folder views code we can discuss how it should be activated.
So how would that find bar under the folder pane appear?

Comment 29

[Magnus Melin [:mkmelin]]

I suppose Ctrl+F with focus on the folder pane? (does nothing at the momment)

Comment 30

[Thomas D. (:thomas8)]

I haven't found time for thorough analysis yet, but here's some feedback (partly inspired by what others have said):

a) Ctrl+F for bringing up folder filter sounds alright to me; it's probably best to  have a key which is distinkt from QFB (I don't have an idea yet where to put a visual trigger like search button).

b) I'm not sure why the folder filter should be at the bottom of folder pane? Not entirely against that, but we tend to read things top to bottom, and for few folders that filter box might be quite far away from folders shown in folder pane, more so after reducing the folder list with the filter (so ux-feedback / results for your folder filter move away from your filter box as you type, which feels wrong). Also, it's easier to forget about an active filter when it's at the bottom. Plus, TAB-sequence to proceed from folder filter into results list will also be more natural just using TAB from folder filter to move down into the folder list.

c) Every folder should indicate somehow which account it belongs to. (I have no opinion yet on how to solve the compact vs. grouped-by-account issue seen on bug for favourite folders, but we'll run into the same thing here! Maybe a switch in the UI to change from compact to account-grouped, similar to how tiny thread-button groups threads in msg-pane?).

d) ESC in folder filter or on folder list (only) should remove filter

e) Menu entry (at least) next to QFB in View > Toolbars > Folder Filter

f) OT: In the future, we might want to explore the idea of making Folder Filter bar more powerful in analogy to quick filter bar: show only folders with unread msgs, only folders bigger than x mb, only folders with msgs tagged so and so... (just a thought)

g) I agree with Josiah et alii that this shouldn't be realized as a separate Folder View Mode; it's just a volatile filter (same UX principles as QFB).

Comment 31

[Magnus Melin [:mkmelin]]

(In reply to Thomas D. from comment #30)
> b) I'm not sure why the folder filter should be at the bottom of folder
> pane? Not entirely against that, but we tend to read things top to bottom,
> and for few folders that filter box might be quite far away from folders
> shown in folder pane, more so after reducing the folder list with the filter
> (so ux-feedback / results for your folder filter move away from your filter
> box as you type, which feels wrong). Also, it's easier to forget about an
> active filter when it's at the bottom. Plus, TAB-sequence to proceed from
> folder filter into results list will also be more natural just using TAB
> from folder filter to move down into the folder list.

The normal find-replace bar is also at the bottom.
I think one thing should be clear here: this isn't a view people would "live in", its just for special cases where you have lots of folder and want to find the one you need. We shouldn't design it like something one could live in when it reasonable is not. Everyone kind of needs more than a couple of folders... and if it's sticky people would miss new mail and so forth. 

> c) Every folder should indicate somehow which account it belongs to. (I have
> no opinion yet on how to solve the compact vs. grouped-by-account issue seen
> on bug for favourite folders, but we'll run into the same thing here! Maybe
> a switch in the UI to change from compact to account-grouped, similar to how
> tiny thread-button groups threads in msg-pane?).

It should IMO just filter the view you're in. Flat or not would bepend on what view you're in.

> e) Menu entry (at least) next to QFB in View > Toolbars > Folder Filter

This woudln't be a toolbar. I'd make it as much as a find bar as possible.

Comment 32

[:aceman]

When a folder is selected but also a message in that folder, Ctrl+F opens the findbar inside the message. Any other suggestions?

Comment 33

[Magnus Melin [:mkmelin]]

"Find folder..." in the folder pane context menu?

Comment 34

[:aceman]

And then what? It exposes a searchbox (top or bottom). User types a term and the folder in the current view (mode) are filtered (i'd just hide/collapse them instead of remove from the tree). How does the user exit/remove the filter and hides the searchbox?

Comment 35

[Michelle]

If this has already been mentioned I apologize, but I need it not just for finding the messages in the folder, but finding the entire path that leads upto the folder. What if I want to delete the folder?

I can't if all your find feature will show me are the e-mails in it.

Thanks & happy to see people are interested in fixing this issue. If you only knew how many folders I have LOL, you'd understand.


Michelle

Comment 36

[Thomas D. (:thomas8)]

(In reply to :aceman from comment #32)
> When a folder is selected but also a message in that folder, Ctrl+F opens
> the findbar inside the message. Any other suggestions?

I'd recommend "Focus respect" for that scenario.

When folder pane has focus, Ctrl+F should search folder pane.
When message list (or msg) has focus, Ctrl+F should search the message. Easy.
Think about it, how likely is it that users want to search just that single random msg which happens to be shown after you focus the folder? To view any other msg, they'll have to click inside the msg list before they can move on, i.e. for all msgs except the first one shown upon folder select, Ctrl+F will always act on the msg because that's where the focus will be (in msg list).

Comment 37

[Thomas D. (:thomas8)]

(In reply to Thomas D. from comment #30)
> I haven't found time for thorough analysis yet, but here's some feedback
> (partly inspired by what others have said):
> [...]
> g) I agree with Josiah et alii that this shouldn't be realized as a separate
> Folder View Mode; it's just a volatile filter (same UX principles as QFB).


h) It was suggested somewhere above that searching for "foo bar" or "foo, bar" should be read as an OR search. I'll definitely advise against introducing yet another search logic here. Let's please make search terms parsing consistent with QFB and the new OR-Operator just introduced there:

"foo bar" -> contains "foo" AND contains "bar". This is ux-consisten with QFB, AB, Recipient autocomplete very soon, FF awesome bar, Google search, etc... it's most powerful because you can narrow down your results easily to find the needle in the haystack of folders. That's what's this search is for.

"foo | bar" -> contains "foo" OR contains "bar"

I think commas don't belong into search syntax at all, they might just as well be part of a folder name, and there's no precedent anywhere else in our UI, nor in other similar searches I know of.

Blake Winton also recently wished that we had "|" or operator everywhere (see his comment on the bug where we introduced "|" as OR for quick filter bar.).

Comment 38

[Magnus Melin [:mkmelin]]

(In reply to :aceman from comment #34)
> And then what? It exposes a searchbox (top or bottom). User types a term and
> the folder in the current view (mode) are filtered (i'd just hide/collapse
> them instead of remove from the tree). How does the user exit/remove the
> filter and hides the searchbox?

May require some trying out, but either clicking the "[x]" close box, and maybe also selecting a message should auto-clear it, the the folder pane going back to normal but with the found folder selected. (As now it's found, you're not really likely to search for another one the next thing you do.)

Comment 39

[Thomas D. (:thomas8)]

Comment on attachment 8404225 [details] [diff] [review]
WIP patch

clearing needinfo; see my previous comments

Comment 40

[Jens Müller (:tessarakt)]

Aceman, what feedback exactly should I give on this?

Comment 41

[:aceman]

Tessarakt, it was about trying out that attachment in comment 20. But it seems it is outdated already as other people propose for this feature to work differently (not folder view, but a search field).
I'll look at reimplementing the patch according to the comments.

Comment 42

[realgrouchy]

I needed to find a folder today, and I’m very happy to see that this bug is already here and there is so much interest (albeit inactive for a while). I think the "quicker filer" suggested in Comment 12 will serve as a workaround for me currently.

I agree with the suggestions in Comment 26 that the unfiltered folder view is not necessary, and Thomas D's feedback in Comment 30 about the UI.

In reply to Magnus Melin from Comment 31:
> The normal find-replace bar is also at the bottom.

Yes, but the filters are at the top.

I agree with Michelle’s Comment 35: If I’m having difficulty finding a folder, it is useful to know where that folder sits in the hierarchy so I can then move the folder somewhere where it would be easier to find next time. 

If I select a folder while the view is filtered, it'd be nice if that folder remains selected when the filter is removed; that would require the folders higher up in the hierarchy are expanded when the folder is removed.

In reply to Thomas D.’s Comment 37 (h) regarding how the search string should be parsed, personally I’d prefer that it just look for the string exactly as entered (commas, slashes and all) but I’m not sure what would be most universally useful for people.

Here's my suggestion for how the UI could potentially work:
- Currently the Quick Filter button is greyed out unless you have a folder selected that has messages (which makes sense, since it's designed for filtering within the selected folder). However, TB does remember the state of the Quick Filter button even if a folder is not selected.
- I would suggest that the button always be available, and when it is toggled on, a folder filter bar (like Aceman's) appears at the top of the folder pane
- An [x] button can be added to the folder filter bar so people can close it if they only want to folder messages within a folder. To make it reappear, they would have to toggle the QF button. 

To be honest, I am relieved to see that nobody said the "solution" is to have a simpler folder structure :) In my use case, I inherited some accounts and need to find where the former user filed some things.

Comment 43

[Wayne Mery (:wsmwk)]

Comment on attachment 8404225 [details] [diff] [review]
WIP patch

clear obsolete FB request per comment 41 - back in aceman's hands

Comment 45

[Michael Nüsken]

Hi all,
still, after that many years, I miss such a feature in tb.
Just found: The addon "Filter on Folder Button" [https://addons.thunderbird.net/de/thunderbird/addon/filter-on-folder-button/?src=search].
It does something in this direction, but with some deficiences:

• It is not displayed in the intuitive place, which -in my eyes- would be above the folder list (where you select the folder views All folders/Unread folders/...).
• There is no visual indicator that tells me that filtering is on.
• One needs to actively turn off the filtering. This might be a problem when I forgot that this has been turned on.
  Well, I am not asking to improve the addon. This just reflects my wishlist. :)

Comment 47

[Alan]

The need for this feature has been acknowledged for more than a decade now. Seriously? Please just choose from any of the proposed UIs. E.g.,

• ctrl-F when the Folder Pane has focus (most intuitive)
• search box at the bottom of the folder pane
• search function in the Folder Pane Options
• just start typing when the Folder Pane has focus

Having any one of these now is better than bikeshedding the final choice.

Comment 51

[Michael Nüsken]

(In reply to Mark Banner (:standard8) from comment #2)

It is already possible to filter the folder view based on a substring search

• use the quick search box on the toolbar and enter a string.

How is this bug different to that?

Need to filter for names of folders, not for subject/name of emails.